Randomness Scoring

Picket native scans attach a deterministic p(random) assessment to each finding before redaction and cache storage. Strict Gitleaks-compatible scans do not run the model and do not apply native randomness thresholds.

p(random) estimates whether a token-like byte segment resembles output sampled uniformly from a token alphabet. It is not the probability that a value is a credential, is active, or is safe to suppress. Provider structure, checksums, context, dependent rules, and offline or live validation remain separate signals.

The score is not calibrated to the prevalence of generated tokens in production repositories. The deterministic calibration corpus is balanced equally between its synthetic classes, so use the score for relative ranking and rule-specific thresholds rather than as a population probability.

Model Contract

The initial model identifier is picket-random-v1. The scorer:

1. Selects the longest ASCII letter-or-digit segment within the first 4,096 bytes and caps feature analysis at 512 bytes.
2. Infers a hexadecimal, alphanumeric, or byte alphabet.
3. Extracts normalized Shannon entropy, expected distinctness, adjacent-transition diversity, longest-run ratio, sequential-pair ratio, short-period repetition, common source/text bigrams, character-class balance, bounded Base64 printable-text evidence, placeholder evidence, and normalized sample length.
4. Applies fixed logistic coefficients generated from the committed deterministic calibration app.
5. Rounds every feature and score to six decimal places for stable cross-platform reports.

The scorer never uses network access, provider credentials, repository history, environment values, or nondeterministic process state. Base64 inspection is limited to the already selected 512-byte sample, uses a fixed stack buffer, and clears decoded bytes after feature extraction.

Calibration Corpus

scripts/Calibrate-RandomnessModel.cs is the reproducible corpus generator and trainer. It creates a balanced training set with 1,024 samples per class and an independent balanced holdout set with 256 samples per class.

The random class uses SHA-256-derived deterministic streams sampled into lower-hexadecimal, upper-hexadecimal, and mixed alphanumeric alphabets at lengths from 16 through 96. It includes unprefixed values and provider-shaped prefixes such as ghp_, github_pat_, sgp_, and sk_live_.

The structured class combines benign source/configuration vocabulary into identifiers, version strings, repeated runs, ascending sequences, periodic strings, year-suffixed values, internal host names, and Base64-encoded human text. It contains no captured production credential material.

The checked-in picket-random-v1 fit has these independent holdout results:

Metric	Value
Accuracy at 0.5	0.998047
Brier score	0.014483
Precision at likely-random	1.000000
Recall at likely-random	0.937500
Recall at likely-structured	0.906250

Reproduce the fit and verify the checked-in coefficients with:

dotnet run --file ./scripts/Calibrate-RandomnessModel.cs -- --verify

Changing the corpus, features, coefficients, sampling rules, quantization, or thresholds requires a new model identifier and updated fixtures.

Thresholds

Scores use stable inclusive classifications:

• 0.0 through 0.2: likely-structured
• greater than 0.2 and less than 0.8: indeterminate
• 0.8 through 1.0: likely-random

The native rule field randomnessThreshold accepts 0.0 through 1.0. Zero disables score filtering. A positive threshold suppresses that rule’s finding only when the score is lower than the configured value. Rules without a threshold still receive report metadata, so rule authors can tune against real reviewed findings before enabling suppression.

Do not add a threshold to fixed-format or checksum-verifiable rules merely because the field exists. Random digests, generated identifiers, minified symbols, and compressed or encrypted content can score as random without being credentials. Conversely, human-created passwords can be real credentials while scoring as structured.

Report Fields

Lossless native JSON, JSON Lines, and SARIF findings include a randomness object with:

• model, score, and classification,
• sample offset, sample length, and inferred alphabet,
• every normalized feature used by the model,
• stable explanatory signal identifiers.

Native CSV and TOON expose the same values as stable columns. HTML presents the score, classification, model, and material signals. JUnit embeds the native JSON finding. Summary-only third-party formats keep their constrained schemas.

Randomness metadata contains no raw secret bytes, but its features describe structural properties derived from the original secret. Any non-zero report redaction removes the complete assessment. Secret-hash-only scan-cache rows preserve the assessment only as authenticated ciphertext and do not persist match, secret, or line evidence.

The scan-cache identity includes the model version. Report input carrying an older model is rescored when raw evidence is available; when evidence is redacted or absent, Picket preserves the older assessment for visibility but never uses it for threshold suppression.