DriftScope

A stationarity audit framework for discrete streams that are uniform by design — with calibrated detector hallucination rates.

Executive summary · scientific research framework · solo build

The problem

Anomaly detectors are easy to fool: point one at random data long enough and it "finds" something. The honest question is not "did we find an anomaly?" but "does the detector fire on a real signal and stay silent where there is none?" DriftScope answers it on a stream with a known ground truth — EuroJackpot, whose euro-number pool changed rules twice (2014, 2022) while the main 1–50 pool never did: a built-in positive/negative control.

The result — 958 real draws (2012–2026)

Positive control — BOCPD on euron	FIRES ✓ — change-points at 2014-11-28 and 2022-03-29, the first draws reflecting each rule change (DoD-1b).
Negative control — main 1–50, 3 pillars, per regime	R1 0/3 · R2 1/3 · R3 0/3. The lone R2 single-pillar flag is classified "requires power context" — not a finding (≈14% chance one of three regimes flags at α=0.05).
Family B FDR — per-number, Benjamini-Yekutieli, 150 hyp.	0 / 150 rejections.
Honest watchlist (DoD-5)	None — an honest null, not an empty list or an extrapolation.

The framework confirms the known signal and refuses to hallucinate one where there is none.

Why the null is credible

Three non-redundant pillars. BOCPD (temporal), MMD (distributional), co-occurrence (joint pair structure). pair_corr is visible only to co-occurrence — the families provably don't overlap, so a clean verdict isn't one blind detector agreeing with itself.
Pre-registration. Every choice (statistics, nulls, thresholds, effect grids) frozen before seeing results; each revision carries a revision_reason split clean vs data-informed.
Calibrated nulls. Each detector's false-positive rate validated ≈ α on an honest uniform null (DoD-2); BOCPD threshold calibrated per field (FPR ≈ 0.05).

Sensitivity proof — same battery on PRNGs

A null is only worth trusting if the instrument is sensitive. The exact same battery on streams with known truth:

Stream	Class	Verdict
MT19937 · Xorshift64	good	clear
ChaCha20 · AES-CTR-DRBG	crypto	clear
MT19937 + marginal bias	defect	FLAG — narrow
MT19937 + period-truncation	defect	FLAG — broad
EuroJackpot main 1–50	real	clear

Defects flag; two crypto-grade RNGs and real data clear. Which pillars fire reveals the defect kind — a marginal bias trips Family B + MMD narrowly; a short period freezes the whole distribution and trips all three.

What it demonstrates

Statistical rigor under adversarial self-scrutiny — family-aware FDR, calibrated FPR, a negative control treated as seriously as the positive one.
Reproducibility engineering — pure-function detectors, content-seeded RNG, a deterministic SHA-256 manifest (cold-machine re-run is bit-identical, DoD-6).
Performance & clean architecture — Numba-JIT permutation hot loops (CPU-only), a detector-agnostic pipeline (a PRNG stream is just a different DrawRecord source — proven on a second independent real game, Multi Multi 20/80, which the unchanged battery audited and cleared: no rule change to find).
Intellectual honesty as a feature — the deliverable is a calibrated "we found nothing", defended — not a manufactured finding.