Equivalent DNA Should Not Change The Answer
Reverse complements and small window shifts preserve the local sequence event, yet a classifier can react to orientation or absolute position instead of biology.
strandshift enumerates both strands and configurable offsets, evaluates one prediction function, and turns disagreement into a release gate.
The Demo
Nine offsets on two strands create 18 equivalent views. Their predictions span 0.3273 around a 0.8182 consensus, so the fixture correctly fails its invariance budget.
This audits a supplied predictor. It does not establish biological validity, calibration, or robustness to unseen sequence families.
The Reverse-Complement Check Compared The Wrong Motif
Reverse-complementing a sequence reverse-complements everything inside it, including any motif occurrence. The same underlying biology appears as the reverse complement of the motif on the flipped strand, not as the original spelling. But the reverse-complement transforms were being scored against the unmodified motif string — checking whether the literal forward-strand spelling happened to reappear by chance after flipping (it structurally almost never does), not whether the model actually recognized the same signal on the other strand.
Verified directly: scoring the reverse-complemented sequence against the reverse-complemented motif gives a perfect match at zero shift, exactly mirroring the forward strand's own baseline — confirming the underlying signal genuinely is present and recognizable on both strands. Scored against the wrong motif, that same comparison landed at 0.40, purely an artifact of comparing against the wrong reference.
Fixed by scoring each strand against its correctly oriented motif. The published instability dropped by roughly half — prediction range from 0.7309 to 0.3273, max deviation from 0.4145 to 0.1818 — but the audit still correctly fails. The demo model's deliberate position sensitivity, favoring matches near the center of the window, is a real and separate source of brittleness the bug fix doesn't touch.
Research Basis
Nucleotide Transformer highlights the growing use of foundation models across diverse genomic prediction tasks, making invariance checks a practical deployment requirement.
Read the Nucleotide Transformer.