The Nearest Blob Is The Wrong Blob
A tracker that remembers only the last location looks excellent until a distractor crosses the target. Then recency becomes an identity-swap machine.
distractrack generates a 42-frame crossing sequence with measurement noise and a deliberate occlusion jump. Both trackers receive the same candidates.
Memory Needs Identity And Motion
The baseline selects the candidate nearest its last position. The distractor-aware tracker predicts motion and penalizes candidates that disagree with the target identity.
The implementation is intentionally coordinate-level. It isolates memory policy from segmentation quality, which is the exact question the benchmark is trying to answer.
Update: "penalizes candidates that disagree with the target identity" turned out to be uncomfortably literal — the formula read the actual ground-truth identity label, not anything inferred from motion. Fixed with a genuine motion-only mechanism. Full writeup below.
The Number
Recency memory tracks 73.8% of frames correctly after the crossing. Distractor-aware memory tracks all frames, a 26.2-point gain.
The generated timeline is committed with the benchmark result. One test reproduces the identity crossing across the CI matrix.
The Motion Model Never Actually Ran
The "distractor-aware" score was distance-to-prediction + 3 × (1 − is_target), where is_target is the ground-truth label handed to the scoring function directly. Checked across 60 seeds: accuracy sits at exactly 1.0 every time, zero variance, because a flat +3 penalty on the distractor swamps any realistic distance difference — the velocity term never gets a chance to matter.
Confirmed it more directly: with only the label term removed, the tracker picks the identical candidate to plain nearest-to-last-position recency on 41–42 of every 42 frames, on every seed tested. Mean gain over naive: exactly 0.0. The motion-extrapolation mechanism the whole repo is built around was contributing nothing; the entire 26.2-point gain was the label.
The fix keeps the velocity model but adds something genuinely label-free: when the two candidates sit close together (an ambiguous crossing, where either pick is defensible), it coasts on the prior velocity estimate instead of letting a noisy pick corrupt it. That recovers a real, modest, never-negative gain — about 1.7–2.8 points across 90 tuning and holdout seeds — because after the crossing the distractor moves in exact lockstep with the target, genuinely indistinguishable from motion alone. The original module is untouched and the published 26.2-point number still reproduces exactly.
Research Basis
Inspired by CVPR 2025's distractor-aware SAM2 memory paper. The portfolio number above comes from this repository's own controlled benchmark, not from the paper.