Checkmate Horizon Scanner

// the problem

Checkmate SG's core product is reactive: a user reports something suspicious, and the platform analyses it. That's valuable. But scams and misinformation campaigns don't wait to be reported — they propagate in the hours before enough people have noticed to submit a report. The team needed a way to see what was emerging, not just what had already arrived.

// why this matters

Early warning changes the response. A scam trend identified on Tuesday can be flagged to users on Wednesday. The same signal identified on Saturday — after it's already reached thousands of people via WhatsApp — is damage limitation, not prevention. The horizon scanner was built to close that gap.

// how it works

An AI-powered monitoring tool that ingests a defined set of sources on a set schedule, extracts and structures signals that match configurable criteria (topic patterns, source credibility, language markers), and surfaces them to the Checkmate team in a reviewable format. Human analysts review the output and decide what to escalate. The AI handles the volume; the humans handle the judgment.

// trade-offs i made

The hardest design decision was how to handle the null state — what to show when the tool finds nothing. This sounds trivial. It isn't. There are three meaningfully different versions of 'nothing found':

Mode 1 — Hallucinated confidence: the model invents a plausible result rather than admit it found nothing. Fix: explicit instruction to return empty rather than fabricate, plus source URL validation.

Mode 2 — Structural null: the model found something real, but a downstream failure (in our case, JSON output exceeding the token budget, truncating mid-response, causing the parser to fail silently) meant the result never reached the interface. The screen showed empty. The model wasn't. Fix: a raw response viewer to make the null state auditable.

Mode 3 — True null: there genuinely is nothing this cycle. Valid and useful — but only trustworthy once you've ruled out Mode 2.

The tension: Mode 1 (fabrication) and Mode 2 (silent structural failure) erode trust in opposite directions. A tool that invents results trains you to ignore it. A tool that goes empty too often trains you to distrust the empty state. Distinguishing them cleanly was the core reliability challenge.

// what i learned

Silent failures are the most dangerous failure mode in AI tooling — not because they're hard to fix, but because they're hard to notice. The model produces output. The pipeline processes it. The UI renders something. Nothing throws an error. And somewhere in that chain, the signal got lost. Building auditability into the null state — making 'nothing found' verifiable rather than opaque — is a design principle I now apply to every AI tool I build.

// what's next

Configurable alert thresholds, so the team can tune sensitivity per topic category. Integration with the proactive WhatsApp alerts feature currently in PRD. And better retrospective tooling — a way to look back at what the scanner flagged two weeks ago and compare it to what actually spread, so the detection criteria can be improved over time.