Why Flashpoint classifies with rules, not an LLM

The decision: Flashpoint's event classifier is a deterministic rule engine, not a model. Incoming reports get normalized, then matched against explicit rules with explicit precedence. Every classification can be traced to the line that produced it.

I prototyped the LLM version first. It was better at weird phrasing and worse at everything else that matters here. Same input, different day, different label. No way to write a regression test that means anything. And when it was wrong, it was wrong confidently, which in a monitoring system is the worst failure mode available.

The tradeoff is real: rules miss paraphrases a model would catch. I accept the misses because they are legible. When a report slips through unclassified, it lands in a review queue and usually becomes a new rule in about two minutes. The test suite pins every rule with real examples, so a change that shifts old classifications fails CI before it reaches the Pi.

The thing that broke: an early rule matched a substring so generic that a sports riot in one feed classified as three different event types at once. That incident is now test case number one.

The LLM is not banned from the system. It normalizes source formatting and writes one-line cluster summaries, both off the critical path. Judgment stays in code I can read.