Field notes · 3 February 2026
Failure signatures worth cataloguing
Not every error deserves a permanent fix. A short catalogue helps teams decide what to automate, what to alert on, and what to ignore.
Streaming applications fail in recurring shapes. Naming those shapes saves time when the same pager fires at 2 a.m. for the third month in a row.
A useful catalogue lists the signature, the first evidence, the usual root cause class, and whether the fix is temporary or permanent. Examples include retry storms after a slow dependency, poison messages that loop a single partition, and producer bursts that exceed consumer concurrency for longer than the recovery window.
We discourage catalogues that grow into encyclopedias. Start with the five signatures that actually woke someone in the last two quarters. Add a sixth only when a new pattern appears twice.
Each entry should mention the consumer group, the approximate volume when it appears, and the owner who can change the handler. Without an owner, the catalogue becomes a museum.
During incident pattern reviews we print the catalogue and mark which signatures still lack a permanent fix. That printout often does more for prioritization than another slide deck.