Open-data integration tests occasionally flake on transient timeouts. Which failure-handling approach is appropriate?

Failure handling may include a bounded number of retries for known-flaky steps while still failing the pipeline on persistent errors. Open-data integration tests that occasionally time out transiently should retry within limits rather than treating every flake as success or disabling the suite. The correct approach works because civic open-data pipelines often touch networked services with brief blips; bounded retries preserve signal quality by absorbing rare transients without masking real breaks that keep failing after the cap.

Disabling the entire test suite permanently after the first flake fails because one transient should not delete coverage for datasets citizens and agencies rely on. Retrying failed jobs forever with no cap until the agent runs out of budget fails conceptually: unbounded retries hide systemic outages, burn municipal CI spend, and delay definitive failure notification. Treating every flake as a successful pass without re-running or investigating fails because auto-passing converts red into fake green and lets broken validators ship.

Exam caveat: retries are for transient faults you have characterized—not for asserting away deterministic bugs; if the same test flakes often, fix the test isolation or the dependency rather than raising the retry ceiling indefinitely. Operational check: configure a small retry limit on the known-flaky open-data step, inject a persistent failure and confirm the job still fails after exhausting retries, inject a one-shot timeout and confirm a retry can pass, and track flake rates so chronic offenders get engineering attention.