Debug Notes

Fail Fast in Ingestion, Rerun Narrowly After Fix

Fail-hard input checks and scoped reruns work best as one design.

Details are generalized and sanitized; numbers are rounded where appropriate to preserve confidentiality while keeping the engineering lessons accurate.

Problem

Permissive ingestion can hide missing required inputs. A run may finish while downstream tables are built from incomplete data.

Approach

Required input failures should stop the workflow immediately. Optional paths remained skippable, but required contracts were enforced.

To make this practical in operations, pair strict failure checks with narrow reruns so recovery does not require broad historical rebuilds.

Reusable pattern

Required input failures should stop downstream work.
Error handling should separate optional skip from required failure.
Recovery scope should stay narrow so fail-fast behavior is usable.