Batch systems frequently fail silently — records are skipped, duplicated, or lost during retries, partial failures, or downstream errors.
This repository demonstrates architectural patterns that ensure:
✅ Every input record is accounted for
✅ Outputs can be reconciled with inputs
✅ Failures are observable and recoverable
✅ Processing is verifiable and auditable
The examples illustrate practical techniques for building lossless batch pipelines.
(originally suggested by ChatGPT, as shown in ai-history folder)
batch-lineage-patterns/ │ ├── README.md ├── ai-history/ (saved conversations to document techniques) ├── docs/ │ ├── concepts.md │ ├── lineage-model.md │ └── failure-modes.md │ ├── src │ └── diagrams/ ├──
The src directory now contains a .NET 10 solution named batch-lineage-patterns
with a console application project filehelpers-examples. The command‑line app
references the FileHelpers NuGet
package (latest at time of creation, v3.5.2) to demonstrate parsing/exporting
fixed-width or delimited files as part of batch-processing lineage patterns.
You can explore the C# code under src/filehelpers-examples and build/run it:
dotnet build src\filehelpers-examples\filehelpers-examples.csproj
dotnet run --project src\filehelpers-examples\filehelpers-examples.csproj