Summary

The custom ingestion documentation instructs users to create a blank manifest.json file to trigger ingestion. However, the ADF blob event trigger is configured with ignoreEmptyBlobs: true, which silently drops events for zero-byte blobs. This means a blank manifest.json never triggers the ingestion pipeline.

Root cause

The trigger in hub-eventTrigger.bicep:79 has:

ignoreEmptyBlobs: true

This setting has been present since the trigger module was first created in #100 (April 2023), when it was watching for .csv files. It was carried forward when the trigger was refactored to watch for manifest.json, but the custom ingestion docs were written assuming a blank file would work.

Impact

Users following the docs for custom/multi-cloud ingestion (e.g., AWS, GCP) create an empty manifest.json and the pipeline never runs. There is no error — the event is silently ignored.

Workaround

Write non-empty content to the manifest file, e.g.:

{
  "generatedUtc": "<timestamp>"
}

The content must change on each write to generate a new BlobCreated event.

Credit to @DMarshall-1 for discovering this workaround in #2046.

Suggested fix

Either:

1. Update the docs to specify that manifest.json must contain non-empty content, and provide a template
2. Change ignoreEmptyBlobs to false in the trigger — though this may introduce noise from empty blob artifacts
3. Both: keep ignoreEmptyBlobs: true for safety, update the docs, and provide a sample pipeline snippet that writes the manifest with a timestamp

Related

• FinOps Hub v13 + Power BI (Storage report) do not show AWS costs when parquet file is dropped directly under ingestion/ #2046 — Original issue where this was discovered