Skip to content

msexports_ExecuteETL pipeline fails for empty files #1535

Description

@NicholasBrand

🐛 Problem

ADF pipeline "msexports_ExecuteETL" failing with the following error when processing Reservation Transaction information. (I have obscured the storage account name, billing account and billing profile from the below.
_Operation on target Get Existing Parquet Files failed: ADLS Gen2 operation failed for: Operation returned an invalid status code 'NotFound'. Account: 'STORAGEACCOUNTNAMEHERE'. FileSystem: 'ingestion'. Path: 'Transactions/2025/04/providers/microsoft.billing/billingaccounts/BILLINGACCOUNTID/billingprofiles/BILLINGPROFILEID'. ErrorCode: 'PathNotFound'. Message: 'The specified path does not exist.'. RequestId: '0ce578c0-101f-0088-5675-b4721f000000'. TimeStamp: 'Wed, 23 Apr 2025 17:27:37 GMT'._

The exported reservation transaction CSV had no records (just the column headers).
No directory structure created in ingestion folder for reservation transactions.
This is with an MCA agreement.

👣 Repro steps

  1. Export reservation transaction data to storage account.
  2. Review ADF pipeline monitoring.
  3. msexports_ExecuteETL will show an error as above.
  4. When looking at the ingestion container no transaction directory is created.
  5. Reservation transaction csv is still in the msexports container.

🤔 Expected

When transaction data is exported into the msexports container ADF pipeline should run without errors.

Metadata

Metadata

Assignees

Type

No fields configured for Bug.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions