Skip to content

Improve resilience of reservation price exports to Blob Storage#2102

Merged
RolandKrummenacher merged 2 commits into
microsoft:devfrom
skmkzyk:skmkzyk/fix-retail-prices-export-resilience
Apr 21, 2026
Merged

Improve resilience of reservation price exports to Blob Storage#2102
RolandKrummenacher merged 2 commits into
microsoft:devfrom
skmkzyk:skmkzyk/fix-retail-prices-export-resilience

Conversation

@skmkzyk

@skmkzyk skmkzyk commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

🛠️ Description

Improves the reliability and efficiency of the retail prices export runbook and documents the change in the changelog:

  1. Retry logic – Added a helper that retries retail prices API requests with exponential backoff before failing the runbook, making transient network errors less likely to cause job failures.
  2. Streaming CSV export – Each API response page is now written directly to CSV as it arrives instead of accumulating the full dataset in memory, significantly reducing memory pressure on large exports.
  3. Progress logging – Periodic progress messages are logged during long exports so operators can confirm the runbook is still making forward progress.
  4. Empty result guard – The runbook now fails explicitly with a clear error when the retail prices API returns no records for the configured filter, avoiding silent no-op runs.
  5. Changelog – Added the release note entry under docs-mslearn/toolkit/changelog.md in Unreleased > Optimization engine v14.

Fixes #

📷 Screenshots

N/A – no UI changes.

📋 Checklist

🔬 How did you test this change?

  • 🤏 Lint tests
  • 🤞 PS -WhatIf / az validate
  • 👍 Manually deployed + verified
  • 💪 Unit tests
  • 🙌 Integration tests

Executed in PowerShell with Pester 5.7.1:

  1. Invoke-Pester -Path ./src/powershell/Tests/Unit/* -Output Detailed → 120 passed, 8 failed, 4 skipped
  2. Invoke-Pester -Path @('./src/powershell/Tests/Integration/Toolkit.Tests.ps1','./src/powershell/Tests/Integration/New-Directory.Tests.ps1') -Output Detailed → 1 passed, 1 failed

Integration coverage was intentionally limited to non-deploying tests. The following Azure-deploying suites were not run:

  • src/powershell/Tests/Integration/Hubs.Tests.ps1
  • src/powershell/Tests/Integration/CostExports.Tests.ps1

📦 Deploy to test?

  • Hubs + ADX (managed)
  • Hubs + Fabric (manual) — URI:
  • Hubs (manual)
  • Hubs (no data)
  • Workbooks
  • Alerts

🙋‍♀️ Do any of the following that apply?

  • 🚨 This is a breaking change.
  • 🤏 The change is less than 20 lines of code.

📑 Did you update docs/changelog.md?

  • ✅ Updated changelog (required for dev PRs)
  • ➡️ Will add log in a future PR (feature branch PRs only)
  • ❎ Log not needed (small/internal change)

📖 Did you update documentation?

  • ✅ Public docs in docs (required for dev)
  • ✅ Public docs in docs-mslearn (required for dev)
  • ✅ Internal dev docs in src (required for dev)
  • ➡️ Will add docs in a future PR (feature branch PRs only)
  • ❎ Docs not needed (small/internal change)

skmkzyk and others added 2 commits April 21, 2026 15:22
Add a helper to retry retail prices API requests before failing the runbook.

Stream each API page directly to CSV instead of collecting the full dataset in memory, and add periodic progress logging during long exports.

This also fails explicitly when the retail prices API returns no records for the configured filter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@skmkzyk

skmkzyk commented Apr 21, 2026

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree

@helderpinto helderpinto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for the contribution, @skmkzyk. The runbook resiliency is definitely better and troubleshooting is easier now.

@helderpinto helderpinto added the Tool: Optimization Engine Azure Optimization Engine label Apr 21, 2026
@RolandKrummenacher RolandKrummenacher merged commit e23de53 into microsoft:dev Apr 21, 2026
2 checks passed
@skmkzyk skmkzyk deleted the skmkzyk/fix-retail-prices-export-resilience branch April 21, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs: Review 👀 PR that is ready to be reviewed Tool: Optimization Engine Azure Optimization Engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants