fix: prevent O(n²) memory allocation in content lookup map generation by ixchio · Pull Request #14973 · withastro/astro

ixchio · 2025-12-06T14:42:59Z

Changes

Replaced O(n²) object spread pattern with direct mutation in generateLookupMap
For each content/data entry, the code was creating new objects via {...lookupMap[collection]?.entries, [key]: value}
With 10,000+ entries, this caused ~50 million string copies (n × n/2)
Now uses nullish coalescing (??=) and direct property assignment - O(n) complexity

Before:

lookupMap[collection] = {
  type: 'content',
  entries: {
    ...lookupMap[collection]?.entries, // ← Creates new object each time
    [slug]: rootRelativePath(root, filePath),
  },
};

After

lookupMap[collection] ??= { type: 'content', entries: {} };
lookupMap[collection].entries[slug] = rootRelativePath(root, filePath);

Testing
All 1326 existing tests pass (3 unrelated failures: clipboard + network flakes)
All content-related tests pass: Content Collections, references, render(), Intellisense

Docs
No docs changes needed - this is an internal performance optimization with no API changes.

changeset-bot · 2025-12-06T14:43:04Z

🦋 Changeset detected

Latest commit: b90634a

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

ematipico · 2025-12-06T14:47:34Z

I'm going to create a preview release so we can test if this change will provide benefits

pkg-pr-new · 2025-12-06T14:50:01Z

npm i https://pkg.pr.new/astro@14973

npm i https://pkg.pr.new/@astrojs/svelte@14973

npm i https://pkg.pr.new/@astrojs/markdown-remark@14973

commit: b90634a

codspeed-hq · 2025-12-06T14:55:36Z

CodSpeed Performance Report

Merging #14973 will not alter performance

_{Comparing amankumarpandeyin:fix/lookup-map-oom (b90634a) with main (ef53716)¹}

Summary

✅ 6 untouched

No successful run was found on main (4264a36) during the generation of this report, so ef53716 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

matthewp · 2025-12-06T16:39:45Z

Wonderful @amankumarpandeyin thank you! Do you have any anecdotal data, did this improve your build performance for example?

ixchio · 2025-12-06T16:45:13Z

Wonderful @amankumarpandeyin thank you! Do you have any anecdotal data, did this improve your build performance for example?

@matthewp thanks! i was experiencing OOM errors with large content collections (10k+ entries)... after this change builds complete without memory issues... the fix eliminates the quadratic object copying that was causing 50M string copies for that scale

This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [astro](https://astro.build) ([source](https://github.com/withastro/astro/tree/HEAD/packages/astro)) | [`5.16.4` -> `5.16.5`](https://renovatebot.com/diffs/npm/astro/5.16.4/5.16.5) | ![age](https://developer.mend.io/api/mc/badges/age/npm/astro/5.16.5?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/astro/5.16.4/5.16.5?slim=true) | --- ### Release Notes <details> <summary>withastro/astro (astro)</summary> ### [`v5.16.5`](https://github.com/withastro/astro/blob/HEAD/packages/astro/CHANGELOG.md#5165) [Compare Source](https://github.com/withastro/astro/compare/astro@5.16.4...astro@5.16.5) ##### Patch Changes - [#14985](withastro/astro#14985) [`c016f10`](withastro/astro@c016f10) Thanks [@florian-lefebvre](https://github.com/florian-lefebvre)! - Fixes a case where JSDoc annotations wouldn't show for fonts related APIs in the Astro config - [#14973](withastro/astro#14973) [`ed7cc2f`](withastro/astro@ed7cc2f) Thanks [@amankumarpandeyin](https://github.com/amankumarpandeyin)! - Fixes performance regression and OOM errors when building medium-sized blogs with many content entries. Replaced O(n²) object spread pattern with direct mutation in `generateLookupMap`. - [#14958](withastro/astro#14958) [`70eb542`](withastro/astro@70eb542) Thanks [@ascorbic](https://github.com/ascorbic)! - Gives a helpful error message if a user sets `output: "hybrid"` in their Astro config. The option was removed in Astro 5, but lots of content online still references it, and LLMs often suggest it. It's not always clear that the replacement is `output: "static"`, rather than `output: "server"`. This change adds a helpful error message to guide humans and robots. - [#14901](withastro/astro#14901) [`ef53716`](withastro/astro@ef53716) Thanks [@Darknab](https://github.com/Darknab)! - Updates the `glob()` loader to log a warning when duplicated IDs are detected - Updated dependencies \[[`d8305f8`](withastro/astro@d8305f8)]: - [@astrojs/markdown-remark](https://github.com/astrojs/markdown-remark)@6.3.10 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Co-authored-by: Renovate Bot <renovate@zarantonello.dev> Co-committed-by: Renovate Bot <renovate@zarantonello.dev>

ixchio added 2 commits December 6, 2025 20:00

refactor: simplify lookup map entry assignment logic

f076977

chore: add changeset for lookup map OOM fix

b90634a

github-actions bot added the pkg: astro Related to the core `astro` package (scope) label Dec 6, 2025

ematipico added the pr preview Apply this label to a PR to generate a preview release label Dec 6, 2025

matthewp approved these changes Dec 9, 2025

View reviewed changes

matthewp merged commit ed7cc2f into withastro:main Dec 9, 2025
28 checks passed

astrobot-houston mentioned this pull request Dec 9, 2025

[ci] release #14959

Merged

This was referenced Dec 10, 2025

chore(deps): update astro monorepo rikuson/rikuson#203

Merged

Update dependency astro to v5.16.5 RevolutionVA/757-community-site-public#164

Merged

github-actions bot mentioned this pull request Dec 19, 2025

fix(deps): update dependency astro to v5.16.5 lacolaco/blog.lacolaco.net#1229

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: prevent O(n²) memory allocation in content lookup map generation#14973

fix: prevent O(n²) memory allocation in content lookup map generation#14973
matthewp merged 2 commits intowithastro:mainfrom
ixchio:fix/lookup-map-oom

ixchio commented Dec 6, 2025

Uh oh!

changeset-bot bot commented Dec 6, 2025

Uh oh!

ematipico commented Dec 6, 2025 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Dec 6, 2025

Uh oh!

codspeed-hq bot commented Dec 6, 2025

Uh oh!

matthewp commented Dec 6, 2025

Uh oh!

ixchio commented Dec 6, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

ixchio commented Dec 6, 2025

Changes

Uh oh!

changeset-bot bot commented Dec 6, 2025

🦋 Changeset detected

Uh oh!

ematipico commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new bot commented Dec 6, 2025

Uh oh!

codspeed-hq bot commented Dec 6, 2025

CodSpeed Performance Report

Merging #14973 will not alter performance

Summary

Footnotes

Uh oh!

matthewp commented Dec 6, 2025

Uh oh!

ixchio commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ematipico commented Dec 6, 2025 •

edited

Loading

ixchio commented Dec 6, 2025 •

edited

Loading