Body

Summary

Content collections currently support on-disk files. This proposal suggests an API for adding collection entries programmatically.

Background & Motivation

Content collections offer a powerful way to use content in Astro and associate strongly typed metadata with each piece of content via Markdown frontmatter. This makes it easy to build tools that can flexibly render user content based on its structured data. Currently, this is limited to content that lives on disk in the src/content/ directory of a repo.

As we’ve built Starlight we’ve seen users like the general experience of working this way, but several have expressed a desire to add pages from additional sources to Starlight sites. For example:

• pages that are auto-generated from source code using a tool like TypeDoc
• pages that are located elsewhere in a (mono)repo (workaround using symlinks)
• pages that are loaded from a remote API, e.g headless CMS
• pages that are programmatically generated from an alternative data format (e.g. on-disk JSON)

Historically, we’ve solved many of these cases with dynamic routes in a project or injectRoute for an integration. However, for a project like Starlight where frontmatter is used to build site navigation, make layout decisions, etc. those tools place content “out of sight” for the framework. An API for adding entries to a content collection programmatically would support something similar to injectRoute but specifically for structured content and metadata.

Goals

• Support adding content to content collections from other sources such that code consuming a collection can treat it the same way as it would content from files in the repo.

Example

There are lot of fairly complex technical considerations around how to implement this, but I’m currently imagining an integration hook similar to injectRoute() like injectContent(). There may be a question of whether the code should run on dev server start or be evaluated lazily when a collection is used. Showing a callback style here for the latter case.

const integration = {
  name: 'simple-programmatic-content',
  hooks: {
    'astro:config:setup': ({ injectContent }) => {
      injectContent('docs', () => {
        return [
          {
            id: 'injected.md',
            slug: 'injected',
            data: { title: 'Injected page' },
            content: 'This page was **injected** via an integration!', 
          },
        ];
      });
    },
  },
};

Not going to go into too much more detail for now, but imagine the callback could be async for file system or network operations.

During build the callback would need to be evaluated while building static endpoints and the result bundled for SSR, I guess? Don’t know if that’s an acceptable design, but it aligns with the static-build behaviour of content collections currently. Very much open to other suggestions.

This could also take a form closer to injectRoute where instead of an inline callback, the integration provides a path to a module that contains the logic for loading content entries, i.e. injectContent('docs', './get-docs.ts').