feat: shallow clone git repository sources by wale · Pull Request #3542 · nuxt/content

wale · 2025-09-16T08:22:53Z

🔗 Linked issue

Reopens and closes #3338, continues #3531

❓ Type of change

📖 Documentation (updates to the documentation or readme)
🐞 Bug fix (a non-breaking change that fixes an issue)
👌 Enhancement (improving an existing functionality like performance)
✨ New feature (a non-breaking change that adds functionality)
⚠️ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description

Creates the option of shallow cloning (--depth=1 --single-branch) a Git repository, by enabling a boolean option cloneRepository.

export default defineContentConfig({
  collections: {
    docs: defineCollection({
      type: 'page',
      source: {
        repository: 'https://github.com/nuxt/content',
        include: 'docs/content/**',
      },
    })
  }
})

This change also adds support for cloning repositories with Git refs (tags and branches only), with another option in the source definition; alongside support for usernames with authentication tokens, as an attempt to support Forgejo-based (and other odd) providers.

export default defineContentConfig({
  collections: {
    docs: defineCollection({
      type: 'page',
      source: {
        repository: {
          url: 'https://github.com/nuxt/content',
          branch: 'v1',
          // tag: 'fake-tag',
          auth: {
            username: 'username',
            password: 'password'
            // token: 'ghp_TOKENHERE'
          }
        },
        include: 'docs/content/**',
      },
    })
  }
})

The change was required as I use a non-Github/BitBucket repository for external Markdown files, and the current solution of submodules + git-sparse-checkout was no longer tenable.

The Git source also stores .content.cache.json differently to Github/Bitbucket sources, like so:

{
  "url": "<repo-url>",
  "hash": "<git-commit-hash>",
  "createdAt": "2025-09-16T08:07:28.525Z",
  "updatedAt": "2025-09-16T08:07:28.525Z"
}

where the Git commit hash is referencing git ls-remote.

Uses isomorphic-git to avoid shelling out to the git CLI. Needs testing on bun environments.

📝 Checklist

I have linked an issue or discussion.
I have updated the documentation accordingly.

vercel · 2025-09-16T08:22:58Z

@wale is attempting to deploy a commit to the NuxtLabs Team on Vercel.

A member of the Team first needs to authorize it.

pkg-pr-new · 2025-09-16T08:25:07Z

npm i https://pkg.pr.new/@nuxt/content@3542

commit: abfc069

farnabaz · 2025-09-18T09:18:17Z

Thanks, @wale.
I'm wondering—can’t we use the same login (with isomorphic-git) for GitHub and Bitbucket?
Ideally, removing cloneRepository and ref and add support for object syntax for repository

repository: string | { url,  branch } | { url, tag }

Also it would be awesome if we can compare performance of old logic and new one

wale · 2025-09-18T12:52:27Z

Thanks, @wale. I'm wondering—can’t we use the same login (with isomorphic-git) for GitHub and Bitbucket? Ideally, removing cloneRepository and ref and add support for object syntax for repository

I had initially figured it was out of scope, but that is a good idea, the change would make it much more cohesive.

Ideally, removing cloneRepository and ref and add support for object syntax for repository
repository: string | { url,  branch } | { url, tag }

That's a good idea, although wouldn't it be more sensible to keep both the previous logic (i.e. downloading a tarball from provider) and this PR? In case one doesn't work, the other might make more sense to use; preferably as Github and Bitbucket specific sources.

farnabaz · 2025-09-18T13:11:18Z

I'd keep only one solution if it's generic enough, So if new solution works for Github and Bitbucket lets use it. But we can provide a compatibility solution and deprecate old usage before removing them all.

As an alternative, I was thinking about using giget package to download repositories. We need to check about the coverage and performance of giget and isomorphic-git

wale · 2025-09-18T13:33:55Z

I'd keep only one solution if it's generic enough, So if new solution works for Github and Bitbucket lets use it. But we can provide a compatibility solution and deprecate old usage before removing them all.

That's true, I'll keep my generic solution, and hide the previous logic behind a boolean (downloadTarball?)

As an alternative, I was thinking about using giget package to download repositories.
We need to check about the coverage and performance of giget and isomorphic-git

Personally I think giget is out of the question, because it has a similar issue of a limited amount of providers (+ not really using Git), kicking my specific issue upstream. I'm using isomorphic-git because it covers my requirements of performing a git clone operation, and users downstream aren't required to implement their own provider.

farnabaz · 2025-09-19T10:54:55Z

That's true, I'll keep my generic solution, and hide the previous logic behind a boolean (downloadTarball?)
By compatibility I don't mean the tar ball download. I mean the config syntax.

What I like is to make it work with current config. And deprecate it in favor of new syntax if needed. So users don't need to change their config on upgrade.

If any change is necessary we force it in v4

src/utils/collection.ts

src/utils/source.ts

src/utils/git.ts

src/utils/source.ts

…et source

docs/content/docs/2.collections/3.sources.md

Co-authored-by: Farnabaz <farnabaz@gmail.com>

src/utils/source.ts

Matheun · 2025-12-04T07:27:33Z

Could we get this merged? Been waiting for this feature quiet a while now :)

farnabaz

LGTM 👍

wale added 8 commits September 16, 2025 15:11

feat: skeleton auth structure

6c700cc

docs: refactor headings

a339a65

feat: get git hash from remote repo

bdc9c77

refactor: rename git hash test file

8d4de63

refactor: rename git hash test file

ac96a30

feat: shallow clone git repo

43f6953

docs: remove commit from docs

f597e2e

docs: elaborate on ref support

f11e033

refactor: remove github/bitbucket sources, change repository call

b468751

vercel bot reviewed Sep 25, 2025

View reviewed changes

src/utils/collection.ts Outdated Show resolved Hide resolved

src/utils/source.ts Show resolved Hide resolved

src/utils/git.ts Outdated Show resolved Hide resolved

fix: properly clone repos without auth, only check commit hash once

c6e1c7b

vercel bot reviewed Sep 25, 2025

View reviewed changes

src/utils/source.ts Outdated Show resolved Hide resolved

src/utils/source.ts Outdated Show resolved Hide resolved

src/utils/source.ts Show resolved Hide resolved

wale added 2 commits September 25, 2025 23:48

fix: folder path collision, ref error checking

c42e800

docs: add documentation for repository object type, remove gh/bitbuck…

a94b503

…et source

farnabaz reviewed Oct 28, 2025

View reviewed changes

docs/content/docs/2.collections/3.sources.md Outdated Show resolved Hide resolved

farnabaz reviewed Oct 28, 2025

View reviewed changes

docs/content/docs/2.collections/3.sources.md Show resolved Hide resolved

farnabaz and others added 5 commits October 28, 2025 17:06

Merge branch 'main' into git-content-source

b557262

lint: fix

718f6bc

up

840886d

Update docs/content/docs/2.collections/3.sources.md

9b6b109

Co-authored-by: Farnabaz <farnabaz@gmail.com>

Update docs/content/docs/2.collections/3.sources.md

b1f4678

Co-authored-by: Farnabaz <farnabaz@gmail.com>

vercel bot reviewed Oct 29, 2025

View reviewed changes

src/utils/source.ts Show resolved Hide resolved

Matheun mentioned this pull request Nov 21, 2025

Handle remote github repository nuxt-content/nuxt-studio#116

Open

farnabaz force-pushed the main branch from a16ddf4 to 1af6adc Compare November 28, 2025 16:25

farnabaz added 2 commits December 17, 2025 11:07

Merge branch 'main' into git-content-source

5306ea8

chore: backward compatibility

aff5a13

vercel bot deployed to Preview December 17, 2025 10:39 View deployment

lint: fix

a6794cf

vercel bot deployed to Preview December 17, 2025 10:44 View deployment

farnabaz added 2 commits December 17, 2025 13:16

chore: cleanup

64f0eb3

chore: update lockfile

177634e

vercel bot deployed to Preview December 17, 2025 12:24 View deployment

farnabaz added 2 commits December 17, 2025 14:17

fix: repository urls backward compatibility

c0cadf8

lint: fix

abfc069

farnabaz approved these changes Dec 17, 2025

View reviewed changes

farnabaz merged commit e6bf06f into nuxt:main Dec 17, 2025
4 of 5 checks passed

farnabaz changed the title ~~feat: shallow clone git repository sources (including auth changes)~~ feat: shallow clone git repository sources Dec 19, 2025

wale deleted the git-content-source branch February 25, 2026 13:04

Uh oh!

Conversation

wale commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Linked issue

❓ Type of change

📚 Description

📝 Checklist

Uh oh!

vercel bot commented Sep 16, 2025

Uh oh!

pkg-pr-new bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

farnabaz commented Sep 18, 2025

Uh oh!

wale commented Sep 18, 2025

Uh oh!

farnabaz commented Sep 18, 2025

Uh oh!

wale commented Sep 18, 2025

Uh oh!

farnabaz commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Matheun commented Dec 4, 2025

Uh oh!

farnabaz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wale commented Sep 16, 2025 •

edited

Loading

pkg-pr-new bot commented Sep 16, 2025 •

edited

Loading

farnabaz commented Sep 19, 2025 •

edited

Loading