Skip to content

feat(tools): add repository update system for automated Nx migrations#32124

Merged
FrozenPandaz merged 4 commits intomasterfrom
update-script
Aug 21, 2025
Merged

feat(tools): add repository update system for automated Nx migrations#32124
FrozenPandaz merged 4 commits intomasterfrom
update-script

Conversation

@FrozenPandaz
Copy link
Copy Markdown
Contributor

@FrozenPandaz FrozenPandaz commented Jul 29, 2025

Current Behavior

There is no automated system for updating multiple repositories with Nx migrations, requiring manual updates across different repositories.

Expected Behavior

A comprehensive TypeScript-based automation system that can efficiently update multiple repositories with Nx migrations using GitHub CLI integration and concurrent processing.

Related Issue(s)

Introduces a new tool for automating repository updates with Nx migrations.

Changes Made

🚀 New Repository Update System

  • Automated repository cloning: Uses GitHub CLI with shallow clones (--depth 1) for fast setup
  • Package manager detection: Automatically detects npm, pnpm, yarn, or bun based on lockfiles
  • Concurrent processing: Clones and updates multiple repositories in parallel
  • Nx migration automation: Runs nx migrate with automatic commit generation
  • Pull request creation: Automatically creates PRs with consistent formatting

📁 Project Structure

tools/update-repos/
├── src/
│   ├── setup-repos.ts          # Clone repositories concurrently
│   └── update-repo.ts          # Update repos with migrations
├── config/
│   └── repos.json              # Repository configuration
├── project.json                # Nx project configuration
├── tsconfig.json               # TypeScript configuration
├── tsconfig.lib.json           # TypeScript library configuration
└── README.md                   # Comprehensive documentation

⚙️ Nx Targets

  • setup-update-repos: Clone all configured repositories
  • update-nx-repo: Update the Nx repository
  • update-ocean-repo: Update the Ocean repository
  • update-nx-examples-repo: Update the Nx examples repository
  • update-all-repos: Update all repositories with progress tracking

🔧 Key Features

  • OS temp directory isolation: Stores clones in os.tmpdir()/updating-nx/repos
  • TypeScript compilation: Full Nx project with proper build pipeline
  • Error handling: Comprehensive error catching and reporting
  • Verbose logging: Real-time progress tracking with timestamps
  • GitHub CLI integration: Authenticated operations for cloning and PR creation

📚 Documentation

  • Complete usage instructions and examples
  • Architecture overview and workflow explanation
  • Troubleshooting guide for common issues
  • Performance optimization details
  • Security considerations and best practices

This system provides a robust, scalable solution for maintaining consistency across multiple repositories when updating Nx versions.

@vercel
Copy link
Copy Markdown

vercel Bot commented Jul 29, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
nx-dev Ready Ready Preview Aug 21, 2025 0:05am

@nx-cloud
Copy link
Copy Markdown
Contributor

nx-cloud Bot commented Jul 29, 2025

View your CI Pipeline Execution ↗ for commit 41836d9

Command Status Duration Result
nx run-many -t check-imports check-commit check... ✅ Succeeded 1m 34s View ↗
nx affected --targets=lint,test,build,e2e,e2e-c... ✅ Succeeded <1s View ↗
nx-cloud record -- nx-cloud conformance:check ✅ Succeeded 2s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 6s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded 5s View ↗
nx documentation ✅ Succeeded 38s View ↗

☁️ Nx Cloud last updated this comment at 2025-08-20 23:19:12 UTC

Comment thread tools/update-repos/src/setup-repos.ts Outdated
Comment on lines +64 to +70
await execAsync(
`gh repo clone "${repoIdentifier}" "${cloneDir}" -- --depth 1`
);

// Checkout the specified branch
log(`Checking out branch: ${repoBranch}`);
await execAsync(`cd "${cloneDir}" && git checkout "${repoBranch}"`);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using a shallow clone with --depth 1, only the latest commit of the default branch is fetched. If repoBranch differs from the repository's default branch, the checkout operation will fail because that branch's commits aren't included in the shallow clone.

Consider modifying the clone command to specify the branch directly:

gh repo clone "${repoIdentifier}" "${cloneDir}" -- --depth 1 --branch "${repoBranch}"

This ensures the shallow clone contains the correct branch from the start, avoiding potential checkout failures.

Suggested change
await execAsync(
`gh repo clone "${repoIdentifier}" "${cloneDir}" -- --depth 1`
);
// Checkout the specified branch
log(`Checking out branch: ${repoBranch}`);
await execAsync(`cd "${cloneDir}" && git checkout "${repoBranch}"`);
await execAsync(
`gh repo clone "${repoIdentifier}" "${cloneDir}" -- --depth 1 --branch "${repoBranch}"`
);

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

@FrozenPandaz FrozenPandaz changed the title fix(repo): use shallow clones in repository setup script feat(tools): improve repository update system with shallow clones and better structure Jul 29, 2025
@FrozenPandaz FrozenPandaz changed the title feat(tools): improve repository update system with shallow clones and better structure feat(tools): add repository update system for automated Nx migrations Jul 29, 2025
// Create and checkout update branch from remote main branch
const updateBranch = 'upnx';
await execWithOutput(
`git checkout -B ${updateBranch} origin/${mainBranch}`,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Git checkout command syntax is incorrect. 'git checkout -B ${updateBranch} origin/${mainBranch}' will fail because origin/${mainBranch} is a remote reference, not a local branch. Should use 'git checkout -B ${updateBranch} origin/${mainBranch}' only after ensuring the remote branch is fetched, or use 'git checkout -b ${updateBranch}' and then 'git reset --hard origin/${mainBranch}'.

Suggested change
`git checkout -B ${updateBranch} origin/${mainBranch}`,
`git fetch origin ${mainBranch} && git checkout -B ${updateBranch} origin/${mainBranch}`,

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

Comment on lines +23 to +32
const CONFIG_FILE = path.join(
SCRIPT_DIR,
'..',
'..',
'..',
'tools',
'update-repos',
'config',
'repos.json'
);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CONFIG_FILE path construction is fragile and error-prone. Using multiple '..' path segments (lines 25-27) creates a brittle path that depends on the exact directory structure and __dirname location. If the script is moved or the build output structure changes, this path will break. Should use path.resolve() with a more reliable base path or environment variable.

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

- Add --create-commits flag to nx migrate for automatic migration commits
- Simplify version detection using require('nx/package.json').version
- Re-enable all repositories (nx, ocean, nx-examples) for updates
- Update PR description format to "Updating Nx from X to Y"

🤖 Generated with [Claude Code](https://claude.ai/code)
- Integrate repository setup into individual update tasks
- Add noop behavior to skip setup if repository already exists
- Add nx reset step after migrations to prevent cache issues
- Add --no-verify flag to git push to skip problematic hooks
- Add convenient pnpm update-all-repos script
- Update documentation to reflect streamlined workflow
- Add optional post-nx-update script detection and execution
- Run script after migrations complete but before nx reset
- Commit changes from post-update script separately
- Graceful handling when script doesn't exist (no failure)
- Support for all package managers (pnpm/yarn/bun/npm)
- Update documentation with post-update hook details
Add --parallel 5 flag to improve performance when updating all repositories.
@FrozenPandaz FrozenPandaz merged commit b3dc005 into master Aug 21, 2025
6 checks passed
@FrozenPandaz FrozenPandaz deleted the update-script branch August 21, 2025 19:49
FrozenPandaz added a commit that referenced this pull request Aug 22, 2025
…#32124)

## Current Behavior

There is no automated system for updating multiple repositories with Nx
migrations, requiring manual updates across different repositories.

## Expected Behavior

A comprehensive TypeScript-based automation system that can efficiently
update multiple repositories with Nx migrations using GitHub CLI
integration and concurrent processing.

## Related Issue(s)

Introduces a new tool for automating repository updates with Nx
migrations.

## Changes Made

### 🚀 New Repository Update System
- **Automated repository cloning**: Uses GitHub CLI with shallow clones
(`--depth 1`) for fast setup
- **Package manager detection**: Automatically detects npm, pnpm, yarn,
or bun based on lockfiles
- **Concurrent processing**: Clones and updates multiple repositories in
parallel
- **Nx migration automation**: Runs `nx migrate` with automatic commit
generation
- **Pull request creation**: Automatically creates PRs with consistent
formatting

### 📁 Project Structure
```
tools/update-repos/
├── src/
│   ├── setup-repos.ts          # Clone repositories concurrently
│   └── update-repo.ts          # Update repos with migrations
├── config/
│   └── repos.json              # Repository configuration
├── project.json                # Nx project configuration
├── tsconfig.json               # TypeScript configuration
├── tsconfig.lib.json           # TypeScript library configuration
└── README.md                   # Comprehensive documentation
```

### ⚙️ Nx Targets
- `setup-update-repos`: Clone all configured repositories
- `update-nx-repo`: Update the Nx repository
- `update-ocean-repo`: Update the Ocean repository
- `update-nx-examples-repo`: Update the Nx examples repository
- `update-all-repos`: Update all repositories with progress tracking

### 🔧 Key Features
- **OS temp directory isolation**: Stores clones in
`os.tmpdir()/updating-nx/repos`
- **TypeScript compilation**: Full Nx project with proper build pipeline
- **Error handling**: Comprehensive error catching and reporting
- **Verbose logging**: Real-time progress tracking with timestamps
- **GitHub CLI integration**: Authenticated operations for cloning and
PR creation

### 📚 Documentation
- Complete usage instructions and examples
- Architecture overview and workflow explanation
- Troubleshooting guide for common issues
- Performance optimization details
- Security considerations and best practices

This system provides a robust, scalable solution for maintaining
consistency across multiple repositories when updating Nx versions.

(cherry picked from commit b3dc005)
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Aug 27, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants