Skip to content

[CLI-Eval] feat: adding in env eval#15118

Merged
brookemosby merged 22 commits intomainfrom
brookemosby/cli-164-vc-env-eval
Feb 24, 2026
Merged

[CLI-Eval] feat: adding in env eval#15118
brookemosby merged 22 commits intomainfrom
brookemosby/cli-164-vc-env-eval

Conversation

@brookemosby
Copy link
Copy Markdown
Contributor

@brookemosby brookemosby commented Feb 18, 2026

Adding in new eval, and then updating other evals so they don't tell the agent what command to use.

Note

Low Risk Change

This PR adds new CLI evaluation tests and improves documentation/error messages for non-interactive CLI usage, with no changes to authentication, authorization, or security controls.

  • New eval test fixtures for env commands (add, ls, pull, remove, update)
  • CI workflow updated to output eval results on failure
  • CLI error messages and help text improved for non-interactive mode

Risk assessment for commit 6bfee74.

@brookemosby brookemosby requested review from a team as code owners February 18, 2026 21:05
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Feb 18, 2026

🦋 Changeset detected

Latest commit: 6bfee74

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
vercel Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 18, 2026

📦 CLI Tarball Ready

The Vercel CLI tarball for this PR is now available!

Quick Test

You can test this PR's CLI directly by running:

npx https://vercel-jmvi7v9it.vercel.sh/tarballs/vercel.tgz --help

Use in vercel.json

To use this CLI version in your project builds, add to your vercel.json:

{
  "build": {
    "env": {
      "VERCEL_CLI_VERSION": "vercel@https://vercel-jmvi7v9it.vercel.sh/tarballs/vercel.tgz"
    }
  }
}

Python Runtime Wheel

A Python runtime wheel was also built for this PR.
To use in your Python project builds, also set this environment variable:

VERCEL_RUNTIME_PYTHON="vercel-runtime @ https://vercel-jmvi7v9it.vercel.sh/tarballs/vercel_runtime-0.6.0.dev1771891153+6bfee74-py3-none-any.whl"

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 18, 2026

🧪 Unit Test Strategy

Comparing: 1abc15b6bfee74 (view diff)

Strategy: Code changed outside of a package - running all unit tests

⚠️ All unit tests will run because global code changes could impact all packages.

Affected packages - 15 (38%)
  1. vercel
  2. @vercel/build-utils
  3. @vercel/client
  4. @vercel/firewall
  5. @vercel/fs-detectors
  6. @vercel/go
  7. @vercel/hydrogen
  8. @vercel/next
  9. @vercel/node
  10. @vercel/python
  11. @vercel/remix-builder
  12. @vercel/ruby
  13. @vercel/rust
  14. @vercel/static-build
  15. examples
Unaffected packages - 25 (63%)
  1. @vercel-internals/get-package-json
  2. @vercel/backends
  3. @vercel/cervel
  4. @vercel/cli-auth
  5. @vercel/config
  6. @vercel/detect-agent
  7. @vercel/edge
  8. @vercel/elysia
  9. @vercel/error-utils
  10. @vercel/express
  11. @vercel/fastify
  12. @vercel/frameworks
  13. @vercel/functions
  14. @vercel/gatsby-plugin-vercel-builder
  15. @vercel/h3
  16. @vercel/hono
  17. @vercel/koa
  18. @vercel/nestjs
  19. @vercel/oidc
  20. @vercel/oidc-aws-credentials-provider
  21. @vercel/python-analysis
  22. @vercel/redwood
  23. @vercel/related-projects
  24. @vercel/routing-utils
  25. @vercel/static-config

Results

  • Unit tests: All affected packages will run unit tests
  • E2E tests: Running in parallel via E2E Tests workflow
  • Type checks: All affected packages will run type checks

This comment is automatically generated based on the affected testing strategy

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
@brookemosby brookemosby merged commit e793931 into main Feb 24, 2026
239 of 246 checks passed
@brookemosby brookemosby deleted the brookemosby/cli-164-vc-env-eval branch February 24, 2026 00:57
styfle pushed a commit that referenced this pull request Feb 24, 2026
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and the packages will
be published to npm automatically. If you're not ready to do a release
yet, that's fine, whenever you add more changesets to main, this PR will
be updated.


# Releases
## vercel@50.23.0

### Minor Changes

- [dev] allow to skip authentication and project linking for `vc dev` by
setting `VERCEL_EXPERIMENTAL_DEV_SKIP_LINK` env var.
([#15122](#15122))

- [services] detect project root to prevent accidental setup of a
service as a standalone project.
([#15187](#15187))

### Patch Changes

- Fix re-authentication for teams with missing scope to use the device
code flow instead of the deprecated SSO redirect flow.
([#15074](#15074))

- Add CLI eval for vc env command
([#15118](#15118))

-   Updated dependencies \[]:
    -   @vercel/node@5.6.7

## @vercel/functions@3.4.3

### Patch Changes

- [functions] Revert "[functions] URL encode cache tags"
([#15213](#15213))


<!-- VADE_RISK_START -->
> [!NOTE]
> Low Risk Change
>
> This PR contains only version bumps and changelog updates from the
Changesets release automation, with no actual code changes.
> 
> - Deletes changeset markdown files consumed during release
> - Updates version numbers in package.json files (50.22.3→50.23.0,
3.4.2→3.4.3)
> - Adds changelog entries documenting previously merged changes
>
> <sup>Risk assessment for [commit
18f0132](https://github.com/vercel/vercel/commit/18f013287efb10cb34e666236f9e34f0c72d0c83).</sup>
<!-- VADE_RISK_END -->

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants