Skip to content

Reenable NPM integration tests#10623

Merged
scidomino merged 1 commit into
mainfrom
tomm_npm_verify
Oct 10, 2025
Merged

Reenable NPM integration tests#10623
scidomino merged 1 commit into
mainfrom
tomm_npm_verify

Conversation

@scidomino

@scidomino scidomino commented Oct 6, 2025

Copy link
Copy Markdown
Collaborator

TLDR

Reverts #10520 along with a one line fix that makes interactive integration tests work.

Dive Deeper

Github sets a CI=true environment variable. For reasons I still don't understand, this breaks the interactive tests when INTEGRATION_TEST_USE_INSTALLED_GEMINI is true.

Reviewer Test Plan

I ran:

gh workflow run verify-release.yml --ref tomm_npm_verify -f ref=tomm_npm_verify -f version='0.8.2' -f npm-package='@google/gemini-cli@latest'

Which ran successfully: https://github.com/google-gemini/gemini-cli/actions/runs/18412811531

Linked issues / bugs

Fixes #10517

@github-actions

github-actions Bot commented Oct 6, 2025

Copy link
Copy Markdown

Size Change: -2 B (0%)

Total Size: 17.8 MB

ℹ️ View Unchanged
Filename Size Change
./bundle/gemini.js 17.7 MB -2 B (0%)
./bundle/sandbox-macos-permissive-closed.sb 1.03 kB 0 B
./bundle/sandbox-macos-permissive-open.sb 830 B 0 B
./bundle/sandbox-macos-permissive-proxied.sb 1.31 kB 0 B
./bundle/sandbox-macos-restrictive-closed.sb 3.29 kB 0 B
./bundle/sandbox-macos-restrictive-open.sb 3.36 kB 0 B
./bundle/sandbox-macos-restrictive-proxied.sb 3.56 kB 0 B

compressed-size-action

@scidomino scidomino force-pushed the tomm_npm_verify branch 29 times, most recently from f4d97af to 1482085 Compare October 7, 2025 22:18
@scidomino scidomino force-pushed the tomm_npm_verify branch 21 times, most recently from 20cae13 to 000c980 Compare October 8, 2025 22:44
@scidomino

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request reverts the temporary removal of NPM integration tests and includes a fix to make them work in the GitHub Actions environment by forcing the CI environment variable to false. The changes look good and correctly address the issue with interactive tests in CI. I have one suggestion to improve the CI workflow's reliability.

Comment thread .github/actions/verify-release/action.yml Outdated

@bobcatfish bobcatfish left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried to dig up more info on this and found some interesting things but not a lot of concrete sources XD

https://github.blog/changelog/2020-04-15-github-actions-sets-the-ci-environment-variable-to-true/ <-- github actions is doing this on purpose, tho there's very little information there

the AI summaries for my searches assert repeatedly that CI=true will change npm behavior but sources are hard to come by, https://stackoverflow.com/questions/62473327/how-does-ci-true-affect-the-npm-install-command seems like the best

I'm wondering if CI=true is such a standard that github actions is asserting it by default, if we're going to have other side effects we don't want from explicitly disabling this 🤔 some of the failures on this PR seem like they could be related:

 FAIL  file-system-interactive.test.ts > Interactive file system > should perform a read-then-write sequence
AssertionError: CLI did not start up in interactive mode correctly: expected false to be true // Object.is equality

- Expected
+ Received

- true
+ false

 ❯ file-system-interactive.test.ts:45:9
     43|         isReady,
     44|         'CLI did not start up in interactive mode correctly',
     45|       ).toBe(true);
       |         ^
     46| 
     47|       // Step 1: Read the file

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/3]⎯

Comment thread integration-tests/test-helper.ts Outdated
@@ -835,6 +835,10 @@ export class TestRig {
) as { [key: string]: string },
};

// Force CI off since it breaks rendering in github actions
// https://github.com/google-gemini/gemini-cli/issues/10517
options.env!['CI'] = 'false';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

@scidomino

Copy link
Copy Markdown
Collaborator Author

Yeah. I actually deleted my request in the chat for a reviewer since I saw these errors :(

Investigating now.

@scidomino

Copy link
Copy Markdown
Collaborator Author

Interestingly, the test failures were unrelated flakes. I disabled them and created #10915 and #10916 to track fixing them.

I reworked the PR to not change how the integration tests run elsewhere.

@bobcatfish bobcatfish left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh, even more flakes! Well anyway this one looks good!

image

I don't think I have permission to approve workflow changes unfortunately but ill see what happens if i try XD

@bobcatfish

Copy link
Copy Markdown
Contributor

oh! i do! hooray!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s A small PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Interactive integration tests fail on github when run with INTEGRATION_TEST_USE_INSTALLED_GEMINI

4 participants