Skip to content

improve os.curdir/os.cd#6854

Merged
waruqi merged 4 commits intodevfrom
opti
Sep 27, 2025
Merged

improve os.curdir/os.cd#6854
waruqi merged 4 commits intodevfrom
opti

Conversation

@waruqi
Copy link
Member

@waruqi waruqi commented Sep 24, 2025

@waruqi waruqi added this to the v3.0.4 milestone Sep 24, 2025
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @waruqi, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a performance optimization for retrieving environment variables within the os module. By implementing a caching mechanism for os.getenvs, repeated calls to fetch environment variables will be significantly faster. To maintain data consistency, the cache is automatically invalidated whenever environment variables are modified, ensuring that the most current environment settings are always available.

Highlights

  • Environment Variable Caching: The os.getenvs function now caches the retrieved environment variables in os._ENVS to improve performance on subsequent calls, avoiding redundant system calls.
  • Cache Invalidation: A new line os._ENVS = nil has been added to os._notify_envs_changed to ensure that the cached environment variables are cleared whenever environment variables are updated, preventing stale data from being used.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces caching for os.getenvs() to improve performance by avoiding repeated parsing of environment variables. The implementation correctly invalidates the cache when the environment is modified. However, the new caching mechanism introduces a critical race condition in concurrent environments (e.g., using coroutines), which could lead to a poisoned cache with stale data and result in incorrect build behavior. I have detailed this issue in a review comment and recommended a fix.

Comment on lines 1296 to 1315
local envs = os._ENVS
if envs == nil then
envs = {}
for _, line in ipairs(os._getenvs()) do
local p = line:find('=', 1, true)
if p then
local key = line:sub(1, p - 1):trim()
-- only translate Path to PATH on windows
-- @see https://github.com/xmake-io/xmake/issues/3752
if os.host() == "windows" and key:lower() == "path" then
key = key:upper()
end
local values = line:sub(p + 1):trim()
if #key > 0 then
envs[key] = values
end
end
end
os._ENVS = envs
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The new caching mechanism introduces a race condition in a concurrent environment (coroutines) that can lead to a poisoned cache with stale data, potentially causing incorrect build behavior.

Here is a scenario:

  1. Coroutine A calls os.getenvs(). It finds os._ENVS is nil and starts computing the environment variables by calling os._getenvs().
  2. While Coroutine A is parsing, Coroutine B calls an environment-modifying function like os.setenv(). This modifies the process environment and then calls os._notify_envs_changed(), which sets os._ENVS to nil to invalidate the cache.
  3. Coroutine A finishes its computation. The data it has is now stale because of Coroutine B's action.
  4. Coroutine A then sets os._ENVS to its stale data, poisoning the cache.
  5. Any subsequent call to os.getenvs() will return incorrect environment variables until the next invalidation.

To fix this, you need to ensure that the check for a nil cache, the computation of the environment variables, and the setting of the cache are atomic. This typically requires a locking mechanism to be used around os.getenvs() and all functions that modify the environment (e.g., os.setenv, os.addenv, etc.).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as there is no process execution, there will be no coroutine switching.

@waruqi waruqi changed the title improve os.getenvs improve os.getenvs and os.curdir/os.cd Sep 24, 2025
@waruqi waruqi force-pushed the opti branch 7 times, most recently from 7d21c83 to 545557f Compare September 24, 2025 15:32
@waruqi waruqi force-pushed the opti branch 12 times, most recently from a0ee805 to 91b8741 Compare September 26, 2025 01:45
@waruqi waruqi changed the title improve os.getenvs and os.curdir/os.cd improve os.curdir/os.cd Sep 27, 2025
@waruqi waruqi merged commit dd07af9 into dev Sep 27, 2025
43 of 44 checks passed
@waruqi waruqi deleted the opti branch September 27, 2025 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant