Skip to content

[CI] Improve ccache performance for Windows Python builds (ccache windows version bump)#42195

Closed
murgatroid99 wants to merge 1 commit into
grpc:masterfrom
murgatroid99:python_windows_ccache_fix
Closed

[CI] Improve ccache performance for Windows Python builds (ccache windows version bump)#42195
murgatroid99 wants to merge 1 commit into
grpc:masterfrom
murgatroid99:python_windows_ccache_fix

Conversation

@murgatroid99

Copy link
Copy Markdown
Member

In the Python Windows builds, ccache currently has a low cache hit rate and fails to populate the direct mode cache. This is an attempt to fix that.

@murgatroid99

Copy link
Copy Markdown
Member Author

We got an 80 minute build with a cache hit ratio of about 75%: https://btx.cloud.google.com/invocations/1d464888-d5d4-4d6a-9c60-1e6da7e54e0f/targets

@sergiitk

Copy link
Copy Markdown
Member

Internal ref b/501458064

@sergiitk

Copy link
Copy Markdown
Member
T:\altsrc\github\grpc>ccache --show-stats 
Cacheable calls:    7300 / 7300 (100.0%)
  Hits:             5348 / 7300 (73.26%)
    Direct:         5348 / 5348 (100.0%)
    Preprocessed:      0 / 5348 ( 0.00%)
  Misses:           1952 / 7300 (26.74%)
Local storage:
  Cache size (GiB):  4.7 /  5.0 (94.46%)
  Hits:             2228 / 7300 (30.52%)
  Misses:           5072 / 7300 (69.48%)
Remote storage:
  Hits:             3120 / 5072 (61.51%)
  Misses:           1952 / 5072 (38.49%)

@sergiitk sergiitk changed the title Improve ccache performance for Windows Python builds Improve ccache performance for Windows Python builds (ccache windows version bump) Apr 22, 2026
@sergiitk sergiitk changed the title Improve ccache performance for Windows Python builds (ccache windows version bump) [CI] Improve ccache performance for Windows Python builds (ccache windows version bump) Apr 22, 2026
copybara-service Bot pushed a commit that referenced this pull request Apr 23, 2026
… build directory consistent across runs) (#42212)

We see ccache cache misses because each python build runs in a different temporary directory, and some header file names in the cache entry's manifest appear to use the full file path, so they fail to match. This change resolves that by making the build directory consistent for each build type.

ref:
- b/501458064
- Previous change: #42195

Closes #42212

COPYBARA_INTEGRATE_REVIEW=#42212 from murgatroid99:python_windows_ccache_basedir a29d1bb
PiperOrigin-RevId: 904139620
asheshvidyut pushed a commit to asheshvidyut/grpc that referenced this pull request Apr 23, 2026
…dows version bump) (grpc#42195)

In the Python Windows builds, ccache currently has a low cache hit rate and fails to populate the direct mode cache. This is an attempt to fix that.

Closes grpc#42195

COPYBARA_INTEGRATE_REVIEW=grpc#42195 from murgatroid99:python_windows_ccache_fix 2205915
PiperOrigin-RevId: 903665805
asheshvidyut pushed a commit to asheshvidyut/grpc that referenced this pull request Apr 23, 2026
… build directory consistent across runs) (grpc#42212)

We see ccache cache misses because each python build runs in a different temporary directory, and some header file names in the cache entry's manifest appear to use the full file path, so they fail to match. This change resolves that by making the build directory consistent for each build type.

ref:
- b/501458064
- Previous change: grpc#42195

Closes grpc#42212

COPYBARA_INTEGRATE_REVIEW=grpc#42212 from murgatroid99:python_windows_ccache_basedir a29d1bb
PiperOrigin-RevId: 904139620
asheshvidyut pushed a commit to a-detiste/grpc that referenced this pull request Jun 10, 2026
…dows version bump) (grpc#42195)

In the Python Windows builds, ccache currently has a low cache hit rate and fails to populate the direct mode cache. This is an attempt to fix that.

Closes grpc#42195

COPYBARA_INTEGRATE_REVIEW=grpc#42195 from murgatroid99:python_windows_ccache_fix 2205915
PiperOrigin-RevId: 903665805
asheshvidyut pushed a commit to a-detiste/grpc that referenced this pull request Jun 10, 2026
… build directory consistent across runs) (grpc#42212)

We see ccache cache misses because each python build runs in a different temporary directory, and some header file names in the cache entry's manifest appear to use the full file path, so they fail to match. This change resolves that by making the build directory consistent for each build type.

ref:
- b/501458064
- Previous change: grpc#42195

Closes grpc#42212

COPYBARA_INTEGRATE_REVIEW=grpc#42212 from murgatroid99:python_windows_ccache_basedir a29d1bb
PiperOrigin-RevId: 904139620
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants