This pull request backports #56275 to the 7-2-stable branch

Motivation / Background

Since #56271 did not reproduce against 7-2-stable branch,
#56275 has not been back ported to 7-2-stable branch.
Recently, it starts failing like https://buildkite.com/rails/rails/builds/124364 with Bundler 4.0.0.

Detail

This pull request backports #56275 to the 7-2-stable branch, conflict with #53647 has been addressed manually.

Additional information

I usually cherry-pick merge commit and push them stable branches manually, this time I have opened it as pull request to make sure it fixes as expected.

Checklist

Before submitting the PR make sure the following are checked:

• This Pull Request is related to one change. Unrelated changes should be opened in separate PRs.
• Commit message has a detailed description of what changed and why. If this PR fixes a related issue include it in the commit message. Ex: [Fix #issue-number]
• Tests are added or updated if you fix a bug or add a feature.
• CHANGELOG files are updated for the changed libraries if there is a behavior change or additional feature. Minor bug fixes and documentation changes should not be included.