I think the failing on Ruby 2.5 is due to a gem caching issue in GitHub actions. You can see from the log that is fetching pysch 2.2.4, but the backtrace shows a cached 5.0.1.

This passed fine in my repo (https://github.com/jeremyevans/rack/actions/runs/3955117688/jobs/6773115549), probably because it wasn't using any cached gems.

I think it's the rdoc entry in the Gemfile causing psych to be loaded, but I don't think that's needed in CI. I'll see if I can fix this in a different way, so it doesn't try to install a psych gem.

I'm not against this PR. However, I don't think this should be necessary either. It's introducing new technical debt in our gem configuration and CI pipeline which I don't think we actually derive any benefit from.

@tenderlove was in support of removing 2.4 and 2.5 and I'm also in support of that. So I'll leave it up to him to decide which PR to merge. I don't really have a strong opinion except that I don't think we should spend time dealing with progressively flaky old Ruby versions. There is no way I can test that stuff locally, and I don't think anyone is going to be deploying Rack 3 applications using Ruby 2.4 or 2.5, or even 2.6 for that matter.

Even if we drop 2.4 and 2.5 support, we should still merge this. There's no reason for our CI jobs to install rdoc/psych gems, or for the non-external CI job to install bake-test-external and its dependencies. It's pure waste in CI.

In terms of Ruby 2.6, JRuby 9.3 is still supported by JRuby and targets Ruby 2.6. So I definitely don't think we want to drop Ruby 2.6 support.

As for Ruby 2.4 and 2.5, I would only support dropping them if there was a benefit to rack users. I was in favor of dropping Ruby 2.3 support so we could use Regexp#match? without forcing the use of refinements. If we have a similar good reason to drop Ruby 2.4 and/or 2.5, I think it should be considered. However, dropping support without any reason at all does a disservice to rack users, IMO.

In terms of dealing with CI for older Ruby versions, I volunteer to deal with any 2.4 or 2.5 CI fallout, should that be an issue.

the non-external CI job to install bake-test-external and its dependencies. It's pure waste in CI.

IIUC, they all use the same cache, so it's largely irrelevant. It's actually might break the cache using environment variables in the gemfile as I believe the checksum of the file is used as part of the cache, but not the environment. Can you check this?

IIUC, they all use the same cache, so it's largely irrelevant. It's actually might break the cache using environment variables in the gemfile as I believe the checksum of the file is used as part of the cache, but not the environment. Can you check this?

Caching is based on the generated lock file (https://github.com/ruby/setup-ruby#how-it-works). The cache will be skipped whenever there is a bump to tiny version of Ruby, or any version updates to any dependencies (even transitive dependencies), since those are included in the lock file.

they all use the same cache is not accurate. Each Ruby version already uses a separate cache.

This will not break any caching, but it will result in the test and test-external jobs using different caches even for the same Ruby version. That won't be a problem.

If for some reason, you wanted the caches to be shared between test and test-external (which has no real benefit), we would still want to skip the rdoc dependency, since nothing in CI uses it.