fix(cache): isolate metadata in prefetch goroutine#7631
Merged
yongtang merged 1 commit intoOct 22, 2025
Merged
Conversation
3f53c95 to
70e7be0
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #7631 +/- ##
==========================================
+ Coverage 55.70% 62.61% +6.91%
==========================================
Files 224 278 +54
Lines 10016 18574 +8558
==========================================
+ Hits 5579 11631 +6052
- Misses 3978 6256 +2278
- Partials 459 687 +228 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Wrap doPrefetch with a fresh metadata context to prevent concurrent writes to the request-scoped metadata map during background prefetch. Add a new integration test configuring a plugin chain, triggering the issue seen here. Hammers concurrent queries while log reads metadata fields repeatedly. Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
210294e to
c25369d
Compare
yongtang
approved these changes
Oct 22, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Why is this pull request needed and what does it do?
Affects the
cacheplugin.Wrap
doPrefetchwith a fresh metadata context to prevent concurrent writes to the request-scoped metadata map during background prefetch. Other plugins in the chain might do that.Add a new integration test which triggers the issue. It hammers concurrent queries while log reads metadata fields repeatedly. If you run the test against
masterwithgo test -race ./testyou will see the following:Details
================== WARNING: DATA RACE Read at 0x00c000820060 by goroutine 843: runtime.mapdelete_fast64() /opt/homebrew/Cellar/go/1.25.2/libexec/src/internal/runtime/maps/runtime_fast64_swiss.go:502 +0x8c github.com/coredns/coredns/plugin/metadata.ValueFunc() /git/coredns/plugin/metadata/provider.go:94 +0x74 github.com/coredns/coredns/plugin/pkg/replacer.replacer.Replace() /git/coredns/plugin/pkg/replacer/replacer.go:273 +0x110 github.com/coredns/coredns/plugin/pkg/replacer.Replacer.Replace() /git/coredns/plugin/pkg/replacer/replacer.go:28 +0x6c github.com/coredns/coredns/plugin/log.Logger.ServeDNS() /git/coredns/plugin/log/log.go:48 +0x424 github.com/coredns/coredns/plugin/log.(*Logger).ServeDNS() :1 +0x90 github.com/coredns/coredns/plugin.NextOrFailure() /git/coredns/plugin/plugin.go:80 +0x1d8 github.com/coredns/coredns/plugin/metadata.(*Metadata).ServeDNS() /git/coredns/plugin/metadata/metadata.go:30 +0x70 github.com/coredns/coredns/core/dnsserver.(*Server).ServeDNS() /git/coredns/core/dnsserver/server.go:305 +0xd00 github.com/coredns/coredns/core/dnsserver.(*Server).ServePacket.func1() /git/coredns/core/dnsserver/server.go:171 +0x9c github.com/miekg/dns.HandlerFunc.ServeDNS() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:37 +0x48 github.com/miekg/dns.(*Server).serveDNS() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:681 +0x5d0 github.com/miekg/dns.(*Server).serveUDPPacket() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:622 +0x2d4 github.com/miekg/dns.(*Server).serveUDP.gowrap2() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:552 +0x90 Previous write at 0x00c000820060 by goroutine 846: runtime.mapaccess2_faststr() /opt/homebrew/Cellar/go/1.25.2/libexec/src/internal/runtime/maps/runtime_faststr_swiss.go:162 +0x29c github.com/coredns/coredns/plugin/metadata.SetValueFunc() /git/coredns/plugin/metadata/provider.go:105 +0x78 github.com/coredns/coredns/plugin/forward.(*Forward).ServeDNS() /git/coredns/plugin/forward/forward.go:154 +0x75c github.com/coredns/coredns/plugin.NextOrFailure() /git/coredns/plugin/plugin.go:80 +0x1d8 github.com/coredns/coredns/plugin/cache.(*Cache).doRefresh() /git/coredns/plugin/cache/handler.go:107 +0x170 github.com/coredns/coredns/plugin/cache.(*Cache).doPrefetch() /git/coredns/plugin/cache/handler.go:96 +0x120 github.com/coredns/coredns/plugin/cache.(*Cache).ServeDNS.gowrap1() /git/coredns/plugin/cache/handler.go:59 +0xc4 Goroutine 843 (running) created at: github.com/miekg/dns.(*Server).serveUDP() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:552 +0x5a4 github.com/miekg/dns.(*Server).ActivateAndServe() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:388 +0x580 github.com/coredns/coredns/core/dnsserver.(*Server).ServePacket() /git/coredns/core/dnsserver/server.go:175 +0x250 github.com/coredns/caddy.startServers.func1.2() /go/pkg/mod/github.com/coredns/caddy@v1.1.4-0.20250930002214-15135a999495/caddy.go:816 +0xb0 Goroutine 846 (running) created at: github.com/coredns/coredns/plugin/cache.(*Cache).ServeDNS() /git/coredns/plugin/cache/handler.go:59 +0x748 github.com/coredns/coredns/plugin.NextOrFailure() /git/coredns/plugin/plugin.go:80 +0x1d8 github.com/coredns/coredns/plugin/log.Logger.ServeDNS() /git/coredns/plugin/log/log.go:36 +0x28c github.com/coredns/coredns/plugin/log.(*Logger).ServeDNS() :1 +0x90 github.com/coredns/coredns/plugin.NextOrFailure() /git/coredns/plugin/plugin.go:80 +0x1d8 github.com/coredns/coredns/plugin/metadata.(*Metadata).ServeDNS() /git/coredns/plugin/metadata/metadata.go:30 +0x70 github.com/coredns/coredns/core/dnsserver.(*Server).ServeDNS() /git/coredns/core/dnsserver/server.go:305 +0xd00 github.com/coredns/coredns/core/dnsserver.(*Server).ServePacket.func1() /git/coredns/core/dnsserver/server.go:171 +0x9c github.com/miekg/dns.HandlerFunc.ServeDNS() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:37 +0x48 github.com/miekg/dns.(*Server).serveDNS() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:681 +0x5d0 github.com/miekg/dns.(*Server).serveUDPPacket() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:622 +0x2d4 github.com/miekg/dns.(*Server).serveUDP.gowrap2() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:552 +0x90 ================== ================== WARNING: DATA RACE Read at 0x00c0009b01b8 by goroutine 843: github.com/coredns/coredns/plugin/metadata.ValueFunc() /git/coredns/plugin/metadata/provider.go:94 +0x7c github.com/coredns/coredns/plugin/pkg/replacer.replacer.Replace() /git/coredns/plugin/pkg/replacer/replacer.go:273 +0x110 github.com/coredns/coredns/plugin/pkg/replacer.Replacer.Replace() /git/coredns/plugin/pkg/replacer/replacer.go:28 +0x6c github.com/coredns/coredns/plugin/log.Logger.ServeDNS() /git/coredns/plugin/log/log.go:48 +0x424 github.com/coredns/coredns/plugin/log.(*Logger).ServeDNS() :1 +0x90 github.com/coredns/coredns/plugin.NextOrFailure() /git/coredns/plugin/plugin.go:80 +0x1d8 github.com/coredns/coredns/plugin/metadata.(*Metadata).ServeDNS() /git/coredns/plugin/metadata/metadata.go:30 +0x70 github.com/coredns/coredns/core/dnsserver.(*Server).ServeDNS() /git/coredns/core/dnsserver/server.go:305 +0xd00 github.com/coredns/coredns/core/dnsserver.(*Server).ServePacket.func1() /git/coredns/core/dnsserver/server.go:171 +0x9c github.com/miekg/dns.HandlerFunc.ServeDNS() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:37 +0x48 github.com/miekg/dns.(*Server).serveDNS() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:681 +0x5d0 github.com/miekg/dns.(*Server).serveUDPPacket() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:622 +0x2d4 github.com/miekg/dns.(*Server).serveUDP.gowrap2() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:552 +0x90 Previous write at 0x00c0009b01b8 by goroutine 846: github.com/coredns/coredns/plugin/metadata.SetValueFunc() /git/coredns/plugin/metadata/provider.go:105 +0x84 github.com/coredns/coredns/plugin/forward.(*Forward).ServeDNS() /git/coredns/plugin/forward/forward.go:154 +0x75c github.com/coredns/coredns/plugin.NextOrFailure() /git/coredns/plugin/plugin.go:80 +0x1d8 github.com/coredns/coredns/plugin/cache.(*Cache).doRefresh() /git/coredns/plugin/cache/handler.go:107 +0x170 github.com/coredns/coredns/plugin/cache.(*Cache).doPrefetch() /git/coredns/plugin/cache/handler.go:96 +0x120 github.com/coredns/coredns/plugin/cache.(*Cache).ServeDNS.gowrap1() /git/coredns/plugin/cache/handler.go:59 +0xc4 Goroutine 843 (running) created at: github.com/miekg/dns.(*Server).serveUDP() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:552 +0x5a4 github.com/miekg/dns.(*Server).ActivateAndServe() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:388 +0x580 github.com/coredns/coredns/core/dnsserver.(*Server).ServePacket() /git/coredns/core/dnsserver/server.go:175 +0x250 github.com/coredns/caddy.startServers.func1.2() /go/pkg/mod/github.com/coredns/caddy@v1.1.4-0.20250930002214-15135a999495/caddy.go:816 +0xb0 Goroutine 846 (running) created at: github.com/coredns/coredns/plugin/cache.(*Cache).ServeDNS() /git/coredns/plugin/cache/handler.go:59 +0x748 github.com/coredns/coredns/plugin.NextOrFailure() /git/coredns/plugin/plugin.go:80 +0x1d8 github.com/coredns/coredns/plugin/log.Logger.ServeDNS() /git/coredns/plugin/log/log.go:36 +0x28c github.com/coredns/coredns/plugin/log.(*Logger).ServeDNS() :1 +0x90 github.com/coredns/coredns/plugin.NextOrFailure() /git/coredns/plugin/plugin.go:80 +0x1d8 github.com/coredns/coredns/plugin/metadata.(*Metadata).ServeDNS() /git/coredns/plugin/metadata/metadata.go:30 +0x70 github.com/coredns/coredns/core/dnsserver.(*Server).ServeDNS() /git/coredns/core/dnsserver/server.go:305 +0xd00 github.com/coredns/coredns/core/dnsserver.(*Server).ServePacket.func1() /git/coredns/core/dnsserver/server.go:171 +0x9c github.com/miekg/dns.HandlerFunc.ServeDNS() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:37 +0x48 github.com/miekg/dns.(*Server).serveDNS() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:681 +0x5d0 github.com/miekg/dns.(*Server).serveUDPPacket() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:622 +0x2d4 github.com/miekg/dns.(*Server).serveUDP.gowrap2() /go/pkg/mod/github.com/miekg/dns@v1.1.68/server.go:552 +0x90 ================== --- FAIL: TestIssue7630 (2.66s) testing.go:1617: race detected during execution of test2. Which issues (if any) are related?
Fixes #7630.
3. Which documentation changes (if any) need to be made?
None.
4. Does this introduce a backward incompatible change or deprecation?
No.