Skip to content

fix(file): data race in tree Elem.Name#7574

Merged
yongtang merged 1 commit into
coredns:masterfrom
thevilledev:fix/file-plugin-tree-lazy-write
Sep 29, 2025
Merged

fix(file): data race in tree Elem.Name#7574
yongtang merged 1 commit into
coredns:masterfrom
thevilledev:fix/file-plugin-tree-lazy-write

Conversation

@thevilledev

@thevilledev thevilledev commented Sep 26, 2025

Copy link
Copy Markdown
Collaborator

1. Why is this pull request needed and what does it do?

Fixes a data race in plugin/file/tree where Elem.Name() lazily wrote the cached owner name without synchronization. Under high concurrency (e.g., clouddns lookups), this could produce torn Go strings and cause a panic in github.com/miekg/dns.PrevLabel while comparing names.

The fix is to eagerly set the name in newElem() and make Name() read-only. The cached name is set once and immutable. An alternative would be a mutex on Name() but it adds lock overhead right in the hot comparator path.

Also added a few tests to cover concurrent access like observed here. If the test TestLess_ConcurrentNameAccess is run against master you'll see a data race.

Details WARNING: DATA RACE Read at 0x00c00000c338 by goroutine 11: github.com/coredns/coredns/plugin/file/tree.(*Elem).Name() /git/coredns/plugin/file/tree/elem.go:59 +0x38 github.com/coredns/coredns/plugin/file/tree.Less() /git/coredns/plugin/file/tree/elem.go:101 +0x30 github.com/coredns/coredns/plugin/file/tree.TestLess_ConcurrentNameAccess.func1() /git/coredns/plugin/file/tree/less_test.go:101 +0x74 Previous write at 0x00c00000c338 by goroutine 13: github.com/coredns/coredns/plugin/file/tree.(*Elem).Name() /git/coredns/plugin/file/tree/elem.go:63 +0x10c github.com/coredns/coredns/plugin/file/tree.Less() /git/coredns/plugin/file/tree/elem.go:101 +0x30 github.com/coredns/coredns/plugin/file/tree.TestLess_ConcurrentNameAccess.func1() /git/coredns/plugin/file/tree/less_test.go:101 +0x74 Goroutine 11 (running) created at: github.com/coredns/coredns/plugin/file/tree.TestLess_ConcurrentNameAccess() /git/coredns/plugin/file/tree/less_test.go:98 +0x204 testing.tRunner() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1934 +0x164 testing.(*T).Run.gowrap1() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1997 +0x3c Goroutine 13 (finished) created at: github.com/coredns/coredns/plugin/file/tree.TestLess_ConcurrentNameAccess() /git/coredns/plugin/file/tree/less_test.go:98 +0x204 testing.tRunner() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1934 +0x164 testing.(*T).Run.gowrap1() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1997 +0x3c ================== --- FAIL: TestLess_ConcurrentNameAccess (0.00s) testing.go:1617: race detected during execution of test

Also two additional CloudDNS tests are added:

  • TestCloudDNSConcurrentServeDNS: Run concurrent zone updates while concurrent queries.
Details ================== WARNING: DATA RACE Write at 0x00c00061c350 by goroutine 27: github.com/coredns/coredns/plugin/file/tree.(*Elem).Name() /git/coredns/plugin/file/tree/elem.go:63 +0x10c github.com/coredns/coredns/plugin/file/tree.Less() /git/coredns/plugin/file/tree/elem.go:101 +0x30 github.com/coredns/coredns/plugin/file/tree.(*Node).search() /git/coredns/plugin/file/tree/tree.go:155 +0x54 github.com/coredns/coredns/plugin/file/tree.(*Tree).Search() /git/coredns/plugin/file/tree/tree.go:145 +0x58 github.com/coredns/coredns/plugin/file.(*Zone).Lookup() /git/coredns/plugin/file/lookup.go:96 +0x680 github.com/coredns/coredns/plugin/clouddns.(*CloudDNS).ServeDNS() /git/coredns/plugin/clouddns/clouddns.go:126 +0x464 github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentServeDNS.func2() /git/coredns/plugin/clouddns/clouddns_test.go:377 +0x2ec github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentServeDNS.gowrap1() /git/coredns/plugin/clouddns/clouddns_test.go:387 +0x40 Previous read at 0x00c00061c350 by goroutine 36: github.com/coredns/coredns/plugin/file/tree.(*Elem).Name() /git/coredns/plugin/file/tree/elem.go:64 +0x148 github.com/coredns/coredns/plugin/file/tree.Less() /git/coredns/plugin/file/tree/elem.go:101 +0x30 github.com/coredns/coredns/plugin/file/tree.(*Node).search() /git/coredns/plugin/file/tree/tree.go:155 +0x54 github.com/coredns/coredns/plugin/file/tree.(*Tree).Search() /git/coredns/plugin/file/tree/tree.go:145 +0x58 github.com/coredns/coredns/plugin/file.(*Zone).Lookup() /git/coredns/plugin/file/lookup.go:96 +0x680 github.com/coredns/coredns/plugin/clouddns.(*CloudDNS).ServeDNS() /git/coredns/plugin/clouddns/clouddns.go:126 +0x464 github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentServeDNS.func2() /git/coredns/plugin/clouddns/clouddns_test.go:377 +0x2ec github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentServeDNS.gowrap1() /git/coredns/plugin/clouddns/clouddns_test.go:387 +0x40 Goroutine 27 (running) created at: github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentServeDNS() /git/coredns/plugin/clouddns/clouddns_test.go:370 +0x5d4 testing.tRunner() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1934 +0x164 testing.(*T).Run.gowrap1() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1997 +0x3c Goroutine 36 (running) created at: github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentServeDNS() /git/coredns/plugin/clouddns/clouddns_test.go:370 +0x5d4 testing.tRunner() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1934 +0x164 testing.(*T).Run.gowrap1() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1997 +0x3c ================== --- FAIL: TestCloudDNSConcurrentServeDNS (0.04s) testing.go:1617: race detected during execution of test
  • TestCloudDNSConcurrentLookupNameCache: Run concurrent queries and lookups, stressing the path even more.
Details ================== WARNING: DATA RACE Read at 0x00c00000ec98 by goroutine 61: github.com/coredns/coredns/plugin/file/tree.(*Elem).Name() /git/coredns/plugin/file/tree/elem.go:59 +0x38 github.com/coredns/coredns/plugin/file/tree.Less() /git/coredns/plugin/file/tree/elem.go:101 +0x30 github.com/coredns/coredns/plugin/file/tree.(*Node).search() /git/coredns/plugin/file/tree/tree.go:155 +0x54 github.com/coredns/coredns/plugin/file/tree.(*Tree).Search() /git/coredns/plugin/file/tree/tree.go:145 +0x58 github.com/coredns/coredns/plugin/file.(*Zone).Lookup() /git/coredns/plugin/file/lookup.go:96 +0x680 github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentLookupNameCache.func1() /git/coredns/plugin/clouddns/clouddns_test.go:437 +0x110 github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentLookupNameCache.gowrap1() /git/coredns/plugin/clouddns/clouddns_test.go:439 +0x50 Previous write at 0x00c00000ec98 by goroutine 86: github.com/coredns/coredns/plugin/file/tree.(*Elem).Name() /git/coredns/plugin/file/tree/elem.go:63 +0x10c github.com/coredns/coredns/plugin/file/tree.Less() /git/coredns/plugin/file/tree/elem.go:101 +0x30 github.com/coredns/coredns/plugin/file/tree.(*Node).search() /git/coredns/plugin/file/tree/tree.go:155 +0x54 github.com/coredns/coredns/plugin/file/tree.(*Tree).Search() /git/coredns/plugin/file/tree/tree.go:145 +0x58 github.com/coredns/coredns/plugin/file.(*Zone).Lookup() /git/coredns/plugin/file/lookup.go:96 +0x680 github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentLookupNameCache.func1() /git/coredns/plugin/clouddns/clouddns_test.go:437 +0x110 github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentLookupNameCache.gowrap1() /git/coredns/plugin/clouddns/clouddns_test.go:439 +0x50 Goroutine 61 (running) created at: github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentLookupNameCache() /git/coredns/plugin/clouddns/clouddns_test.go:429 +0x604 testing.tRunner() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1934 +0x164 testing.(*T).Run.gowrap1() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1997 +0x3c Goroutine 86 (running) created at: github.com/coredns/coredns/plugin/clouddns.TestCloudDNSConcurrentLookupNameCache() /git/coredns/plugin/clouddns/clouddns_test.go:429 +0x604 testing.tRunner() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1934 +0x164 testing.(*T).Run.gowrap1() /opt/homebrew/Cellar/go/1.25.1/libexec/src/testing/testing.go:1997 +0x3c ================== --- FAIL: TestCloudDNSConcurrentLookupNameCache (0.05s) testing.go:1617: race detected during execution of test

These both panic on master branch.

2. Which issues (if any) are related?

Fixes #7561.

3. Which documentation changes (if any) need to be made?

None.

4. Does this introduce a backward incompatible change or deprecation?

No.

@codecov

codecov Bot commented Sep 26, 2025

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.60%. Comparing base (93c57b6) to head (c84cf13).
⚠️ Report is 1661 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7574      +/-   ##
==========================================
+ Coverage   55.70%   62.60%   +6.90%     
==========================================
  Files         224      274      +50     
  Lines       10016    18329    +8313     
==========================================
+ Hits         5579    11475    +5896     
- Misses       3978     6187    +2209     
- Partials      459      667     +208     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@thevilledev thevilledev force-pushed the fix/file-plugin-tree-lazy-write branch from 2bcc531 to a11aa6c Compare September 27, 2025 11:42
Eagerly set name in newElem and make Name() read-only to avoid
racy lazy writes under concurrent lookups. Add tests for empty-name
comparisons and concurrent access to Less/Name(). In addition,
regression tests to CloudDNS plugin.

Signed-off-by: Ville Vesilehto <ville@vesilehto.fi>
@thevilledev thevilledev force-pushed the fix/file-plugin-tree-lazy-write branch from a11aa6c to c84cf13 Compare September 27, 2025 12:00
@yongtang yongtang merged commit 70fb03f into coredns:master Sep 29, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recovered from panic in server: "dns://:53" runtime error: invalid memory address or nil pointer dereference

2 participants