feat(synced): add SetIfNotExistsMulti for single-lock bulk conditional insert#114
Merged
Conversation
…l insert SetIfNotExists acquires the write-lock and releases it once per key. For bucket-affinity bulk inserts where a single goroutine has pre-partitioned its workload to a single bucket's map, calling SetIfNotExists in a loop incurs N Lock/Unlock cycles where 1 would suffice. On uncontended paths the acquire/release pair is still measurable per-call work. Add SetIfNotExistsMulti(keys, values) that takes the lock once, walks both slices in parallel, and inserts each pair via setUnlocked when the key is absent. Returns a []bool of length min(len(keys), len(values)) where wasInserted[i] mirrors the (V, bool) return of SetIfNotExists for the i-th pair — true if newly added, false if the key already existed. Edge cases covered by tests: mismatched slice lengths (truncates to the shorter), duplicate keys within a single call (first wins), nil/empty input returns an empty bool slice. The new function is additive; no existing API changes.
|
icellan
added a commit
to bsv-blockchain/teranode
that referenced
this pull request
May 15, 2026
PutMultiBucketTxInpoints previously delegated to per-entry SyncedMap.SetIfNotExists, which acquires and releases the SyncedMap's internal RWMutex on every call. In the bucket-affinity hot path each worker owns one or more buckets exclusively, so the lock is never contended across workers — but the per-entry Lock/Unlock pair is still measurable work (profiling attributed ~17 % of every SetIfNotExists call to Lock/Unlock paths). go-tx-map upstream now exposes SetIfNotExistsMulti (see bsv-blockchain/go-tx-map#114) which takes the write-lock once for a batch of (key, value) pairs and walks both slices, inserting each pair via setUnlocked iff the key is absent. Per-element wasInserted semantics match SetIfNotExists exactly. This change: - Bumps go-tx-map to a pseudo-version pinned to commit e509e98cad on go-tx-map#114. go.mod carries a multi-line comment explaining why we are on a pseudo-version and that it should be replaced with the next tagged release (>= v1.3.6) once the upstream PR ships. - Switches PutMultiBucketTxInpoints to parallel-slice signature (keys []Hash, values []*TxInpoints) so it can pass through to the new bulk method without re-packing. Drops the unused TxInpointsEntry helper struct. - Updates the sole caller bucketShardedGetAndSetIfNotExists to build the parallel slices directly while iterating its bucket-local indices. - Updates the corresponding unit test to use the new signature via a small testInpointPair helper (renamed and tidied so the inline struct literals don't bloat the assertions). No semantics change. Existing tests in services/blockassembly/subtreeprocessor continue to pass under -race; staticcheck and golangci-lint on the package report zero issues.
icellan
added a commit
to bsv-blockchain/teranode
that referenced
this pull request
May 15, 2026
PutMultiBucketTxInpoints previously delegated to per-entry SyncedMap.SetIfNotExists, which acquires and releases the SyncedMap's internal RWMutex on every call. In the bucket-affinity hot path each worker owns one or more buckets exclusively, so the lock is never contended across workers — but the per-entry Lock/Unlock pair is still measurable work (profiling attributed ~17 % of every SetIfNotExists call to Lock/Unlock paths). go-tx-map upstream now exposes SetIfNotExistsMulti (see bsv-blockchain/go-tx-map#114) which takes the write-lock once for a batch of (key, value) pairs and walks both slices, inserting each pair via setUnlocked iff the key is absent. Per-element wasInserted semantics match SetIfNotExists exactly. This change: - Bumps go-tx-map to a pseudo-version pinned to commit e509e98cad on go-tx-map#114. go.mod carries a multi-line comment explaining why we are on a pseudo-version and that it should be replaced with the next tagged release (>= v1.3.6) once the upstream PR ships. - Switches PutMultiBucketTxInpoints to parallel-slice signature (keys []Hash, values []*TxInpoints) so it can pass through to the new bulk method without re-packing. Drops the unused TxInpointsEntry helper struct. - Updates the sole caller bucketShardedGetAndSetIfNotExists to build the parallel slices directly while iterating its bucket-local indices. - Updates the corresponding unit test to use the new signature via a small testInpointPair helper (renamed and tidied so the inline struct literals don't bloat the assertions). No semantics change. Existing tests in services/blockassembly/subtreeprocessor continue to pass under -race; staticcheck and golangci-lint on the package report zero issues.
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
SyncedMap.SetIfNotExistsMulti(keys, values) []bool— a bulk conditional-insert variant that takes the write-lock once for the whole batch instead of N times.Why
The existing single-key
SetIfNotExistsacquires the internalRWMutex, checks, optionally inserts, and releases. For callers that have already pre-partitioned their workload so a single goroutine owns a single SyncedMap shard (bucket-affinity bulk insert pattern), callingSetIfNotExistsin a loop is correct but wasteful: each call does a separate Lock/Unlock even though the lock is held by the same goroutine across the loop.This shape is common in sharded-map wrappers. For example, the consumer in teranode block-assembly partitions ~30M tx hashes across 4096-16384 buckets and dispatches one goroutine per non-empty bucket. With the per-call lock pattern, profiling shows ~17 % of CPU inside
Lock/Unlockeven with no cross-worker contention — purely the per-call acquire/release overhead.A single-lock bulk variant eliminates that overhead while preserving exact per-key SetIfNotExists semantics.
API
Lock. For eachi, ifkeys[i]is absent insertsvalues[i]via the existingsetUnlocked(so the configuredlimit-based eviction still applies); otherwise leaves the existing value untouched.[]boolof lengthmin(len(keys), len(values));wasInserted[i]istrueif newly added,falseif the key already existed — mirrors the(V, bool)return shape of single-keySetIfNotExists.Edge-case behaviour (covered by tests)
len(values) < len(keys)len(values); trailing keys ignored.len(keys) < len(values)len(keys); trailing values ignored.[]bool{}.wasInserted=false.Test plan
go build ./...cleango vet ./...cleango test -race ./...passes (existing + new tests)TestSyncedMapSetIfNotExistsMultiwith five subtests covering the cases aboveNotes
setUnlockedhelper already exists and is reused as the per-element insert primitive, so the limit-based eviction logic is shared.