feat: CHAR(36) UUID primary keys for collision-free Dolt federation#2575
Closed
trillium wants to merge 1 commit into
Closed
feat: CHAR(36) UUID primary keys for collision-free Dolt federation#2575trillium wants to merge 1 commit into
trillium wants to merge 1 commit into
Conversation
Replace BIGINT AUTO_INCREMENT with CHAR(36) UUID primary keys on 6 tables (events, comments, issue_snapshots, compaction_snapshots, wisp_events, wisp_comments) to eliminate duplicate PK collisions during Dolt federation. - Change Event.ID and Comment.ID from int64 to string - Change GetAllEventsSince to use time.Time instead of int64 high-water marks - Add migration 010 to convert existing BIGINT PKs to UUID - Remove resetAutoIncrements band-aid (no longer needed) - Generate UUIDs in Go before INSERT for comments (ImportIssueComment) - Schema version bumped from 6 to 7
Contributor
|
Cherry-picked to main — thanks @trillium! Exceptional work on this PR. The AUTO_INCREMENT collision problem is real and well-documented, and UUID v7 keys follow DoltHub's own recommendation for multi-clone scenarios. The migration is clean and idempotent. Welcome contribution for the Wasteland federation story. If you want to continue with the remaining 58 files from your full branch, we'd welcome a follow-up PR. |
This was referenced Mar 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Dolt's
AUTO_INCREMENTcounter is per-server-instance with no cross-clone reconciliation. When multiple Dolt clones independently write to the same table and then sync viadolt_push/dolt_pull, INSERT operations fail with:This is documented behavior:
Six tables in beads use
BIGINT AUTO_INCREMENT PRIMARY KEYand are all vulnerable in any multi-clone deployment.How it happens
Impact
bd create,bd update,bd closefail on any node whose counter is behindPrior mitigation (insufficient)
beads#2133 added
ALTER TABLE <tbl> AUTO_INCREMENT = MAX(id)+1after everyDOLT_PULL(resetAutoIncrements). This works whenbddoes the pull, but fails when external sync scripts use rawdolt_pull/dolt_push, when other tools write independently, or when two nodes write concurrently. Even within a single server, dolthub/dolt#7702 documents AUTO_INCREMENT race conditions.Solution
Replace
BIGINT AUTO_INCREMENT PRIMARY KEYwithCHAR(36) NOT NULL PRIMARY KEY DEFAULT (UUID())on all six affected tables, following DoltHub's official recommendation for multi-clone scenarios.Why UUID v7
Where the application generates IDs explicitly (e.g.,
ImportIssueComment), we use UUID v7 (RFC 9562) — time-sorted UUIDs that preserve chronological ordering (ORDER BY id≈ creation order).Tables affected
eventscommentsbd commentissue_snapshotscompaction_snapshotswisp_eventswisp_commentsMigration strategy
Dolt cannot
ALTER COLUMNto change a PK's type in place. Migration 010 uses a 4-step pattern for each table:char(36)or table doesn't existALTER TABLE ADD COLUMN uuid_id CHAR(36) NOT NULL DEFAULT (UUID())UPDATE SET uuid_id = UUID()MODIFY id BIGINT NOT NULL(remove AUTO_INCREMENT first — Dolt requires this before DROP PRIMARY KEY),DROP PRIMARY KEY,DROP COLUMN idRENAME COLUMN uuid_id TO id,ADD PRIMARY KEY (id)Dolt DDL quirk
ALTER TABLE ... DROP PRIMARY KEYfails on Dolt when the column hasAUTO_INCREMENT. The workaround is toMODIFYthe column first to remove it. This is not documented in Dolt's migration docs.Breaking changes
GetAllEventsSincesignature:sinceID int64→since time.Time(UUIDs aren't sequential, timestamps are the natural cursor)Event.IDandComment.IDchanged fromint64tostringresetAutoIncrementsremoved — root cause fixed, band-aid unnecessaryTesting
This change passes the existing test suite — test modifications are minimal (signature updates and
int64→stringassertions):dolt_test.go:comment1.ID == 0→comment1.ID == ""(string zero-value check)tracker_test.go:GetAllEventsSincemock updated to accepttime.TimeScope of this PR and remaining work
This PR is the minimal core change — the schema migration, type updates, and interface changes needed to make UUID primary keys work. It compiles, passes tests, and functions correctly.
However, a complete adoption would benefit from updating additional files that still reference or assume the old AUTO_INCREMENT behavior. I'm pausing here to see if this direction is desired before doing more work. The files below were updated in my full working branch but are not included in this PR to keep the diff reviewable:
Files touched in the full change (58 additional files, click to expand)
Deleted — dead code after UUID migration:
internal/storage/issueops/create.go— inlined intoDoltStore.CreateIssueinternal/storage/issueops/helpers.go— inlined intoDoltStoreinternal/storage/dolt/credentials.go— credential CLI routing (removed)internal/storage/dolt/credentials_test.gointernal/storage/dolt/federation_test.gointernal/storage/dolt/git_remote_test.gointernal/storage/embeddeddolt/blocked.go— embedded backend simplified to stubsinternal/storage/embeddeddolt/config_metadata.gointernal/storage/embeddeddolt/create_issue.gointernal/storage/embeddeddolt/create_issue_test.gointernal/storage/embeddeddolt/flock.gointernal/storage/embeddeddolt/flock_stub.gointernal/storage/embeddeddolt/statistics.gointernal/storage/embeddeddolt/version_control.gocmd/bd/backup_export_git.go— git export backup removedcmd/bd/backup_export_git_test.gocmd/bd/doctor/tracked_runtime.gocmd/bd/doctor/tracked_runtime_test.gocmd/bd/init_embedded_test.gocmd/bd/simple_helpers_test.gocmd/bd/prime_test.godocs/GIT_INTEGRATION.mdModified — stale references, inlined logic, cleanup:
internal/storage/dolt/issues.go—CreateIssueinlined fromissueopsinternal/storage/dolt/wisps.go—createWispinlined, UUID PKsinternal/storage/dolt/federation.go— credential CLI routing removedinternal/storage/dolt/adaptive_length.go— new: birthday-paradox ID length scalinginternal/storage/embeddeddolt/store.go— simplified to stubsinternal/storage/embeddeddolt/schema.gointernal/config/config.gointernal/beads/beads.gointernal/beads/beads_test.gocmd/bd/init.go,cmd/bd/init_contributor.go,cmd/bd/init_team.go,cmd/bd/init_git_hooks.gocmd/bd/backup.go,cmd/bd/backup_git.gocmd/bd/doctor.go,cmd/bd/doctor_fix.go,cmd/bd/doctor_test.gocmd/bd/dolt.go,cmd/bd/import_shared.go,cmd/bd/list.go,cmd/bd/prime.gocmd/bd/store_factory.go,cmd/bd/store_factory_embedded.gorename_test.go,schema_parity_test.go,wisp_gc_test.go,wisp_validation_test.go,backup_auto_test.goOf particular note:
issueops/create.gohas a comment still referencing "auto-increment PK" (line 278) andPersistCommentsworks by coincidence — it omits theidcolumn from INSERT, soDEFAULT(UUID())handles it. Correct behavior, but the code isn't aware of the change.Request for review
This is a significant schema change and we want to make sure we haven't missed anything. We'd especially appreciate a second set of eyes on:
time.Timehigh-water mark inGetAllEventsSincehandles all the scenarios the oldint64cursor did