Unable to initialize DB: duplicate key value violates unique constraint "tag_postids_tagid_idx" #39
Description
Hi there. First: thank you for your work on this project.
Unfortunately, I've been encountering an error when rebuilding the database this past week. Log output is below. Might be a consequence of excessive tag sibling/parent loops.
I addressed it by adding ON CONFLICT DO NOTHING to applyTagSiblings, which matches all of the other INSERT INTO ... SELECT ... FROM ... statements in tag-related functions in that file.
In a couple days, if this turns out to be a sufficient fix, I can make a PR if you'd like.
Log output
Posts (153553/153553) [####################] Done in 3.24s
Posts (3299/3299) [####################] Done in 45ms
Posts (2470/2470) [####################] Done in 61ms
Posts (628/628) [####################] Done in 25ms
Urls (601716/601716) [####################] Done in 2.02s
Notes [####################] Done in 0ms
Tags (39961003/39961003) [####################] Done in 2:04
Mappings (2031885282/2031885282)[####################] Done in 53:36
Mappings (1205433/1205433) [####################] Done in 19.84s
Mappings (45176/45176) [####################] Done in 253ms
Tag parents (29235/29235) [####################] Done in 156ms
Tag siblings (302637/302637) [####################] Done in 1.59s
Importing options... [####################] Done in 12ms
Resolving file relations... [####################] Done in 103ms
Normalizing tags (6/6) [####################] Done in 46:12:14
Warning! Detected 71 loops in tag siblings!
Warning! Detected 46 loops in tag parents!
Resolving tag parents... [####################] Done in 36:40:41
Indexing (24/24) [####################] Done in 3:25:06
Resolving tag siblings... [--------------------] Unable to initialize DB!
error: duplicate key value violates unique constraint "tag_postids_tagid_idx"
at Parser.parseErrorMessage (/repos/hydribooru/node_modules/pg-protocol/src/parser.ts:369:69)
at Parser.handlePacket (/repos/hydribooru/node_modules/pg-protocol/src/parser.ts:188:21)
at Parser.parse (/repos/hydribooru/node_modules/pg-protocol/src/parser.ts:103:30)
at Socket.<anonymous> (/repos/hydribooru/node_modules/pg-protocol/src/index.ts:7:48)
at Socket.emit (node:events:514:28)
at addChunk (node:internal/streams/readable:376:12)
at readableAddChunk (node:internal/streams/readable:349:9)
at Socket.Readable.push (node:internal/streams/readable:286:10)
at Pipe.onStreamRead (node:internal/stream_base_commons:190:23) {
length: 215,
severity: 'ERROR',
code: '23505',
detail: 'Key (tagid)=(X) already exists.',
hint: undefined,
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: 'public',
table: 'tag_postids',
column: undefined,
dataType: undefined,
constraint: 'tag_postids_tagid_idx',
file: 'nbtinsert.c',
line: '663',
routine: '_bt_check_unique'
}