incorrect handling of concurrent duplicate QueueLeaf calls with mysql storage backend #3603
Description
Scenario
If two callers concurrently invoke the QueueLeaf rpc proposing that the same entry be integrated into the log: both transactions in log_server will get to this point, and whichever transaction ends up going second will see a duplicateErr raised from mysql (since the primary keys in LeafData are TreeId and LeafIdentityHash).
If the second transaction (the one that sees a duplicate entry in the LeafData table) falls through to
trillian/storage/mysql/log_storage.go
Line 486 in 433d79f
SequencedLeafData, the SQL query will not return the full number of rows (i.e. it deduplicates it in the response prematurely), and therefore throws the error seen on trillian/storage/mysql/log_storage.go
Line 491 in 433d79f
Expected Behavior
I'd expect the rpc call to instead succeed with a status.Code of AlreadyExists.
Steps to reproduce
This is reproducible by adding a test case to log_storage_test.go:
diff --git a/storage/mysql/log_storage_test.go b/storage/mysql/log_storage_test.go
index d0cfc50e..62e97eae 100644
--- a/storage/mysql/log_storage_test.go
+++ b/storage/mysql/log_storage_test.go
@@ -139,6 +139,7 @@ func TestQueueDuplicateLeaf(t *testing.T) {
leaves := createTestLeaves(int64(count), 10)
leaves2 := createTestLeaves(int64(count), 12)
leaves3 := createTestLeaves(3, 100)
+ leaves4 := createTestLeaves(3, 105)
// Note that tests accumulate queued leaves on top of each other.
tests := []struct {
@@ -161,6 +162,11 @@ func TestQueueDuplicateLeaf(t *testing.T) {
leaves: []*trillian.LogLeaf{leaves[0], leaves3[0], leaves[1], leaves3[1], leaves[2]},
want: []*trillian.LogLeaf{leaves[0], nil, leaves[1], nil, leaves[2]},
},
+ {
+ desc: "[10, 10, 11, 11, 12]",
+ leaves: []*trillian.LogLeaf{leaves4[0], leaves4[0], leaves4[1], leaves4[1], leaves4[2]},
+ want: []*trillian.LogLeaf{leaves4[0], leaves4[0], leaves4[1], leaves4[1], leaves4[2]},
+ },
}