Overview

The CI MongoDB replica set runs with the default maxTransactionLockRequestTimeoutMillis of 5ms. Under CPU contention, a transient lock hold during seed/onInit (for example, concurrent version writes) pushes a transaction past 5ms, so it fails with a TransientTransactionError (LockTimeout, code 24) instead of waiting. That crashes onInit and takes down the whole E2E suite: every test and retry then times out against a dead server.

This raises the lock timeout so transient contention is absorbed.

Key Changes

• Raise the mongod lock timeout in the CI entrypoint
  • Add --setParameter maxTransactionLockRequestTimeoutMillis=100 to both long-running mongod starts in the MongoDB docker-compose entrypoint (the already-initialized path and the first-run start-with-auth path).
  • The temporary no-auth mongod used only to initialize the replica set and users is left unchanged, since it is killed before the application connects and runs no application transactions.

Design Decisions

• Why an infra-level timeout rather than retry-in-code. The failure is a transient lock contention, exactly the case the lock-wait timeout governs. Widening the wait window is targeted and touches no production code path. Retrying TransientTransactionError in the seed/adapter layer is the alternative, but it carries more risk and changes runtime behavior for all consumers.
• Why 100ms. Twenty times the 5ms default, enough headroom to ride out a brief lock hold under a loaded CI runner, while still short enough to fail fast on a genuine deadlock.
• Verification limits. The crash only reproduces under CI CPU starvation and cannot be triggered locally. Its signal is the absence of the LockTimeout-at-onInit signature over future runs, so confidence accrues over time rather than from a single red-to-green flip.

Overall Flow

sequenceDiagram
    participant Init as Payload onInit (seed)
    participant Mongo as MongoDB (CI replica set)

    Note over Init,Mongo: Before
    Init->>Mongo: insertOne into _versions (in transaction)
    Mongo-->>Init: IX lock busy, give up after 5ms (TransientTransactionError)
    Init--xInit: onInit crashes, entire suite fails

    Note over Init,Mongo: After
    Init->>Mongo: insertOne into _versions (in transaction)
    Mongo-->>Init: lock acquired within 100ms window
    Init->>Init: seed completes, suite runs

References / Links

• MongoDB: maxTransactionLockRequestTimeoutMillis