Summary

This issue documents the original concurrency limitation in the prepared‑statement caching subsystem and the new refactor that improves scalability by removing locking operations (global map lookup) from the hot path. The goal of the change is to enable significantly higher throughput and lower latency under concurrent workloads, while simplifying lifetime management and reducing memory usage.

The previous approach was correct and reliable, but relied on a global shared map that required locking for lookups. The new design evolves this architecture by giving each client and backend connection a direct, reference-counted handle to global statement metadata, eliminating the need for global map access during Bind/Describe/Execute/Close.

Old Behavior

The system maintained:

• A per-client connection map for client statement names mapped to global statement id. (and vice-versa)
• A Backend connection map for backend statement ids mapped to global statement id. (and vice-versa)
• A global statement map containing query hash mapped to prepared statement metadata.

During Parse / Bind / Describe / Execute operations, the server followed this sequence:

• Look up the client connection map.
• Look up the global shared map to retrieve metadata.

Because the global map was accessed by many threads, each lookup—read or write—required synchronization. Under high concurrency, this introduced contention and limited scalability.

This behavior was fully correct, but it imposed overhead on extremely frequent operations.

New Design — Lock-Free Hot Path

The refactor improves concurrency and simplifies the design by:

1. Storing a direct, reference-counted handle to global metadata in each client and backend map entry

Instead of performing a second lookup into the global map, the client and backend map can now provide metadata directly.

Parse still requires a global map lookup if an entry is not found in the client map.

2. Hot-path operations no longer acquire locks

Bind / Describe / Execute / Close now:

• Perform a single client-map lookup
• Access the referenced metadata directly
• Avoid the global map entirely
• Avoid all global locking

Making the hot path fully lock-free.

3. Lifetime and purge handled via the reference-counted ownership model

The new model uses reference-counted handles to determine when metadata is still in active use. When no client or backend connection holds a reference, metadata can be safely removed from global map.

This enhances safety and removes the need for manual refcount adjustments.

4. Metrics updates are now lock-free

5. Memory usage reduced

New desing have made several structures redundant and are no longer required:

• global_id_to_stmt_names map in client connection.
• map_stmt_id_to_info in the global map

This reduces per-statement and per-connection overhead.