-
Notifications
You must be signed in to change notification settings - Fork 4.1k
sql: OOM during TPCC with schema changes #69317
Description
Describe the problem
This is meta issue to clearly phrase observations from debugging #69100 and #69086. In both of those we see OOMs. We've tracked this down to a change in prepared statement handling due to #68608. We observe that pgx is issuing lots of DEALLOCATE commands, presumably because of something involving its LRU. However, I did try increasing the LRU size to no avail.
https://github.com/jackc/pgconn/blob/53f5fed36c570f0b5c98d6ec2415658c7b9bd11c/stmtcache/lru.go#L118
The issue is not about the DEALLOCATE or cache misses but rather about the OOM that seems to come with it.
To Reproduce
Run schemachange/index/tpcc/w=1000 at 7c36a9d and it seems to happen every time.
Additional data / screenshots
One observation is that we have a lot of allocations in the optimizer and the in numeric constants. These numeric constants hold on to strings which are backed by the pgwire buffer reader. That might be part of the problem and might point to why we don't see the data in the heap profile. However, there's still the question of why we're holding on to all of these. Maybe it's related to the statements metrics.
Another point of confusion is the extent to which the schema changes are actually relevant here. Is it the straw that breaks the camels back?
Jira issue: CRDB-9542
Metadata
Metadata
Assignees
Labels
Type
Projects
Status