Skip to content

sql: OOM during TPCC with schema changes #69317

@ajwerner

Description

@ajwerner

Describe the problem

This is meta issue to clearly phrase observations from debugging #69100 and #69086. In both of those we see OOMs. We've tracked this down to a change in prepared statement handling due to #68608. We observe that pgx is issuing lots of DEALLOCATE commands, presumably because of something involving its LRU. However, I did try increasing the LRU size to no avail.

https://github.com/jackc/pgconn/blob/53f5fed36c570f0b5c98d6ec2415658c7b9bd11c/stmtcache/lru.go#L118

The issue is not about the DEALLOCATE or cache misses but rather about the OOM that seems to come with it.

To Reproduce

Run schemachange/index/tpcc/w=1000 at 7c36a9d and it seems to happen every time.

Additional data / screenshots

One observation is that we have a lot of allocations in the optimizer and the in numeric constants. These numeric constants hold on to strings which are backed by the pgwire buffer reader. That might be part of the problem and might point to why we don't see the data in the heap profile. However, there's still the question of why we're holding on to all of these. Maybe it's related to the statements metrics.

Another point of confusion is the extent to which the schema changes are actually relevant here. Is it the straw that breaks the camels back?

Jira issue: CRDB-9542

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.T-sql-queriesSQL Queries Teambranch-masterFailures and bugs on the master branch.

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions