Skip to content

sql: COPY seemingly spends a lot of time in the optimizer #52071

@otan

Description

@otan

Trying out the osm2pgsql tool, which utilizes copies. Thus far I have found it spends most of its time in the optimizer trying to copy over ~10M rows (~4 of these in parallel). Here is a CPU profile over 10s during the COPY period:

image

Repro steps:

Processing: Node(6965k 6965.0k/s) Way(356k 14.24k/s) Relation(5290 661.2/s)

PostgreSQL is able to do this in ~7s, we take ~49s. (maybe fodder for https://accidentallyquadratic.tumblr.com/)

Jira issue: CRDB-3984

Epic CRDB-14477
Epic CRDB-14916

Metadata

Metadata

Assignees

Labels

A-sql-optimizerSQL logical planning and optimizations.C-investigationFurther steps needed to qualify. C-label will change.T-sql-foundationsSQL Foundations Team (formerly SQL Schema + SQL Sessions)

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions