Skip to content

sql: Django's table introspection query is slow (seconds to minutes, depending on number of tables) #57924

@timgraham

Description

@timgraham

What is your situation?

Django uses this query for table introspection:

SELECT
    a.attname AS column_name,
    NOT (a.attnotnull OR ((t.typtype = 'd') AND t.typnotnull)) AS is_nullable,
    pg_get_expr(ad.adbin, ad.adrelid) AS column_default
FROM pg_attribute AS a
LEFT JOIN pg_attrdef AS ad ON (a.attrelid = ad.adrelid) AND (a.attnum = ad.adnum)
JOIN pg_type AS t ON a.atttypid = t.oid JOIN pg_class AS c ON a.attrelid = c.oid
JOIN pg_namespace AS n ON c.relnamespace = n.oid
WHERE (
    (
        (c.relkind IN ('f', 'm', 'p', 'r', 'v')) AND
        (c.relname = '<target table>')
    ) AND (n.nspname NOT IN ('pg_catalog', 'pg_toast'))
) AND pg_table_is_visible(c.oid)

Observed performance

The query takes 2-5 seconds with tens of tables or several minutes with hundreds of tables. The Django test suite issues the query around 175 times totaling around 8 minutes of the test suite's 68 minutes run time.

Also, it's currently infeasible to run Django's test suite all at once because the query takes about 4 minutes if the ~1400 tables for all of the test suite are present.

Build Tag: v21.1.0-alpha.1-289-g960b4cfc54
Build Time: 2020/12/12 06:45:40
Build Commit ID: 960b4cf

Requested resolution

  • I want CockroachDB to be optimized for my use case.

Metadata

Metadata

Assignees

Labels

A-sql-builtinsSQL built-in functions and semantics thereof.C-investigationFurther steps needed to qualify. C-label will change.O-communityOriginated from the communityX-blathers-triagedblathers was able to find an owner

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions