sql,tree: make ::REGPROC cast efficient#61211
Conversation
rafiss
left a comment
There was a problem hiding this comment.
awesome! i think there are enough existing tests for correctness, but could you add a benchmark similar to this one using that activerecord query: https://github.com/cockroachdb/cockroach/pull/59880/files#diff-389fae48c6f52a8edcfdb2f3ef5780fb9e349bfad03480d6271470977a24f2d0
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @jordanlewis)
pkg/sql/sem/tree/casts.go, line 1279 at r1 (raw file):
case oid.T_regtype: // Mapping an oid to a regtype is easy: we have a hardcoded map. typ, ok := types.OidToType[oid.Oid(v.DInt)]
i guess this is separate from your PR, but does this work for user-defined enum types?
pkg/sql/sem/tree/casts.go, line 1291 at r1 (raw file):
// from integer to procedure name. // Mapping an oid to a regtype is easy: we have a hardcoded map. name, ok := OidToBuiltinName[int(v.DInt)]
just curious, is there a way we can warn ourselves in the future if we do support user-defined functions?
pkg/sql/sem/tree/casts.go, line 1299 at r1 (raw file):
return ret, nil default:
is the default case still used anywhere? it would be nice to remove the queryOid function altogether if not!
pkg/sql/sem/tree/casts.go, line 1315 at r1 (raw file):
case oid.T_oid: return &DOid{semanticType: t, DInt: i}, nil default:
do we need to look out for this case too?
pkg/sql/sem/tree/function_definition.go, line 165 at r1 (raw file):
// to their name. We populate this from the pg_catalog.go file in the sql // package because of dependency issues: we can't use oidHasher from this file. var OidToBuiltinName map[int]string
nit: could it be map[uint32]string ?
pkg/sql/sem/tree/overload.go, line 91 at r1 (raw file):
// Oid is the cached oidHasher.BuiltinOid result for this Overload. It's // populated at init-time. Oid DInt
nit: shouldn't the type here be DOid?
i think we want to move in a direction where we are more consistent in the code about the fact that OIDs are always uint32s, so i hesitate to make this a DInt
26eb9f9 to
66a0a53
Compare
jordanlewis
left a comment
There was a problem hiding this comment.
Added a test, but not sure how to adjust the expectations file (and that test seems to be skipped too): #57771
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @jordanlewis and @rafiss)
pkg/sql/sem/tree/casts.go, line 1279 at r1 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
i guess this is separate from your PR, but does this work for user-defined enum types?
Apparently not. Definitely something that needs to be fixed! Filed #61216
pkg/sql/sem/tree/casts.go, line 1291 at r1 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
just curious, is there a way we can warn ourselves in the future if we do support user-defined functions?
Hmm............... I can't think of one. I added it to the tracking issue though. #58356
pkg/sql/sem/tree/casts.go, line 1299 at r1 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
is the default case still used anywhere? it would be nice to remove the
queryOidfunction altogether if not!
It is for ::regnamespace. I was thinking about getting rid of this, but we're not ready yet. queryOid is also still used for several other cases.
pkg/sql/sem/tree/casts.go, line 1315 at r1 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
do we need to look out for this case too?
Good point! We had some special cases above that were not being done here properly. I've refactored a function out that can be used in both places.
jordanlewis
left a comment
There was a problem hiding this comment.
I figured out how to adjust the expectations file actually!
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @jordanlewis and @rafiss)
66a0a53 to
402acbc
Compare
rafiss
left a comment
There was a problem hiding this comment.
amazing!! thank you for taking this on
left optional comments about typing choices.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @jordanlewis and @rafiss)
pkg/sql/sem/tree/function_definition.go, line 165 at r2 (raw file):
// to their name. We populate this from the pg_catalog.go file in the sql // package because of dependency issues: we can't use oidHasher from this file. var OidToBuiltinName map[uint32]string
err i realize i was the one who suggested making the key be uint32, but perhaps it should actually be a oid.Oid (same as the existing OidToType map)
pkg/sql/sem/tree/overload.go, line 91 at r2 (raw file):
// Oid is the cached oidHasher.BuiltinOid result for this Overload. It's // populated at init-time. Oid uint32
and here, perhaps this should either be a oid.Oid or a DOid?
i'll leave this up to you
rafiss
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @jordanlewis and @rafiss)
pkg/sql/sem/tree/casts.go, line 1274 at r2 (raw file):
switch v := d.(type) { case *DOid: return performIntToOidCast(ctx, t, v.DInt)
oh hmm it seems like something has tripped up the oid::regclass cast in the pgoidtype logic test
|
@jordanlewis @rafiss do you folks consider this backport-able to 20.2? |
Previously, the ::REGPROC cast (which converts an OID of a builtin function to its name or vice versa) was implemented extremely inefficiently: via a query from the internal executor that ultimately has to re-materialize every builtin into memory and filter. This can lead to pathologic quadratic behavior for some common ORM queries. Now, both the oid-to-name and name-to-oid directions are made into constant-time lookups by simply preparing a map of the oid to name and vice versa at startup time. Release note (performance improvement): introspection queries that use casts into the REGPROC pseudo-type are made much more efficient: they're now implemented as a constant-time lookup instead of an internal query. Release justification: low risk, high benefit changes to existing functionality
402acbc to
2712ba5
Compare
|
@mgartner yes, this feels pretty backportable. |
|
@rafiss PTAL |
|
bors r+ |
|
Build succeeded: |
|
@jordanlewis if we can get this backported before the 20.2.6 SHA is picked (I believe on March 8) that would be great. If you're too busy to get to it by then, I can attempt the backport, just LMK! |
|
@rafiss could you provide the decision on whether we can backport this? |
|
I think this should be backported. Let's do it! |
|
We tried out the backport, but looks like it depends on db0f4bf which we think is too risky to backport. So this will not be in 20.2. It's less important to backport because our ActiveRecord adapter will be changed to stop using this cast: cockroachdb/activerecord-cockroachdb-adapter#182 |
Previously, the ::REGPROC cast (which converts an OID of a builtin
function to its name or vice versa) was implemented extremely
inefficiently: via a query from the internal executor that ultimately
has to re-materialize every builtin into memory and filter. This can
lead to pathologic quadratic behavior for some common ORM queries.
Now, both the oid-to-name and name-to-oid directions are made into
constant-time lookups by simply preparing a map of the oid to name and
vice versa at startup time.
Closes #61082.
Release note (performance improvement): introspection queries that use
casts into the REGPROC pseudo-type are made much more efficient: they're
now implemented as a constant-time lookup instead of an internal query.
Release justification: low risk, high benefit changes to existing functionality