-
Notifications
You must be signed in to change notification settings - Fork 550
bug: Cloudquery SDK: transformers.TransformWithStruct does not produce non nullable fields #20286
Copy link
Copy link
Closed
cloudquery/plugin-sdk
#2084Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
When developing an SDK plugin and using the transformers.TransformWithStruct for some tables, passing a struct with non pointer fields produces nullable fields on the destinations (tested with SQLite and Postgres).
Expected Behavior
Fields that are pointers becomes nullable fields, native fields (string, int...) are non nullable
CloudQuery (redacted) config
kind: source
spec:
name: "cq-nullable-fields"
registry: "local"
path: "./cq-source-cloudquery-struct-nullable-fields"
version: "v1.0.0"
tables:
- "cloudquery-struct-nullable-fields_sample_table"
destinations:
- "sqlite"
---
kind: destination
spec:
name: sqlite
path: "cloudquery/sqlite"
registry: cloudquery
version: "v2.10.7"
spec:
connection_string: ./db.sql
Steps To Reproduce
- Checkout https://github.com/jeromewir/cq-source-cloudquery-struct-nullable-fields
- Run
go build . && cloudquery sync . - Check nullability of fields with a SQLite db viewer
CloudQuery (redacted) logs
N/A
CloudQuery version
6.15.0
Additional Context
From my investigation, it seems like this could be achieve by providing a NotNull param in the SDK internals: https://github.com/cloudquery/plugin-sdk/blob/fc27b14c801d44ffba458fc676a6e3f0d577dfec/transformers/struct.go#L159
Would it be possible to have an option in the transformer to make those fields non-nullable?
Pull request (optional)
- I can submit a pull request
Reactions are currently unavailable