Skip to content

fix: Timestamp field replacement#20809

Merged
kodiakhq[bot] merged 2 commits intomainfrom
fix/timestamp_field_replacement
May 20, 2025
Merged

fix: Timestamp field replacement#20809
kodiakhq[bot] merged 2 commits intomainfrom
fix/timestamp_field_replacement

Conversation

@erezrokah
Copy link
Copy Markdown
Member

@erezrokah erezrokah commented May 20, 2025

Summary

We've seen a panic on our Cloud syncs when syncing from ClickHouse to BigQuery, the panic is

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x60 pc=0xf9bd50]

goroutine 138 [running]:
github.com/cloudquery/plugin-sdk/v4/schema.ReplaceFieldInRecord({0x0, 0x0}, {0x15ff4f5, 0xf}, {0x182bda0, 0xc0003844b0})
	/home/runner/go/pkg/mod/github.com/cloudquery/plugin-sdk/v4@v4.79.1/schema/arrow.go:99 +0x30
github.com/cloudquery/cloudquery/cli/v6/internal/transformer.(*RecordTransformer).Transform(0xc0005c29c0, {0x182a050, 0xc0007d0720})
	/home/runner/work/cloudquery/cloudquery/cli/internal/transformer/transformer.go:154 +0x46d
github.com/cloudquery/cloudquery/cli/v6/cmd.syncConnectionV3.func6()
	/home/runner/work/cloudquery/cloudquery/cli/cmd/sync_v3.go:494 +0xc15
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/home/runner/go/pkg/mod/golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
	/home/runner/go/pkg/mod/golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x93
{"severity":"info","module":"launcher","timestamp":"2025-05-19T00:15:07.413712727Z","message":"post-flight checks passed"}
Sync run completed with status 2

The issue was that the schema of the source sync time column didn't not have the timezone (was an empty string) so when we tried to replace it with a UTC timezone schema.ReplaceFieldInRecord failed due to type mismatch returning a nil record.

@erezrokah erezrokah requested review from a team and murarustefaan May 20, 2025 10:10
@erezrokah erezrokah added the automerge Automatically merge once required checks pass label May 20, 2025
@kodiakhq kodiakhq bot merged commit ee81ce9 into main May 20, 2025
18 checks passed
@kodiakhq kodiakhq bot deleted the fix/timestamp_field_replacement branch May 20, 2025 10:42
@cq-bot cq-bot mentioned this pull request May 20, 2025
Merged
kodiakhq bot pushed a commit that referenced this pull request May 20, 2025
@cq-bot
chore(main): Release cli v6.20.1 (#20804)
164f75d 
🤖 I have created a release *beep* *boop*
---


## [6.20.1](cli-v6.20.0...cli-v6.20.1) (2025-05-20)


### Bug Fixes

* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.80.3 ([#20802](#20802)) ([2ba2f8e](2ba2f8e))
* Timestamp field replacement ([#20809](#20809)) ([ee81ce9](ee81ce9))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@murarustefaan murarustefaan Awaiting requested review from murarustefaan

1 more reviewer

@maaarcelino maaarcelino maaarcelino approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area/cli automerge Automatically merge once required checks pass

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@erezrokah @maaarcelino @cq-bot