fix: Improve errors in batch modes#1441
Merged
jackc merged 1 commit intojackc:masterfrom Dec 21, 2022
Merged
Conversation
This was referenced Dec 18, 2022
kodiakhq bot
pushed a commit
to cloudquery/cloudquery
that referenced
this pull request
Dec 18, 2022
This is a follow-up to #5733 (see discussion there). and needed so we can incorporate this nice bit: jackc/pgx#1441 The jackc/pgx#1441 is not a pre-requirement for this to go through. even better we should prob ship this first and see that `v5` works for us and for our users and if not we can do the same fix for `v4` (we will just have to fork it as I don't think `v4` accepts anymore contributions)
makalaaneesh
added a commit
to yugabyte/yb-voyager
that referenced
this pull request
Oct 21, 2024
When using pgx SendBatch, there can be two types of errors thrown: - Error while preparing the statement - this is preprocessing (parsing, preparinng statements, etc) that pgx will do before sending the batch. Examples - syntax error. No matter which statement in the batch has an issue, the error will be thrown on calling br.Exec() for the first time. - Error while executing the statement - this is the actual execution of the statement, and the error comes from the DB. Examples - constraint violation, etc. In this case, we get the error on the appropriate br.Exec() call associated with the statement that failed. Therefore, if error is thrown on the first br.Exec() call, it could be either of the above cases. Modified the logs to indicate this as part of this PR. Reference - jackc/pgx#872 This ideally needs to be fixed in pgx library, and has been partially fixed in jackc/pgx#1441 (pgx v5)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi there! Thanks for this great library and we are long time users of
v4(in process of migrating tov5already have a PR in place but have a few more tests to run).Specifically we are using batch jobs heavily and we have an issue that errors don't include which query caused the error even though this information is available in
pgx. This adds the needed error wrapping and tested with this code (this code still usesv4but I tested locally also on ourv5branch).Also, to clarify when the error is hapening on the Postgres side
pgxis already returning the useful information viapgErr.TableNameso it's only when it fails in the prepared statement before hand on the client side.Hopefully this fix will help others as well.