Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #140 +/- ##
==========================================
- Coverage 64.70% 63.43% -1.27%
==========================================
Files 33 34 +1
Lines 2207 2210 +3
==========================================
- Hits 1428 1402 -26
- Misses 779 808 +29 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
karel-rehor
left a comment
There was a problem hiding this comment.
Looking at #134 I think the new classes InfluxdDBClientError and InfluxdbClientQueryError are a good start, but oclyke is looking for a taxonomy of more specific error definitions. Also class names could include the version 3 value to make sure they are distinct from other influxdb libraries.
Compare this to the Arrow Flight taxonomy of error classes. (https://arrow.apache.org/docs/python/generated/pyarrow.flight.FlightError.html).
Firstly from oclyke's comment I think the user does not want to see these Arrow Flight errors or to deal with them.
Secondly also from oclyke's comment I think the user wants to see an error class that is self documenting or self explanatory so that it is readily understood.
Thirdly we could implement a taxonomy similar to that of Arrow, which would wrap specific Arrow exceptions and their contents.
e.g.
class InfluxDB3ClientQueryServerError(InfluxDB3ClientError):
...
class InfluxDB3ClientQueryServerUnkownDBError(InfluxDB3ClientQueryServerError):
...
class InfluxDB3ClientQueryUnauthenticatedError(InfluxDB3ClientError):
...
class InfluxDB3ClientQueryUnavailableError(InfluxDB3ClientError):
...
etc.We could then maybe have an ArrowErrorHandler class with a static method to map caught Arrow errors to classes in our error taxonomy
There was a problem hiding this comment.
- An item for a more finely defined error taxonomy across all client libraries has been opened in the backlog. So this PR represents only a partial solution to #140
- 1 older unit test was failing locally, so I've updated it. You may want to review or revert these changes.
- Some refactoring comments
- Since there already exists an InfluxDBError class in the
write_clientpackage, I think this needs to be taken into account. - Since
InfluxdbClientQueryErroronly deals with query errors, it could be declared in thequerypackage - another alternative might be to declare all exceptions/errors in a new
exceptionspackage.
- Since there already exists an InfluxDBError class in the
BTW. When running unit tests locally, even though all tests now pass, once they are finished, I keep running across a threading issue associated with urllib3 after the tests complete. I don't recall seeing this in the past. Needs further investigation.
...
RuntimeError: can't create new thread at interpreter shutdown
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPSConnection object at 0x71b678a54b90>: Failed to resolve 'none' ([Errno -3] Temporary failure in name resolution)'.
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPSConnection object at 0x71b678a57e00>: Failed to resolve 'none' ([Errno -3] Temporary failure in name resolution)'.
...
Hi @karel-rehor |
karel-rehor
left a comment
There was a problem hiding this comment.
The package exceptions could be improved with an import statement in __init__.py.
It's best to avoid, whenever possible, imports that repeat the same token twice, like...
from influxdb_client_3.exceptions.exceptions import InfluxDB3ClientQueryError
karel-rehor
left a comment
There was a problem hiding this comment.
I located the source of the problem with urllib3 connection attempts cycling in _write_batching, so I updated this test with a forced disposal.
RuntimeError: can't create new thread at interpreter shutdown
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPSConnection object at 0x79d5a779a780>: Failed to resolve 'none' ([Errno -3] Temporary failure in name resolution)'.
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPSConnection object at 0x79d5a7799520>: Failed to resolve 'none' ([Errno -3] Temporary failure in name resolution)'.
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPSConnection object at 0x79d5a779a630>: Failed to resolve 'none' ([Errno -3] Temporary failure in name resolution)'.
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPSConnection object at 0x79d5a77986b0>: Failed to resolve 'none' ([Errno -3] Temporary failure in name resolution)'.
The batch item wasn't processed successfully because: HTTPSConnectionPool(host='none', port=443): Max retries exceeded with url: /api/v2/write?org=org&bucket=bucket&precision=ns (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x79d5a77aa7e0>: Failed to resolve 'none' ([Errno -3] Temporary failure in name resolution)"))
Test pipeline now takes 2 minutes instead of 5 minutes.
I see code coverage is missing something now in write/retry.py. This is likely a side-effect of this change. Retry was being called after the test_write_api_custom_options_no_error() test had completed. Execution of these code blocks in retry.py was inadvertent, because the test is supposed to be testing only options. A specific retry test for these dropped blocks could be added. But that now goes too far beyond the scope of this PR.
|
@karel-rehor Thank you 👨🔬 |
Closes #134
Proposed Changes
InfluxdbClientQueryErrorwhen catchingArrowExceptionexceptions from gRPC serversInfluxdbClientWriteErrorbut it will break backwards compatibility for the write api If users already useInfluxDBErrorclassExceptionclass then throws - I thinkChecklist