Describe the bug, including details regarding any error messages, version, and platform.
We have an internal application that is using the arrow::RecordBatchReader with a Python generator (but in C++). Right now, when the Python generator raises an error, the stack trace disappears (e.g. in our code base we just see:
Python exception: AssertionError
without the stack trace.
I am not super familiar with C++ or the code, but I think the problem can be traced to this function:
|
Status ConvertPyError(StatusCode code) { |
|
auto detail = PythonErrorDetail::FromPyError(); |
|
if (code == StatusCode::UnknownError) { |
|
code = MapPyError(detail->exc_type()); |
|
} |
|
|
|
std::string message; |
|
RETURN_NOT_OK(internal::PyObject_StdStringStr(detail->exc_value(), &message)); |
|
return Status(code, message, detail); |
|
} |
which calls into
|
std::string ToString() const override { |
|
// This is simple enough not to need the GIL |
|
const auto ty = reinterpret_cast<const PyTypeObject*>(exc_type_.obj()); |
|
// XXX Should we also print traceback? |
|
return std::string("Python exception: ") + ty->tp_name; |
|
} |
where we only attach the exception type name and no other information. My C++ is not very good, but how hard would it be to actually attach the full Python stacktrace inside the message whenever we hit a Python exception?
Component(s)
Python
Describe the bug, including details regarding any error messages, version, and platform.
We have an internal application that is using the
arrow::RecordBatchReaderwith a Python generator (but in C++). Right now, when the Python generator raises an error, the stack trace disappears (e.g. in our code base we just see:without the stack trace.
I am not super familiar with C++ or the code, but I think the problem can be traced to this function:
arrow/python/pyarrow/src/arrow/python/common.cc
Lines 144 to 153 in 750670e
which calls into
arrow/python/pyarrow/src/arrow/python/common.cc
Lines 91 to 96 in 750670e
where we only attach the exception type name and no other information. My C++ is not very good, but how hard would it be to actually attach the full Python stacktrace inside the message whenever we hit a Python exception?
Component(s)
Python