Describe the bug
Fault objects seem to get wrapped in DataObjects, which don't have a __traceback__ attribute and causes some error handlers to crash.
This has been mentioned in #1080 (comment), and I believe it is also related to #1086.
Reproduction steps
- Use a framework or write an error handler that explicitly checks
__traceback__. In my case, I'm using Celery.
- Allow the fault Exception to raise.
This is similar to my actual usage:
def start_resize(vm, user, **options):
check_permissions(user, vm)
result = tasks.resize_vm.s(vm, **options).apply_async()
result.get()
# tasks.py
@celeryapp.task
def resize_vm(vm, **options):
vc = get_vsphere_client()
try:
vc.change_hardware(vm, **options)
except Exception:
logger.exception("...")
raise
# class VsphereClient
def change_hardware(vm, **options):
configspec = vim.vm.ConfigSpec(**options)
if vm.runtime.powerState == "poweredOn":
self.power_off_vm(vm) # raises vim.fault.InvalidPowerState if the VM is already stopped
task = vm.ReconfigVM_Task(spec=configspec)
WaitForTask(task, si=self.si)
self.power_on_vm(vm) # raises vim.fault.InvalidPowerState if the VM is already running
When the InvalidPowerState exception is raised, the resize_vm task correctly logs the exception:
pyVmomi.VmomiSupport.vim.fault.InvalidPowerState: (vim.fault.InvalidPowerState) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
msg = 'The attempted operation cannot be performed in the current state (Powered on).',
faultCause = <unset>,
faultMessage = (vmodl.LocalizableMessage) [],
requestedState = 'poweredOff',
existingState = 'poweredOn'
}
The problem begins after the call to logger.exception, when the exception is re-raised:
< the InvalidPowerState traceback omitted for brevity >
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.13/site-packages/celery/app/trace.py", line 605, in trace_task
I, R, state, retval = on_error(task_request, exc)
~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/site-packages/celery/app/trace.py", line 421, in on_error
R = I.handle_error_state(
task, request, eager=eager, call_errbacks=call_errbacks,
)
File "/usr/local/lib/python3.13/site-packages/celery/app/trace.py", line 199, in handle_error_state
return {
~
RETRY: self.handle_retry,
~~~~~~~~~~~~~~~~~~~~~~~~~
FAILURE: self.handle_failure,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
}[self.state](task, req,
~~~~~~~~~~~~~^^^^^^^^^^^
store_errors=store_errors,
^^^^^^^^^^^^^^^^^^^^^^^^^^
call_errbacks=call_errbacks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/site-packages/celery/app/trace.py", line 249, in handle_failure
if exc.__traceback__ is None:
^^^^^^^^^^^^^^^^^
AttributeError: 'DataObject' object has no attribute '__traceback__'
InvalidPowerState happens to be the one that happened today, but I've run into this with other VimFault types. In #1080, it was a NotFound fault.
Expected behavior
Re-raising a vim.fault.VimFault should not pass a DataObject up to the next exception handler. Or, if that can't be avoided, a DataObject that wraps a VimFault should proxy the __traceback__ attribute to the wrapped fault object's __traceback__.
Additional context
While I am running into this with Celery, this could be expected to occur with any exception handler that accesses __traceback__ on exception objects.
I mentioned this same problem in #1080, which was primarily focused on static typing. However, this is a runtime erasure of exception information.
#1086 also points out that VimFault itself is not an exception type. I mentioned the same concern in #1080. It appears Exception doesn't get added to the MRO for faults until runtime, which is certainly a pain for type checkers, but, more importantly, it also seems that there are circumstances where the VimFault is wrapped in a DataObject--which is definitely not a BaseException even at runtime. That would make it impossible to catch, in addition to erasing the __traceback__.
Describe the bug
Fault objects seem to get wrapped in
DataObjects, which don't have a__traceback__attribute and causes some error handlers to crash.This has been mentioned in #1080 (comment), and I believe it is also related to #1086.
Reproduction steps
__traceback__. In my case, I'm using Celery.This is similar to my actual usage:
When the
InvalidPowerStateexception is raised, theresize_vmtask correctly logs the exception:The problem begins after the call to
logger.exception, when the exception is re-raised:InvalidPowerStatehappens to be the one that happened today, but I've run into this with otherVimFaulttypes. In #1080, it was aNotFoundfault.Expected behavior
Re-raising a
vim.fault.VimFaultshould not pass aDataObjectup to the next exception handler. Or, if that can't be avoided, aDataObjectthat wraps aVimFaultshould proxy the__traceback__attribute to the wrapped fault object's__traceback__.Additional context
While I am running into this with Celery, this could be expected to occur with any exception handler that accesses
__traceback__on exception objects.I mentioned this same problem in #1080, which was primarily focused on static typing. However, this is a runtime erasure of exception information.
#1086 also points out that
VimFaultitself is not an exception type. I mentioned the same concern in #1080. It appearsExceptiondoesn't get added to the MRO for faults until runtime, which is certainly a pain for type checkers, but, more importantly, it also seems that there are circumstances where theVimFaultis wrapped in aDataObject--which is definitely not aBaseExceptioneven at runtime. That would make it impossible to catch, in addition to erasing the__traceback__.