Skip to content

Add return_exceptions argument to modal.FunctionCall.gather. #3014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions modal/_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -1760,13 +1760,15 @@ async def from_id(function_call_id: str, client: Optional[_Client] = None) -> "_
return fc

@staticmethod
async def gather(*function_calls: "_FunctionCall[T]") -> typing.Sequence[T]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to mean that the return type is "wrong" in the case of an exception right? Any idea how asyncio.gather handles this? Might need to @overload

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 - add an @overload which differentiates the return type based on a Literal[True] value for the return_exceptions argument, and otherwise only returns Sequence[T] - that should fix the existing type assertion

"""Wait until all Modal FunctionCall objects have results before returning.
async def gather(
*function_calls: "_FunctionCall[T]",
return_exceptions: bool = False, # propagate exceptions (False) or aggregate them in the results list (True)
) -> typing.Sequence[T | BaseException]:
"""Returns a list of results from a variable number of Modal `FunctionCall` objects (as returned by
`Function.spawn()`).

Accepts a variable number of `FunctionCall` objects, as returned by `Function.spawn()`.

Returns a list of results from each FunctionCall, or raises an exception
from the first failing function call.
By default, this will raise an exception from the first failing function call. If `return_exceptions=True`, it
will instead aggregate exceptions in the results list.

Examples:

Expand All @@ -1779,6 +1781,9 @@ async def gather(*function_calls: "_FunctionCall[T]") -> typing.Sequence[T]:

*Added in v0.73.69*: This method replaces the deprecated `modal.functions.gather` function.
"""
if return_exceptions:
return await asyncio.gather(*[fc.get() for fc in function_calls], return_exceptions=True)

try:
return await TaskContext.gather(*[fc.get() for fc in function_calls])
except Exception as exc:
Expand Down
18 changes: 18 additions & 0 deletions test/function_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,24 @@ def test_function_exception(client, servicer):
assert "foo!" in str(excinfo.value)


def test_function_exception_gather(client, servicer):
app = App()

servicer.function_body(failure)
failure_modal = app.function()(failure)
with app.run(client=client):
with pytest.raises(CustomException) as excinfo:
FunctionCall.gather(failure_modal.spawn(), failure_modal.spawn())
assert "foo!" in str(excinfo.value)

with app.run(client=client):
results = FunctionCall.gather(failure_modal.spawn(), failure_modal.spawn(), return_exceptions=True)
for result in results:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be nice to assert that you can have a mixture of exceptions with expected successful results.

assert isinstance(result, UserCodeException)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this UserCodeException type?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a synchronicity-internal type ("User" in this case is user of synchronicity, i.e. modal sdk devs - not user of Modal) that isn't intended to leak into user code.

Internally in synchronicity it's used to wrap exceptions when they first happen and then unwrap them when they are raised to users. This is used a a shortcut to remove as much synchronicity context leak as possible in tracebacks etc. since those parts are usually not relevant to users.

However in this case, since we never raise the exceptions from the synchronized function (Function.gather(return_exceptions=True) in this case) it never triggers the unwrapping code in synchronicity and the internal type "leaks", since it's a return value instead.

I'm not sure what would be the cleanest fix tbh... It does feel like this is synchronicity's responsibility and the fix should happen on that side - I'll try something.

I'm personally not convinced the exception wrapping/unwrapping does us much good, and it's something I'd like to get rid of in synchronicity 2.0 if we can somehow get the traces relatively simple there through other means.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some more investigation, the problem might be worse than I thought, apparently the client has some explicit wrapping using UserCodeException too in _process_result, which seems like the more likely culprit here 🤯

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to get a 😵‍💫 option for GitHub reactions 😁

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm inclined to believe this isn't actually a synchronicity issue but a Modal hack introducing explicit UserCodeException wrapping on the modal client side. This was introduced in 0.0.35 :D

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohhhh no, synchronicity.UserCodeException is apparently being returned by .map(return_exceptions=True) as well :(

Now I'm not sure what to do - will have to discuss in client team...

assert isinstance(result.exc, CustomException)
assert "foo!" in str(result)


@pytest.mark.asyncio
async def test_function_exception_async(client, servicer):
app = App()
Expand Down
Loading