The API for determining `Job` states is a bit messy

[taras-i-fedyk]

The current state of affairs

There’s some messiness in the API for determining Job states, which boils down mainly to the following two things:

• there’s no clear distinction between a failure and a cancellation of a Job. Everything is too blurred in that context. Hence it’s difficult to reason about things.

• the isCompleted property is for determining if a Job has finished for whatever reason. But at the same time, the Completed state is solely for a successful finish (not also for a failure or a cancellation). Hence a contradiction.

The proposed state of affairs

For the sake of good order, everything has been implemented in the following way (or something along these lines):

• the isCompleted property returns true solely if the Job has finished successfully (not also when it has failed or has been cancelled). Consequently, the Completed state represents solely such situations.

• the isFailed property returns true if the Job is failing / has failed. Consequently, the Failing / Failed states represent such situations. In addition to that, the fail function replaces completeExceptionally.

• the isCancelled property returns true solely if the Job is being cancelled / has been cancelled (not also when it is being failed / has been failed). Consequently, the Cancelling / Cancelled states represent solely such situations.

• the isFailedOrCancelled property returns true if isFailed or isCancelled returns true.

• the isFinished property returns true if isCompleted or isFailed or isCancelled returns true.

[qwwdfsad]

there’s no clear distinction between a failure and a cancellation of a Job. Everything is too blurred in that context. Hence it’s difficult to reason about things.

In our first implementations, we indeed had a clear distinction between cancellation and failure.
While this may be sometimes convenient (tho we didn't really find any real use-cases for that), it raises a question regarding structured concurrency, parallel decomposition transparency and consistency.

For example, when parent coroutine launches a child (e.g. to decompose the work) and the child fails with an exception, the child will be failed, but the parent will be cancelled.
Now, the same scenario, but the same code is executed by the parent coroutine itself: it will be failed!

Thus, the fact of the parallel decomposition leaks to the caller, tho it should be an implementation detail that can be swapped in and out. With isCancelled and isFailed it is not the case.

Alternatively, if the child fails, we could fail parent as well, but then a more complicated questions arise: how to handle exceptions once? What is "failure cause" when multiple children fail? etc.

In the face of these design issues, potential inconsistencies, implementation details leaks and, what's more important, absence of real-world demand for such API, we've merged cancelled and failed states.
You can read more here: #585 and #220

the isCompleted property is for determining if a Job has finished for whatever reason. But at the same time, the Completed state is solely for a successful finish (not also for a failure or a cancellation). Hence a contradiction.

Here we definitely should improve our documentation, because Completed from our KDoc ASCII diagram and isCompleted mean different things.
Thanks for spotting!

Also, it is really interesting how you bumped into this issue: have you found this API limitation when trying to solve some application-specific problem? Or you've found it by thoughtful documentation studying?

[taras-i-fedyk]

@qwwdfsad

Alternatively, if the child fails, we could fail parent as well,

Yes, I mean exactly this alternative. If a child has failed, it should lead to a failure of the parent. And it should be an explicit failure which is represented by the Failing state or by the Failed state, not an implicit failure which is represented by the Cancelling state or by the Cancelled state. The Failing state would mean that the parent has to cancel the active children and wait until they have finished their execution.

(And in such a case, there wouldn't be any leaking of details to the caller, it seems.)

but then a more complicated questions arise: how to handle exceptions once? What is "failure cause" when multiple children fail? etc.

But are the above-mentioned questions something new? They are present anyway, even now when there's no clear distinction between a failure and a cancellation. (For example, what is the cause of a CancellationException in the parent when multiple children have failed?)

Obviously, those questions have been addressed already. And the way in which they have been addressed wouldn't have to be changed while introducing a clear distinction between a failure and a cancellation. (Unless I'm really missing something.)

Also, it is really interesting how you bumped into this issue: have you found this API limitation when trying to solve some application-specific problem? Or you've found it by thoughtful documentation studying?

I came to the understanding that such an issue exists not while trying to solve some application-specific problem, but while trying to understand how coroutines as a whole work (which includes studying the documentation, trying certain things in practice, and forming a big picture of how everything works).