Skip to content

Enable question retries on single questions #671

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions docs/source/asking_questions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,13 @@ You can also set the following options when you call :mod:`Child.ask <octue.reso
- ``asynchronous`` - if ``True``, don't wait for an answer to the question (the result and other events can be :ref:`retrieved from the event store later <retrieving_asynchronous_answers>`)
- ``timeout`` - how long in seconds to wait for an answer (``None`` by default - i.e. don't time out)

If the question fails:
- If ``raise_errors=False``, the unraised error is returned
- If ``raise_errors=False`` and ``max_retries > 0``, the question is retried up to this number of times
- If ``raise_errors=False``, ``max_retries > 0``, and ``prevent_retries_when`` is a list of exception types, the question is retried unless the error type is in the list
- If ``raise_errors=False``, ``log_errors=True``, and the question fails after its final retry, the error is logged


Exceptions raised by a child
----------------------------
If a child raises an exception while processing your question, the exception will always be forwarded and re-raised in
Expand Down Expand Up @@ -185,8 +192,8 @@ access the event store and run:

Asking multiple questions in parallel
=====================================
You can also ask multiple questions to a service in parallel. By default, if any of the questions fail, an error is
raised and no answers are returned.
You can also ask multiple questions to a service in parallel - just provide questions as dictionaries of `Child.ask`
arguments:

.. code-block:: python

Expand All @@ -203,18 +210,11 @@ raised and no answers are returned.

This method uses multithreading, allowing all the questions to be asked at once instead of one after another.

**Options**
.. hint::

- If ``raise_errors=False`` is provided, answers are returned for all successful questions while unraised errors are
logged and returned for unsuccessful ones
- If ``raise_errors=False`` is provided with ``max_retries > 0``, failed questions are retried up to this number of
times
- If ``raise_errors=False`` is provided with ``max_retries > 0`` and ``prevent_retries_when`` is set to a list of
exception types, failed questions are retried except for those whose exception types are in the list
- ``max_workers``: The maximum number of threads that can be used to ask questions in parallel can be set via this
argument. It has no effect on the total number of questions that can be asked via ``Child.ask_multiple``.
- ``log_errors``: If `True` and ``raise_errors=False``, any errors remaining once retries are exhausted are logged in
addition to being returned
The maximum number of threads that can be used to ask questions in parallel can be set via the ``max_workers``
argument. It has no effect on the total number of questions that can be asked, just how many can be in progress at
once.


Asking a question within a service
Expand Down
168 changes: 85 additions & 83 deletions docs/source/inter_service_compatibility.rst

Large diffs are not rendered by default.

7 changes: 0 additions & 7 deletions octue/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
logger = logging.getLogger(__name__)


PYTHONUNBUFFERED = "PYTHONUNBUFFERED"
__all__ = ("Runner",)


Expand All @@ -22,9 +21,3 @@
include_process_name=bool(int(os.environ.get("INCLUDE_PROCESS_NAME_IN_LOGS", 0))),
include_thread_name=bool(int(os.environ.get("INCLUDE_THREAD_NAME_IN_LOGS", 0))),
)

if not os.environ.get(PYTHONUNBUFFERED):
logger.warning(
f"The {PYTHONUNBUFFERED!r} environment variable isn't set - logs may not appear in real time. Set "
f"{PYTHONUNBUFFERED}=1 to fix this."
)
Loading
Loading