Skip to content

get_run_logger() raises MissingContextError in threaded contexts with no fallback option #21027

@zionts

Description

@zionts

Description

get_run_logger() raises MissingContextError when called outside a flow or task run context. This is the correct default behavior, but there is no way to opt into a graceful fallback — making it unusable from threads (e.g., ThreadPoolExecutor workers) without monkey-patching.

Use case

When running parallel file processing inside a Prefect task using ThreadPoolExecutor, worker threads don't inherit the Prefect run context. Any library code or shared utility that calls get_run_logger() crashes in these threads:

@task
def process_files(files: list[str]):
    with ThreadPoolExecutor(max_workers=4) as executor:
        executor.map(process_single_file, files)

def process_single_file(path: str):
    logger = get_run_logger()  # MissingContextError — no active context in thread
    logger.info(f"Processing {path}")

Current workaround

Users must monkey-patch prefect.get_run_logger globally to add fallback behavior:

_original = prefect.get_run_logger

def _patched_get_run_logger(**kwargs):
    try:
        return _original(**kwargs)
    except Exception:
        return logging.getLogger("prefect.fallback")

prefect.get_run_logger = _patched_get_run_logger

This is fragile, affects all code globally, and can conflict with other patches.

Proposed solution

Add an on_missing_context parameter to get_run_logger():

def get_run_logger(
    context=None,
    on_missing_context: Literal["raise", "warn", "ignore"] = "raise",
    **kwargs,
):
  • "raise" (default): current behavior, fully backward compatible
  • "warn": return a standard prefect logger and emit a debug message
  • "ignore": return a standard prefect logger silently
# Works in threads without monkey-patching
logger = get_run_logger(on_missing_context="warn")
logger.info("Processing file")  # Uses Prefect logger in context, fallback in threads

I have a PR ready with implementation and tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions