-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
Description
get_run_logger() raises MissingContextError when called outside a flow or task run context. This is the correct default behavior, but there is no way to opt into a graceful fallback — making it unusable from threads (e.g., ThreadPoolExecutor workers) without monkey-patching.
Use case
When running parallel file processing inside a Prefect task using ThreadPoolExecutor, worker threads don't inherit the Prefect run context. Any library code or shared utility that calls get_run_logger() crashes in these threads:
@task
def process_files(files: list[str]):
with ThreadPoolExecutor(max_workers=4) as executor:
executor.map(process_single_file, files)
def process_single_file(path: str):
logger = get_run_logger() # MissingContextError — no active context in thread
logger.info(f"Processing {path}")Current workaround
Users must monkey-patch prefect.get_run_logger globally to add fallback behavior:
_original = prefect.get_run_logger
def _patched_get_run_logger(**kwargs):
try:
return _original(**kwargs)
except Exception:
return logging.getLogger("prefect.fallback")
prefect.get_run_logger = _patched_get_run_loggerThis is fragile, affects all code globally, and can conflict with other patches.
Proposed solution
Add an on_missing_context parameter to get_run_logger():
def get_run_logger(
context=None,
on_missing_context: Literal["raise", "warn", "ignore"] = "raise",
**kwargs,
):"raise"(default): current behavior, fully backward compatible"warn": return a standardprefectlogger and emit a debug message"ignore": return a standardprefectlogger silently
# Works in threads without monkey-patching
logger = get_run_logger(on_missing_context="warn")
logger.info("Processing file") # Uses Prefect logger in context, fallback in threadsI have a PR ready with implementation and tests.