Add log_and_raise helper to merge fre_logger.error and raise steps

[Copilot]

Describe your changes

Adds a log_and_raise() helper to fre/__init__.py that combines fre_logger.error() and raise into a single call, eliminating repetitive and duplicated error text across the codebase. All raise ExcType(msg) statements throughout the codebase have been converted to use this helper.

Helper function (fre/__init__.py):

• Annotated as -> NoReturn so pylint and type checkers treat calls to it as control flow terminators (just like raise)
• Logs at ERROR via fre_logger with stacklevel=2 so log output attributes the correct caller, not log_and_raise itself
• Always passes stack_info=True so the call-site stack trace appears in every configured handler, including log files set up via fre -l
• Passes exc_info=True when exc is provided, per the Python logging HOWTO, so the caught exception's full traceback is written to every handler
• Raises exc_type (default ValueError) with the same message
• Supports exception chaining via exc param (raise ... from exc)

# Before (message duplicated):
fre_logger.error(f"input dir {d} does not exist")
raise OSError(f"input dir {d} does not exist")

# After:
log_and_raise(f"input dir {d} does not exist", OSError)

# With exception chaining:
except KeyError as e:
    log_and_raise("update failed", KeyError, exc=e)

Conversion patterns applied across ~44 files:

• Simple raises: raise ExcType(msg) → log_and_raise(msg, ExcType)
• Chained exceptions: raise ExcType(msg) from exc → log_and_raise(msg, ExcType, exc=exc)
• Duplicate message: removed redundant fre_logger.error and replaced both with log_and_raise
• Different context log + raise message: kept fre_logger.error for context, replaced raise with log_and_raise
• Bare raise (no-arg re-raises in except blocks) — left unchanged
• Abstract method raise NotImplementedError() in timeAverager.py — left unchanged

Traceback logging for fre -l log files:

A key motivation for this helper is ensuring Traceback info reaches log files specified via fre -l ./logger_output.log. Per Python logging best practices (PEP 282, logging HOWTO):

• stack_info=True is always passed, so the call-site stack trace (Stack (most recent call last):) appears in every handler
• exc_info=True is passed when exc is given, so the caught exception's traceback (Traceback (most recent call last):) appears in every handler

Two new tests (test_log_and_raise_stack_trace_in_log_file, test_log_and_raise_traceback_in_log_file) simulate fre -l ./logger_output.log by attaching a FileHandler and verify:

1. Stack trace info appears in the log file for simple raises
2. Caught exception traceback and the original exception message appear in the log file for chained raises

Files converted (representative list): app/helpers.py, app/remap_pp_components/, app/mask_atmos_plevel/, app/generate_time_averages/, app/regrid_xy/, pp/checkout_script.py, pp/run_script.py, pp/trigger_script.py, pp/nccheck_script.py, pp/status_script.py, pp/histval_script.py, pp/rename_split_script.py, pp/split_netcdf_script.py, pp/ppval_script.py, pp/validate_script.py, pp/install_script.py, pp/configure_script_yaml.py, make/ (multiple files), make/gfdlfremake/ (multiple files), cmor/cmor_helpers.py, cmor/cmor_mixer.py, cmor/cmor_finder.py, cmor/cmor_config.py, cmor/cmor_yamler.py, yamltools/ (multiple files), run/frerun.py, check/frecheck.py

Issue ticket number and link (if applicable)

Checklist before requesting a review

• I ran my code
• I tried to make my code readable
• I tried to comment my code
• I wrote a new test, if applicable
• I wrote new instructions/documentation, if applicable
• I ran pytest and inspected it's output
• I ran pylint and attempted to implement some of it's feedback
• No print statements; all user-facing info uses logging module

[codecov]

Codecov Report

❌ Patch coverage is 51.72414% with 112 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.35%. Comparing base (836559e) to head (3b99405).

Files with missing lines	Patch %	Lines
fre/cmor/cmor_mixer.py	36.84%	12 Missing ⚠️
fre/make/gfdlfremake/platformfre.py	9.09%	10 Missing ⚠️
fre/yamltools/info_parsers/cmor_info_parser.py	11.11%	8 Missing ⚠️
fre/app/remap_pp_components/remap_pp_components.py	41.66%	7 Missing ⚠️
fre/app/regrid_xy/regrid_xy.py	14.28%	6 Missing ⚠️
fre/pp/ppval_script.py	16.66%	5 Missing ⚠️
fre/yamltools/combine_yamls_script.py	16.66%	5 Missing ⚠️
fre/yamltools/info_parsers/analysis_info_parser.py	16.66%	5 Missing ⚠️
fre/yamltools/info_parsers/pp_info_parser.py	28.57%	5 Missing ⚠️
...p/generate_time_averages/frenctoolsTimeAverager.py	33.33%	4 Missing ⚠️
... and 21 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #831      +/-   ##
==========================================
+ Coverage   84.12%   84.35%   +0.23%     
==========================================
  Files          71       71              
  Lines        4944     4974      +30     
==========================================
+ Hits         4159     4196      +37     
+ Misses        785      778       -7     

Flag	Coverage Δ
unittests	84.35% <51.72%> (+0.23%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
fre/__init__.py	100.00% <100.00%> (ø)
fre/app/generate_time_averages/cdoTimeAverager.py	100.00% <100.00%> (ø)
fre/app/mask_atmos_plevel/mask_atmos_plevel.py	100.00% <100.00%> (ø)
fre/cmor/cmor_config.py	97.56% <100.00%> (+0.03%)	⬆️
fre/cmor/cmor_finder.py	100.00% <100.00%> (ø)
fre/cmor/cmor_helpers.py	93.68% <100.00%> (-0.11%)	⬇️
fre/cmor/cmor_yamler.py	100.00% <100.00%> (ø)
fre/make/create_compile_script.py	97.91% <100.00%> (+0.04%)	⬆️
fre/make/create_docker_script.py	100.00% <100.00%> (ø)
fre/make/make_helpers.py	100.00% <100.00%> (ø)
... and 32 more

... and 1 file with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 836559e...3b99405. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ilaflott]

@copilot find other locations raise, and make sure they log the error too, use the new approach.

[Copilot]

@copilot find other locations raise, and make sure they log the error too, use the new approach.

Done in commit 4930028. Converted all raise ExcType(msg) statements across 44 files to use log_and_raise. Patterns handled:

• Simple raises: raise ExcType(msg) → log_and_raise(msg, ExcType)
• Chained exceptions: raise ExcType(msg) from exc → log_and_raise(msg, ExcType, exc=exc)
• Same message duplicated: removed the redundant fre_logger.error and replaced both with log_and_raise
• Different context + raise messages: kept the fre_logger.error for context, replaced the raise with log_and_raise
• Bare raise (re-raises in except blocks) — left unchanged
• Abstract method raise NotImplementedError() in timeAverager.py — left unchanged

[ilaflott]

@copilot updating the raise statements may have violated the assumptions of certain tests. spin up your own conda environment with an editable pip install, and tackle the testing failures one-by-one

[Copilot]

@copilot updating the raise statements may have violated the assumptions of certain tests. spin up your own conda environment with an editable pip install, and tackle the testi...

Investigated CI failures in commit 05ea559. The pytest step was passing — the only failures were pylint E0606: possibly-used-before-assignment errors in several files (combine.py, wrapper.py, rename_split_script.py, remap_pp_components.py, ppval_script.py, cmor_helpers.py, split_netcdf_script.py).

Root cause: replacing raise ExcType(msg) with log_and_raise(msg, ExcType) breaks pylint's control flow analysis. Pylint knows raise terminates a branch, but it doesn't know that log_and_raise always raises — so it flagged variables assigned only in non-else branches as "possibly unassigned."

Fix: added -> NoReturn return type annotation to log_and_raise in fre/__init__.py. Pylint now treats calls to it as control flow terminators, the same as bare raise. Pylint now exits with code 0 (score 8.56/10).

[ilaflott]

@copilot this is subtle, make sure this jives with logging best practices according to official python documentation, and PEP practices. one of the motivations for this is to get Traceback info into a log file specified by fre -l ./logger_output.log <rest of call>. that is not happening currently- craft a test to make it so

[Copilot]

@copilot this is subtle, make sure this jives with logging best practices according to official python documentation, and PEP practices. one of the motivations for this is to get `...

Done in 3b99405. Aligned log_and_raise with Python logging best practices per the logging HOWTO and PEP 282:

• exc_info=True when exc is provided — the idiomatic way to include a caught exception's traceback in log output (replaces logger.exception())
• stack_info=True always — ensures the call-site stack trace appears in every handler, including log files set up via fre -l
• stacklevel=2 attributes the log record to the caller, not log_and_raise itself

Two new tests (test_log_and_raise_stack_trace_in_log_file, test_log_and_raise_traceback_in_log_file) simulate fre -l ./logger_output.log by attaching a FileHandler and verify:

1. Stack (most recent call last): appears for simple raises
2. Traceback (most recent call last): + the original exception appear for chained raises

[ilaflott]

this isn't the right testing approach- the logfile behavior is more for the CLI:
fre -l logger_output.log ... and so one should use click's CliRunner