Refactor roman_catalog_process.py to prevent create_informer_stage() from running every time the module is executed.

[stscijgbot-rstdms]

Issue RCAL-1039 was created on JIRA by Mairan Teodoro:

The create_informer_stage() method in roman_catalog_process.py is currently executed every time the process() method is called, regardless of whether the informer stage has already created the model needed for the estimator. This results in unnecessary computation and inefficiency, especially when processing large datasets.

To optimize the workflow, the code should be refactored to check if the result from the informer stage already exists (e.g., by verifying the presence of the model file or a cached object) before creating it. If the informer stage exists, it should be reused instead of being recreated.

 

Technical Notes:

• The model file is saved at {}$LEPHAREWORK/roman_model.pk{}. This file can be used as a marker to determine if the informer stage has already been created.
• Perhaps we should consider using a lazy initialization pattern or caching mechanism to manage the informer stage.

Acceptance Criteria:

Add a mechanism to check if the informer stage or its output (e.g., the model file) already exists.

If the informer stage exists, skip the creation process and reuse the existing stage or model.

Ensure the refactored code maintains compatibility with the current workflow and configuration.

Add appropriate logging to indicate whether the informer stage is being reused or recreated.

Update the unit tests to cover both scenarios: creating a new informer stage and reusing an existing one.

[stscijgbot-rstdms]

Comment by Eddie Schlafly on JIRA:

Do you know if technically that pickle file is all that lephare needs for the estimator step?  That was my impression from the flow but I did not actually test whether it used more stuff in $LEPHAREWORK.

[stscijgbot-rstdms]

Comment by Mairan Teodoro on JIRA:

As far as I know, yes, the pickle file with the model is all the estimator needs.

[stscijgbot-rstdms]

Comment by Mairan Teodoro on JIRA:

PRs for review (implementation and RTD updates):

• RCA-1039: refactor roman_catalog_process.py to only run informer stage when necessary. roman_photoz#14
• Update docs after refactoring roman_photoz. roman_photoz#16