FinishingFieldsUpdates#78
Merged
Merged
Conversation
…form.py to handle “SN, WR, MR, PI” id labels. - Made format_filename_with_indices() function in create_peakindexing.py to modularize and improve how the indices get added to the filename string. - Made _extract_indices_from_files() function in callback_registrars.py to modularize index recognition, including allowing for a dynamic number of indices. - Updated ‘*’ wildcard generation to retain %d components in callback_registrars.py. - Created get_data_from_id() function in db.utils.py to read data_path and filenamePrefix from either reconstructed or original data. - Redefined number of pooled jobs from number of provided paths to longest sequence of semicolon-separated entries with the added get_num_inputs_from_fields() function in validation_helpers.py. - Added depthRange parsing to create_peakindexing.py validation - Removal of Threshold and Threshold Ratio from in create_peakindexing.py. - Renamed “param” to “field” for objects in validation_helpers.py and create_.py pages. - Added hdf file extensions to config.yaml.template and _filter_files_by_extension() function to callback_registrars.py to filter by them.
- Moved shared get_num_inputs_from_fields(), make_IDnumber(), and parse_IDnumber() functions to db_utils.py. - Changed both create_ files’ submit_parameters’s all_submit_params dict to use ctx.states rather than hard-coded items.
…nt scanNumber validation in create_peakindexing.py and create_wire_reconstruction.py - Added root_path to submit_parameters() callback in create_peakindexing.py and create_wire_reconstruction.py - create_peakindexing.py and create_wire_reconstruction.py’s submit_parameters() functions now cleanly handle any existing ids to make the output folder - create_peakindexing.py’s validate_peakindexing_inputs()function now uses get_data_from_id() function to get data_path - Unused parameters like “detectorCropX1” commented out in submit_parameters() and validate_inputs() callbacks
…roach for both wire reconstruction and peak indexing pages: ## Changes Made: ### 1. **Created Helper Functions** (in both files) - `build_output_folder_template()` in `create_wire_reconstruction.py` - `build_output_folder_template()` in `create_peakindexing.py` These functions build output folder templates based on available IDs from the database chain, with only the final action ID remaining as `%d`. ### 2. **Updated Page Load Callbacks** - Modified `load_scan_data_from_url()` in both files to use the helper functions - Templates are now built when the page loads with URL parameters - All known IDs (scanNumber, wirerecon_id, recon_id) are filled in at load time ### 3. **Simplified Submit Functions** - Replaced complex conditional logic (~25 lines) with simple `%` replacement - Submit functions now just replace the final `%d` placeholder with the next action ID - Reduced from multiple nested conditionals to a single try/except block ### 4. **Enhanced Shared Callback** - Updated `register_update_path_fields_callback()` in `callback_registrars.py` to accept: - `output_folder_id` parameter (optional) - `build_template_func` parameter (optional) - When both are provided, the callback now also updates the output folder field using the template function ### 5. **Updated Callback Registrations** - Both `create_wire_reconstruction.py` and `create_peakindexing.py` now pass: - `output_folder_id='outputFolder'` - `build_template_func=build_output_folder_template` ## Benefits: 1. **Cleaner Code**: Reduced complexity in submit functions 2. **Better UX**: Output folder template is built immediately when page loads 3. **Consistency**: Same approach used in both wire reconstruction and peak indexing 4. **Maintainability**: Template building logic centralized in helper functions 5. **Flexibility**: "Update Path Fields" button now also updates output folder ## Path Template Examples: **Wire Reconstruction:** - With scanNumber: `analysis/scan_276994/rec_%d/data` - Without scanNumber: `analysis/data_path/rec_%d/data` **Peak Indexing:** - With scanNumber only: `analysis/scan_276994/index_%d` - With scanNumber + wirerecon_id: `analysis/scan_276994/rec_5/index_%d` - With scanNumber + recon_id: `analysis/scan_276994/rec_3/index_%d` - Without scanNumber: `analysis/data_path/index_%d` All implementations follow the principle: build the template on page load with all known IDs, then simply replace the final `%d` with the next action ID on submit.
…ters by creating the _merge_field_values() helper function in callback_registrars.py, which is used in callback_registrars.py’s update_path_fields_from_id() callback and the load_scan_data_from_url() callback on both create_pages. Incidentally, this function can be used more generally for any field with comma-separated values. - Applied wildcard sorting to filename templates dropdown and applied auto-selection to choose the first in the list - Updated Filename dropdown options to show results for all data paths as delimiter-joined strings and be updated when valid results are entered between the delimiter (; default). Created _generate_dropdown_for_position() helper function to assist. The helper function `_generate_dropdown_for_position()` (lines 155-216): - Takes base pattern parts and a position to vary - Generates dropdown options with informative labels showing file counts/ranges - Only shows the informative label for the pattern being varied at that position - Other positions show plain patterns
…ters by creating the _merge_field_values() helper function in callback_registrars.py, which is used in callback_registrars.py’s update_path_fields_from_id() callback and the load_scan_data_from_url() callback on both create_pages. Incidentally, this function can be used more generally for any field with comma-separated values. - Applied wildcard sorting to filename templates dropdown and applied auto-selection to choose the first in the list - Updated Filename dropdown options to show results for all data paths as delimiter-joined strings and be updated when valid results are entered between the delimiter (; default). Created _generate_dropdown_for_position() helper function to assist. - Updated create_wire_reconstruction.py to parse IDnumbers like create_peakindexing, so that it can use WR values
…g the output path of WRs (like it would do for create_peakindexing) instead of the original input data path by providing "context" parameter to get_data_from_id() function in db_utils.py.
…unction so that that the lengths of all ID-type lists in the returned dict match the number of parent IDs (padded with Nones)
- Removed a bit of unnecessary code in create_wire_reconstruction.py for storing wirerecon_id.
…ead of delimited (";") per path
…for_input to avoid repeated inputs, as well as adding/expanding optional_params lists (create_wire_reconstruction.py and create_peakindexing.py). - Standardized semicolon delimiter usage. - Fixed an issue with comma-delimiting filenamePrefix values by adding that step to wire_recon_form and peakindex_form. - Fixed an href id alignment issue in scan.py.
- Removed _generate_dropdown_for_position() helper function in favor of custom dropdown_options generation for Modes 1 (generates for all positions) and 2 (uses single position). -- Mode 1 now displays options for each field independently. -- Added styling for labels in Mode 2 (when working with mismatched position). - New _generate_pattern_label() helper function that applies pattern and range information to all positions. - Use _merge_field_values in both Modes for finalizing generated field options.
matthewrdiamond
commented
Dec 3, 2025
matthewrdiamond
left a comment
matthewrdiamond
left a comment
Collaborator
Author
There was a problem hiding this comment.
Resolves Issues #65-67
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Complete Synopsis of Changes in the FinishingFieldsUpdates Branch
I've successfully reviewed the git commits in your current branch. Here's a comprehensive synopsis of all the changes, organized by functional area:
1. File Name Pattern Handling & UI Improvements
_generate_dropdown_for_position()in favor of mode-specific dropdown generation_generate_pattern_label()helper applies patterns across all positions_merge_field_valuesfor finalizing options2. Validation System Enhancements
validate_for_inputto eliminate redundancy3. ID Number Management
IDnumber implementation (07daa3e): Added comprehensive ID handling
get_num_inputs_from_fields(),make_IDnumber(),parse_IDnumber()) in db_utils.pyDeduplication fix (a935ac6): Fixed parse_IDnumber() to ensure ID-type lists match parent ID counts with proper None padding
4. Path and Data Handling Fixes
_merge_field_values()helper to properly handle comma-separated values during page loads5. Output Folder Management
6. Bug Fixes
7. Code Organization
Impact Summary
These changes represent a significant improvement in:
The branch appears ready for integration, with comprehensive improvements to form handling, validation, and data processing workflows.