feat: handle zip lcms#212
Draft
baolanlequang wants to merge 74 commits into
Draft
Conversation
* Fix conditional header generation for NMR JCAMP export * Add tests to ensure that empty headers are not added to nmr files + Add a test file with empty headers
…ter (#220) * Fix data normalization and improve handling of runs in JcampMSConverter * Add unit test for JCAMP single point handling in the last line * Update requirements.txt to use specific branch of nmrglue * Change nmrglue version for unit test in CI
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.3 to 3.0.6. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@3.0.3...3.0.6) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.6. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@3.1.4...3.1.6) --- updated-dependencies: - dependency-name: jinja2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
…MP-DX files (#225) ensure the data block from `__gen_ms_spectra` ends with a new-line so that '##END NTUPLES=MASS SPECTRUM' is on a separate line in MS JCAMP-DX files and spectra viewer can handle threshold properly
Bump jinja2 from 3.1.4 to 3.1.6
Bumps [gunicorn](https://github.com/benoitc/gunicorn) from 22.0.0 to 23.0.0. - [Release notes](https://github.com/benoitc/gunicorn/releases) - [Commits](benoitc/gunicorn@22.0.0...23.0.0) --- updated-dependencies: - dependency-name: gunicorn dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
- Add `--simAsSpectra` flag to the msconvert command to handle SIM data properly. - Update tests to cover ESI SIM RAW file conversion, ensuring future compatibility. - Update CI workflow to start the required Docker container (`msconvert_docker`) and set correct permissions on the temporary directory (`chem_spectra/tmp`) for successful test execution.
Bump gunicorn from 22.0.0 to 23.0.0
Fix IR preview image margin regression
fix: certificates issue
There was a problem hiding this comment.
Pull Request Overview
This PR introduces support for LCMS (Liquid Chromatography–Mass Spectrometry) data by adding a new converter (LCMSBaseConverter), composer (LCMSComposer), integrating them into the transformer model and API endpoint, and providing unit tests to validate the end-to-end flow.
- Added
tests/lib/converter/lcms/test_lcms_converter.pyto verify raw CSV reading into structured data. - Added
tests/lib/composer/test_lcms_composer.pyto ensure JCAMP-DX generation for TIC, UV-VIS, and m/z spectra. - Updated
TransformerModel,transform_api.py, and new composer/converter modules to wire LCMS support into the existing pipeline.
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/lib/converter/lcms/test_lcms_converter.py | New unit tests for LCMSBaseConverter |
| tests/lib/composer/test_lcms_composer.py | New unit tests for LCMSComposer |
| chem_spectra/model/transformer.py | Integrated LCMS detection and dispatch in TransformerModel |
| chem_spectra/lib/converter/lcms/base.py | Implemented LCMSBaseConverter for reading LCMS CSV files |
| chem_spectra/lib/composer/lcms.py | Implemented LCMSComposer for JCAMP-DX output |
| chem_spectra/controller/transform_api.py | Extended API to handle LCMSComposer responses |
Comments suppressed due to low confidence (4)
tests/lib/converter/lcms/test_lcms_converter.py:5
- There's a typo in the alias:
LCMSConvetershould beLCMSConverterto match the class name.
from chem_spectra.lib.converter.lcms.base import LCMSBaseConverter as LCMSConveter
tests/lib/composer/test_lcms_composer.py:92
- This test function duplicates the name of the positive m/z test defined at line 77; rename one to
test_lcms_composer_mz_spectra_negativeto avoid collision.
def test_lcms_composer_mz_spectra_positive(zip_file):
chem_spectra/controller/transform_api.py:83
- [nitpick] The
spc_typefor LCMSComposer is set to'hplc', which may be confusing. Consider using'lcms'to reflect the actual data type.
rsp.headers['X-Extra-Info-JSON'] = json.dumps({'spc_type': 'hplc', 'invalid_molfile': invalid_molfile})
chem_spectra/lib/converter/lcms/base.py:31
- There's a typo in the variable name
tic_postive_data; it should betic_positive_datato maintain consistency and avoid confusion.
tic_postive_data = data_frame.to_dict(orient='list')
| ] | ||
| for idx, _ in enumerate(xs): | ||
| my_content = '{}, {};\n'.format(xs[idx], ys[idx]) | ||
| msspc += my_content |
There was a problem hiding this comment.
Concatenating a string into a list using += will split it into characters. Use msspc.append(my_content) or msspc += [my_content] instead.
Suggested change
| msspc += my_content | |
| msspc.append(my_content) |
| '##ORIGIN=\n', | ||
| '##OWNER=\n', | ||
| '##SPECTROMETER/DATA SYSTEM=\n', | ||
| '##$CSCATEGORY=UVVIS SPECTRUM\n', |
There was a problem hiding this comment.
The m/z spectra composer mistakenly includes the UVVIS category header. Replace this line with the appropriate category variable or remove it to avoid conflicting headers.
Suggested change
| '##$CSCATEGORY=UVVIS SPECTRUM\n', |
Bumps [requests](https://github.com/psf/requests) from 2.32.2 to 2.32.4. - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.32.2...v2.32.4) --- updated-dependencies: - dependency-name: requests dependency-version: 2.32.4 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…240) If the first .jdx inside a BagIt archive had no XY data, JcampNIConverter raised a KeyError. ChemSpectra stopped processing and returned a 500 error, even when valid files followed. (Issue #238) - Wraps the creation of JcampNIConverter in a try/except. - Files that miss both XYDATA and PEAKTABLE / XYPOINTS are skipped and logged. - Processing continues with the remaining (valid) files. => - ChemSpectra now imports BagIt archives even if some .jdx files are empty. - Users see the valid spectra; empty files are silently ignored (a warning is logged on the backend).
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.19 to 2.5.0. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@1.26.19...2.5.0) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.5.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… add devcontainer (#232) - remove unncessary and old files - update install instructions - add a devcontainer - Introduce pyproject.toml for build configuration - Move dependency management from requirements.txt to pyproject.toml - Adopte loose versioning (e.g., `flask==3.*` instead of fixed patch versions) for core packages to ease maintenance and allow minor/patch upgrades automatically - Add project metadata: name, version, description, license, URLs, keywords, and Python version requirement (>=3.12) - Define build backend as setuptools - Replace `nmrglue` editable install with pinned commit `4005640` - Ensure version pins for Python 3.12 compatibility - Updated dependencies in requirements.txt: • Flask: 2.2.5 → 3.1.1 • Gunicorn: 20.0.0 → 23.0.0 • idna: 3.7 → 3.10 • itsdangerous: 2.0 → 2.2.0 • Jinja2: 3.1.2 → 3.1.6 • MarkupSafe: 2.1.2 → 3.0.2 • matplotlib: 3.7.3 → 3.10.5 • netCDF4: 1.5.3 → 1.7.2 • numpy: 1.22.4 → 2.3.2 • pandas: 2.0.3 → 2.3.1 • Pillow: 10.3.0 → 11.3.0 • pluggy: 0.12.0 → 1.6.0 • pymzml: 2.5.2 → 2.5.11 • pyopenms: 2.6.0 → 3.4.0 • pyparsing: 2.4.2 → 3.2.3 • pytest: 7.2.0 → 8.4.1 • python-dateutil: 2.8.2 → 2.9.0.post0 • pytz: 2023.3 → 2025.2 • rdkit: 2023.9.1 → 2025.3.5 • regex: 2019.4.9 → 2025.7.34 • scipy: 1.7.3 → 1.16.1 • six: 1.11.0 → 1.17.0 • urllib3: 1.x/2.x mix → 2.5.0 • Werkzeug: 3.0.6 → 3.1.3
bump tag from 1.2.7 to v1.3.0 https://github.com/ComPlat/chem-spectra-app/releases/tag/v1.3.0
* add dockerfile * add files for dev2deploy process
…F-8 JCAMP support (#248)
Handle incomplete Bruker pdata folders: skip missing 1r/2rr and fall back to FID skip Bruker processed dirs without binaries instead of raising error
…poser, update dependencies to Flask 2.x and Gunicorn 22.x, and remove obsolete LCMS files. Introduce new chemotion converter profiles for LCMS.
Nicolass67
force-pushed
the
211-process-lcms-data
branch
from
fa8eb12 to
0d8f16a
Compare
Co-authored-by: Cursor <cursoragent@cursor.com>
…S image generation from peak.jdx files
…e ZIP responses with JCAMP and images
…t helper functions in share.py for improved filename handling
…ms functions, and clean up related code in chemotion_converter_lcms.py to streamline LCMS processing.
…improve retention of retention time source
implements backend support for the Current / Current Density mode in the CV layout. It introduces: - Parsing and persistence of new CV parameters: - areaValue - areaUnit - useCurrentDensity - Writing these parameters into the JCAMP metadata: - ##$CSWEAREAVALUE - ##$CSWEAREAUNIT - ##$CSCURRENTMODE - Automatic Y-axis label override when current density mode is enabled. - Dynamic Y-axis scaling and formatting when current density is active, including area-based normalization and scientific notation handling. When useCurrentDensity is enabled, the backend converts the current to current density based on the provided electrode area and adapts both metadata and rendering accordingly. This ensures consistent data export, correct labeling, and proper visualization behavior for CV current density mode.
…params functions, and implement data extraction for MS peaks in chemotion_converter_lcms.py to enhance LCMS processing.
…by refining active MS threshold handling and implementing conditional coloring for MS peaks based on the threshold. Default to 5% when the threshold is missing or uninitialized.
…upport for lcms_mz_page header extraction and conditional inclusion in JCAMP output.
… requests and refresh logic, enhancing data integrity checks and file reading methods. Update TransformerModel to ensure robust filename handling when retrieving JCAMP files.
…ra. Consolidate LCMS response building into dedicated helper functions, streamline parameter extraction, and enhance file handling for improved data integrity. Update configuration settings and remove unused code to optimize performance.
…Update to_zip_flat_bagit_response for single-level zips, enhancing file naming conventions and entry stem management. Adjust tests to reflect changes in UV/Vis handling within LCMS composer path.
…SConverterAppComposer for improved file response generation. Update TransformerModel to utilize build_lcms_composer for LC/MS file processing, ensuring robust handling of various data types. Refactor LCMSConverterAppComposer to include tf_csv method and optimize image preview logic.
…sponse generation. Integrate UVVIS peak detection and streamline the classification of LCMS stems. Update tests to validate new functionality and ensure robust handling of mixed data types in composer logic.
All contributors consented. Copyright holder: Nicole Jung - Karlsruhe Institute of Technology (KIT), 2016–2026.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.