Pydantic 2 (#3)

* change install pin

* normalise imports

* finalise imports

* try bulk replace imports

* Revert "finalise imports"

This reverts commit baad442.

* Revert "Revert "finalise imports""

This reverts commit 840abd5.

* Revert "try bulk replace imports"

This reverts commit 1cb8a82.

* bulk replace again

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add a shortcut for not training if all epochs have already been trained for.

* Add weight decay handling in Trainer.

* Fix weight_decay math.

* Use torch.no_grad instead of model.eval.

* Add model.eval() and model.train() calls.

* Add Trainer weight decay loss.

* Switch default batch size to be -1.

* Add device Field for DatasetConfig to automatically move data to given device.

* Add option for batch gradient normalization.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix comment.

* Add forgotten optimizer step.

* Remove print statement.

* bump CI

* Add classes for L1Loss and SmoothL1Loss.

* Fix loss calculation for multiple prediction/loss values.

* Add LossFunctionConfig options for new loss classes.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add CLI arg for LayerNorm option.

* Fix typo.

* Handle missing column.

* Some plotting improvements.

* Add convenience methods for loading in and formatting groups of TrainingPredictionTrackers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add multiprocessing capability to TPT loading functions.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add handling for no extract_epochs passed.

* Add a shortcut for not training if all epochs have already been trained for.

* Add weight decay handling in Trainer.

* Fix weight_decay math.

* Use torch.no_grad instead of model.eval.

* Add model.eval() and model.train() calls.

* Add Trainer weight decay loss.

* Switch default batch size to be -1.

* Add device Field for DatasetConfig to automatically move data to given device.

* Add option for batch gradient normalization.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix comment.

* Add forgotten optimizer step.

* Remove print statement.

* Add classes for L1Loss and SmoothL1Loss.

* Fix loss calculation for multiple prediction/loss values.

* Add LossFunctionConfig options for new loss classes.

* Add CLI arg for LayerNorm option.

* Fix typo.

* Handle missing column.

* Some plotting improvements.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add convenience methods for loading in and formatting groups of TrainingPredictionTrackers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add multiprocessing capability to TPT loading functions.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add handling for no extract_epochs passed.

* Fix minor errors.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update asapdiscovery-ubuntu-latest.yml

* Update asapdiscovery-macOS-latest.yml

* Update asapdiscovery-macOS-12.yml

* Separate environment files (#9)

* add initial ubuntu and mac latest environments and update data.yaml github workflow to point to it

* add a python script to find imports in files

* add plip dependency

* temporarily fix failing openfe warning catching

* update detect_imports script and add modeling yaml files

* update data and modeling workflows

* update docking CI and add docking yaml

* add dataviz to docking dependencies

* add simulation yamls

* add simulation yamls

* add modeling to simulation dependencies

* add modeling to simulation dependencies

* add macos to data workflow

* update workflows

* remove unnecessary packages from modeling workflow

* add simulation workflow package

* update simulation deps

* add ml env and update env names

* add torchdata to docking deps

* add openmmtools to deps

* add ml to simulation deps

* remove multiprocessing from ML deps

* remove multiprocessing from simulation deps

* update ml envs

* make mtenn simulation dep

* add modeling to ml deps

* add semver to docking deps

* move cli_args to have lazy imports and add spectrum env files

* also run on the pydantic_2 branch

* bump ci

* test running with multiple env files

* move openeye testing

* use different micromamba command

* fix missing asapdiscovery-modeling install

* Update ML workflow file to include MacOS and latest micromamba version.

* add workflows

* Rename macOS conda-envs directory.

* add workflows

* Environment file for cli ci ubuntu

* Environment file for cli ci macos

* workflow file for CLI CI

* add workflows

* fix workflows for simulation and spectrum

* Remove MacOS from ML workflow.

* add workflows

* install environments for other packages

* install environments for other packages

* adding data missing dependencies

* environment file for alchemy ubuntu-latest

* environment file for alchemy macos-latest

* workflow file update for alchemy

* macos uncapitalized in workflows

* Add pydantic-settings.

* add dataviz env files

* Add mamba installing dependencies in ML workflow file.

---------

Co-authored-by: Mike Henry <[email protected]>
Co-authored-by: kaminow <[email protected]>
Co-authored-by: Iván Pulido <[email protected]>
Co-authored-by: Maria Castellanos <[email protected]>
Co-authored-by: kaminow <[email protected]>

* make a real workflows env file

* remove workflow deps yaml

* don't run workflows CI on macos

* use bioconda for logomaker

* fix dataviz workflow

* fix dataviz workflow

* update dataviz workflow install

* rename dataviz env

* fix dataviz env

* remove modeling as a dep for data

* move trim_small_chains to backend

* move trim_small_chains to backend

* adapt to str enum update

* fix .schema import

* fix .schema import

* remove macos-ci workflow

* fix from .backend import

* move prepped structure and docking stuff to modeling and docking envs

* move plip to dataviz

* fix PreppedComplex use in conftests

* most postera api to workflows

* most postera api to workflows

* fix imports

* fix imports

* test fintscore in dataviz

* remove other subpackages for data ci workflow

* fix prep tests

* fix prep tests

* fix imports

* fix imports

* fix imports

* fix imports

* fix imports

* remove cudatoolkit dep from simulation.yaml (#39)

* switch to using the new buckets (#41)

* switch to using the new buckets

* missed a few urls

---------

Co-authored-by: Alexander Matthew Payne <[email protected]>

* Spectrum struct changes (#10)

* add rmsd-coloring

* Fix chain visualization

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix pre-commit-ci

* Add the option to align struct directory and pdb file

* Needed changes on seq_alignment

* Implement fitness-alignment to align wrt fasta

* Add more options to struct-alignment

* changes to fitness-alignment

* Additions to align_seq_match

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* precommit ci fixes

* fix seq-align test

* struct-alignment tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add structure alignment tests

* Fix struct-alignment cfold test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* bump CI

* bump CI

* Added a check for blast.csv file

* Bump CI

* TODO to fix test here with "hidden" error

* Checking subprocess return code, print error

* added mafft dependancy

* add mafft to ubuntu yaml

* remove macOS skips from tests

---------

Co-authored-by: Castellanos Morales <[email protected]>
Co-authored-by: Castellanos Morales <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Maria Castellanos <[email protected]>
Co-authored-by: hmacdope <[email protected]>
Co-authored-by: Iván Pulido <[email protected]>

* Dropping bespokefit in codebase (#22)

* Dropping bespokefit in codebase

* using pydantic.v1 instead

* trying to fix CI issues by pinning to torchdata 0.8.0 for compatibility with dgl 2.3

* Fixing env path for alchemy CI

* fixing package name for alchemiscale

* fixing package name for alchemiscale on macos

* pinning openfe versions

* Bump CI

* pinning openfe versions for linux

* pinning compatibility to minor versions

* dropping python pins

* dropping python pins for every package

---------

Co-authored-by: Mike Henry <[email protected]>
Co-authored-by: chrisiacovella <[email protected]>

* update cli-ci (#48)

* update cli-ci

* change order of OE license checks

* add simulation to cli deps

* add dataviz and spectrum to deps

* fixing pydantic issues in alchemy (#52)

* FIXING ISSUES with pydantic models to handle ScopedKey (including serialization); fixed issues with passing json files to openfe 
---------

Co-authored-by: Alexander Matthew Payne <[email protected]>

* separate docking scorers into separate files (#57)

* separate scorers into separate files to make it possible to import one without depending on the others

* fix broken imports

* xfail ml scorer tests

* Lazy ml imports workflows (#60)

* Lazy imports for ML scorer in workflows.

* Adding warning message. Experimental ML scorer.

* Moving warnings before imports. Since imports are expected to fail.

* XFailing ML workflow tests separately for now

* Correcting the multiple parameters

* Changing default for ml_score option

* Moving second ML import as lazy import

* Missed one import in ligand transfer docking.

* Just testing new default behavior. No ML score.

* set ml_score cli default to false

* add try except to ml cli

* remove ml score from docking test input

* make ml score default off in all docking workflows

* update asapdiscovery.cli.cli to comment out ml cli

---------

Co-authored-by: Alexander Matthew Payne <[email protected]>

* xskip ML testing in CLI package (#61)

* xskip ml in cli

* changed to skip in parameterize

* Add scoring scripts to spectrum (#55)

* Add score files to spectrum

* fix package error

* update data imports

* Adding tests for scoring

* Add vina requirements

* Fix vina test to remove bad requirements

* Add minimization test

* calculate_rmsd-docs

* Move spectrum cli to workflows

* Address comments

* Apply suggestions from code review

Co-authored-by: Iván Pulido <[email protected]>

* delete hcc channel

* Apply suggestions from code review

Co-authored-by: Iván Pulido <[email protected]>

* Comments on score.py

* Update asapdiscovery-spectrum/asapdiscovery/spectrum/score.py

Co-authored-by: Iván Pulido <[email protected]>

* Improve logging

* Last requested changes

---------

Co-authored-by: Maria Castellanos <[email protected]>
Co-authored-by: Iván Pulido <[email protected]>

---------

Co-authored-by: hmacdope <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: kaminow <[email protected]>
Co-authored-by: kaminow <[email protected]>
Co-authored-by: Alexander Matthew Payne <[email protected]>
Co-authored-by: kaminow <[email protected]>
Co-authored-by: Iván Pulido <[email protected]>
Co-authored-by: Maria Castellanos <[email protected]>
Co-authored-by: Alex <[email protected]>
Co-authored-by: Maria Castellanos <[email protected]>
Co-authored-by: Castellanos Morales <[email protected]>
Co-authored-by: Castellanos Morales <[email protected]>
Co-authored-by: chrisiacovella <[email protected]>
Co-authored-by: Alexander Matthew Payne <[email protected]>

Author: 12 people

.github/workflows/alchemy.yaml

@@ -31,7 +31,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [ubuntu-latest]
+        os: [ubuntu-latest, macos-latest]
         python-version: ["3.11"]
 
     env:
@@ -53,13 +53,9 @@ jobs:
 
     # More info on options: https://github.com/mamba-org/provision-with-micromamba
     - name: Setup Conda Environment
-      uses: mamba-org/setup-micromamba@v1
+      uses: mamba-org/setup-micromamba@v2
       with:
-        # default - will pull down 2.0 which we don't want!
-        # micromamba-version: latest
-        # pin to latest 1.x release
-        micromamba-version: '1.5.10-0'
-        environment-file: devtools/conda-envs/asapdiscovery-${{ matrix.os }}.yml
+        environment-file: devtools/conda-envs/${{ matrix.os }}/alchemy.yaml
         environment-name: asapdiscovery
         cache-environment: true
         cache-downloads: true
@@ -78,14 +74,10 @@ jobs:
     - name: Install packages
       run: |
         python -m pip install -e ./asapdiscovery-alchemy --no-deps
-        python -m pip install -e ./asapdiscovery-cli --no-deps
         python -m pip install -e ./asapdiscovery-data --no-deps
-        python -m pip install -e ./asapdiscovery-dataviz --no-deps
         python -m pip install -e ./asapdiscovery-docking --no-deps
-        python -m pip install -e ./asapdiscovery-ml --no-deps
         python -m pip install -e ./asapdiscovery-modeling --no-deps
-        python -m pip install -e ./asapdiscovery-simulation --no-deps
-        python -m pip install -e ./asapdiscovery-spectrum --no-deps
+        python -m pip install -e ./asapdiscovery-workflows --no-deps
         micromamba list
 
     - name: Run tests
@@ -96,9 +88,10 @@ jobs:
         # run each package test suite; append to coverage file
         # Exit immediately if a command exits with a non-zero status.
         set -e
-        pytest -n auto --durations=10 -v --cov-report=xml --cov-report=term --color=yes \
+        # can only run tests on a single worker because "Only one live display may be activate at once"
+        pytest -n 1 --durations=10 -v --cov-report=xml --cov-report=term --color=yes \
                        --cov=asapdiscovery-alchemy \
-                       asapdiscovery-alchemy/asapdiscovery/alchemy/tests
+                       asapdiscovery-alchemy/asapdiscovery/alchemy/
 
     - name: Upload Code Coverage to Codecov
       uses: codecov/codecov-action@v3

.github/workflows/cli.yaml

@@ -31,7 +31,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [ubuntu-latest]
+        os: [ubuntu-latest, macos-latest]
         python-version: ["3.11"]
 
     env:
@@ -53,42 +53,47 @@ jobs:
 
     # More info on options: https://github.com/mamba-org/provision-with-micromamba
     - name: Setup Conda Environment
-      uses: mamba-org/setup-micromamba@v1
+      uses: mamba-org/setup-micromamba@v2
       with:
-        # default - will pull down 2.0 which we don't want!
-        # micromamba-version: latest
-        # pin to latest 1.x release
-        micromamba-version: '1.5.10-0'
-        environment-file: devtools/conda-envs/asapdiscovery-${{ matrix.os }}.yml
-        environment-name: asapdiscovery
+        environment-file: devtools/conda-envs/${{ matrix.os }}/cli.yaml
+        environment-name: cli-ci
         cache-environment: true
         cache-downloads: true
         cache-environment-key: environment-${{ steps.date.outputs.date }}
         cache-downloads-key: downloads-${{ steps.date.outputs.date }}
         create-args: >-
           python==${{ matrix.python-version }}
 
-    - name: Test OE License & Write License to File
-      env:
-        OE_LICENSE_TEXT: ${{ secrets.OE_LICENSE }}
-      run: |
-        echo "${OE_LICENSE_TEXT}" > ${OE_LICENSE}
-        python -c "import openeye; assert openeye.oechem.OEChemIsLicensed(), 'OpenEye license checks failed!'"
-
     - name: Install packages
       run: |
-        python -m pip install -e ./asapdiscovery-alchemy --no-deps
-        python -m pip install -e ./asapdiscovery-cli --no-deps
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/data.yaml
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/docking.yaml
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/modeling.yaml
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/alchemy.yaml
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/cli.yaml
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/workflows.yaml
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/simulation.yaml
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/dataviz.yaml
+        micromamba install -n cli-ci --file devtools/conda-envs/${{ matrix.os }}/spectrum.yaml
         python -m pip install -e ./asapdiscovery-data --no-deps
-        python -m pip install -e ./asapdiscovery-dataviz --no-deps
         python -m pip install -e ./asapdiscovery-docking --no-deps
         python -m pip install -e ./asapdiscovery-ml --no-deps
         python -m pip install -e ./asapdiscovery-modeling --no-deps
+        python -m pip install -e ./asapdiscovery-alchemy --no-deps
+        python -m pip install -e ./asapdiscovery-workflows --no-deps
+        python -m pip install -e ./asapdiscovery-cli --no-deps
         python -m pip install -e ./asapdiscovery-simulation --no-deps
+        python -m pip install -e ./asapdiscovery-dataviz --no-deps
         python -m pip install -e ./asapdiscovery-spectrum --no-deps
-        python -m pip install -e ./asapdiscovery-workflows --no-deps
         micromamba list
 
+    - name: Test OE License & Write License to File
+      env:
+        OE_LICENSE_TEXT: ${{ secrets.OE_LICENSE }}
+      run: |
+        echo "${OE_LICENSE_TEXT}" > ${OE_LICENSE}
+        python -c "import openeye; assert openeye.oechem.OEChemIsLicensed(), 'OpenEye license checks failed!'"
+
     - name: Run tests
       run: |
         # run each package test suite; append to coverage file

.github/workflows/data.yaml

@@ -31,7 +31,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [ubuntu-latest]
+        os: [ubuntu-latest, macos-latest]
         python-version: ["3.11"]
 
     env:
@@ -53,14 +53,10 @@ jobs:
 
     # More info on options: https://github.com/mamba-org/provision-with-micromamba
     - name: Setup Conda Environment
-      uses: mamba-org/setup-micromamba@v1
+      uses: mamba-org/setup-micromamba@v2
       with:
-        # default - will pull down 2.0 which we don't want!
-        # micromamba-version: latest
-        # pin to latest 1.x release
-        micromamba-version: '1.5.10-0'
-        environment-file: devtools/conda-envs/asapdiscovery-${{ matrix.os }}.yml
-        environment-name: asapdiscovery
+        environment-file: devtools/conda-envs/${{ matrix.os }}/data.yaml
+        environment-name: data
         cache-environment: true
         cache-downloads: true
         cache-environment-key: environment-${{ steps.date.outputs.date }}
@@ -77,14 +73,7 @@ jobs:
 
     - name: Install packages
       run: |
-        python -m pip install -e ./asapdiscovery-cli --no-deps
         python -m pip install -e ./asapdiscovery-data --no-deps
-        python -m pip install -e ./asapdiscovery-dataviz --no-deps
-        python -m pip install -e ./asapdiscovery-docking --no-deps
-        python -m pip install -e ./asapdiscovery-ml --no-deps
-        python -m pip install -e ./asapdiscovery-modeling --no-deps
-        python -m pip install -e ./asapdiscovery-simulation --no-deps
-        python -m pip install -e ./asapdiscovery-spectrum --no-deps
         micromamba list
 
     - name: Run tests

.github/workflows/dataviz.yaml

@@ -53,39 +53,40 @@ jobs:
 
     # More info on options: https://github.com/mamba-org/provision-with-micromamba
     - name: Setup Conda Environment
-      uses: mamba-org/setup-micromamba@v1
+      uses: mamba-org/setup-micromamba@v2
       with:
-        # default - will pull down 2.0 which we don't want!
-        # micromamba-version: latest
-        # pin to latest 1.x release
-        micromamba-version: '1.5.10-0'
-        environment-file: devtools/conda-envs/asapdiscovery-${{ matrix.os }}.yml
-        environment-name: asapdiscovery
+        environment-file: devtools/conda-envs/${{ matrix.os }}/dataviz.yaml
+        environment-name: dataviz
         cache-environment: true
         cache-downloads: true
         cache-environment-key: environment-${{ steps.date.outputs.date }}
         cache-downloads-key: downloads-${{ steps.date.outputs.date }}
         create-args: >-
           python==${{ matrix.python-version }}
 
-    - name: Test OE License & Write License to File
-      env:
-        OE_LICENSE_TEXT: ${{ secrets.OE_LICENSE }}
-      run: |
-        echo "${OE_LICENSE_TEXT}" > ${OE_LICENSE}
-        python -c "import openeye; assert openeye.oechem.OEChemIsLicensed(), 'OpenEye license checks failed!'"
-
     - name: Install packages
       run: |
-        python -m pip install -e ./asapdiscovery-cli --no-deps
+        micromamba install -n dataviz --file devtools/conda-envs/${{ matrix.os }}/data.yaml
+        micromamba install -n dataviz --file devtools/conda-envs/${{ matrix.os }}/docking.yaml
+        micromamba install -n dataviz --file devtools/conda-envs/${{ matrix.os }}/simulation.yaml
+        micromamba install -n dataviz --file devtools/conda-envs/${{ matrix.os }}/modeling.yaml
+        micromamba install -n dataviz --file devtools/conda-envs/${{ matrix.os }}/spectrum.yaml
+        
         python -m pip install -e ./asapdiscovery-data --no-deps
+        python -m pip install -e ./asapdiscovery-modeling --no-deps
+        python -m pip install -e ./asapdiscovery-cli --no-deps
         python -m pip install -e ./asapdiscovery-dataviz --no-deps
         python -m pip install -e ./asapdiscovery-docking --no-deps
-        python -m pip install -e ./asapdiscovery-ml --no-deps
-        python -m pip install -e ./asapdiscovery-modeling --no-deps
         python -m pip install -e ./asapdiscovery-simulation --no-deps
         python -m pip install -e ./asapdiscovery-spectrum --no-deps
         micromamba list
+    
+    - name: Test OE License & Write License to File
+      env:
+        OE_LICENSE_TEXT: ${{ secrets.OE_LICENSE }}
+      run: |
+        echo "${OE_LICENSE_TEXT}" > ${OE_LICENSE}
+        python -c "import openeye; assert openeye.oechem.OEChemIsLicensed(), 'OpenEye license checks failed!'"
 
     - name: Run tests
       env:

.github/workflows/docking.yaml

@@ -64,7 +64,7 @@ jobs:
         # micromamba-version: latest
         # pin to latest 1.x release
         micromamba-version: '1.5.10-0'
-        environment-file: devtools/conda-envs/asapdiscovery-${{ matrix.os }}.yml
+        environment-file: devtools/conda-envs/${{ matrix.os }}/docking.yaml
         environment-name: asapdiscovery
         cache-environment: true
         cache-downloads: true
@@ -82,13 +82,11 @@ jobs:
 
     - name: Install packages
       run: |
-        python -m pip install -e ./asapdiscovery-cli --no-deps
         python -m pip install -e ./asapdiscovery-data --no-deps
         python -m pip install -e ./asapdiscovery-dataviz --no-deps
         python -m pip install -e ./asapdiscovery-docking --no-deps
         python -m pip install -e ./asapdiscovery-ml --no-deps
         python -m pip install -e ./asapdiscovery-modeling --no-deps
-        python -m pip install -e ./asapdiscovery-simulation --no-deps
         python -m pip install -e ./asapdiscovery-spectrum --no-deps
         micromamba list