Skip to content

Fix: pre-commit hook dataset-config-validation #269

Open
@Bento007

Description

@Bento007

The hook makes assumptions that dependencies are install in the local python environment. This causes it to fail if those dependencies are missing. The pre-commit hook should be as independent from the local environment as possible to avoid running into issues running on different machines. The dependencies should be included in the pre-commit hook or installed as part of the make recipe it uses.

Error Message

dataset-config-validation................................................Failed
- hook id: dataset-config-validation
- exit code: 2

python ingestion_config/v1.0.0/ingestion_config_validate.py
Traceback (most recent call last):
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_validate.py", line 12, in <module>
    from ingestion_config_models_extended import ExtendedValidationContainer
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_models_extended.py", line 13, in <module>
    from async_lru import alru_cache
ModuleNotFoundError: No module named 'async_lru'
make: *** [validate-configs] Error 1
python ingestion_config/v1.0.0/ingestion_config_validate.py
Traceback (most recent call last):
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_validate.py", line 12, in <module>
    from ingestion_config_models_extended import ExtendedValidationContainer
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_models_extended.py", line 13, in <module>
    from async_lru import alru_cache
ModuleNotFoundError: No module named 'async_lru'
make: *** [validate-configs] Error 1
python ingestion_config/v1.0.0/ingestion_config_validate.py
Traceback (most recent call last):
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_validate.py", line 12, in <module>
    from ingestion_config_models_extended import ExtendedValidationContainer
  File "/Users/trentsmith/workspace/cryoet/cryoet-data-portal-backend/schema/ingestion_config/v1.0.0/ingestion_config_models_extended.py", line 13, in <module>
    from async_lru import alru_cache
ModuleNotFoundError: No module named 'async_lru'

Definition of Done

  • only run this pre-commit on ingest config files that have changes
  • in a GHA run the same validation against all ingest configs
  • run this hook in an isolated environment with the needed dependencies installed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions