Fix circular imports and config - #13
Merged
Merged
Conversation
- Updated data_loader.py to utilize configurable data directories for local storage. - Modified experiment_tracking.py to implement config-based paths for model and metadata storage. - Introduced constants for target columns and batch size in model_evaluation.py for better maintainability.
- Removed PROJECT_ROOT constant and updated asset resolution to use config-based paths. - Consolidated logging setup into a new logger module, replacing the base logger. - Introduced AWS utility functions for S3 operations to streamline file uploads and downloads. - Deleted the old base_logger module to simplify the logging structure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
core.loggerand config-driven paths.aws_utilsfor S3 ops.constantsmodule for training utilities.Details
Refactor logging and configuration management
core/base_logger.pywith centralizedcore/logger.py(renamed fromlogging_config.py).PROJECT_ROOT;core/asset_resolution.pynow resolves via config.core/aws_utils.py; updatedcore/aws.py.src/core/__init__.py,src/core/asset_resolution.py,src/core/aws.py,src/core/aws_utils.py,src/core/logger.py(fromlogging_config.py), removedsrc/core/base_logger.py.Refactor data loading and experiment tracking to use config-based paths
sklearn_training/utils/data_loader.pynow uses configurable local data dirs.sklearn_training/utils/experiment_tracking.pywrites models/metadata to config-defined locations.sklearn_training/utils/model_evaluation.pyintroduces constants for targets/batch size.sklearn_training/train_model.pyandfastapi_backend/main.py; adjustedconfig.yaml.Add new constants module
src/sklearn_training/utils/constants.pyfor shared training constants.Notes
PROJECT_ROOTandbase_logger; ensure imports usecore.loggerand path resolution via config.pytest: 33 passed.