Add YAML Configuration Documentation and Template - #431
Open
e-lo wants to merge 23 commits into
Open
Conversation
- Fix dataclass mutable default issue in WranglerConfig by using Field(default_factory=...) - Fix pandera/pandas dtype error in scopes.py by changing DataFrame type annotation to pd.DataFrame for _filter_exploded_df_to_scope function parameter - This prevents pydantic from trying to introspect pandera schemas that contain list types, which pandas doesn't support as a dtype Fixes import errors when loading the network_wrangler package.
…ort time Change DataFrame[RoadLinksTable] to pd.DataFrame in prop_for_scope function signature to prevent pydantic from trying to introspect the pandera schema at import time. The validation still happens inside the function via validate_df_to_model().
The helper functions _default_*_config() are unnecessary since the classes themselves are callable and can be used directly in Field(default_factory=...). This simplifies the code without changing functionality.
Change DataFrame[Model] type annotations to pd.DataFrame in RoadwayNetwork class fields to prevent pydantic from trying to introspect pandera schemas at class definition time. Validation is now done in field validators using validate_df_to_model(). This fixes the 'dtype list not understood' error when importing the package.
- Fix field_validator decorator order (classmethod before field_validator) - Fix bug in Feed.__init__ where dict iteration was missing .items() - These changes ensure compatibility with Python 3.10 through 3.12
- Add Python 3.11 and 3.12 to test matrix - Add explicit import test that would fail before the hotfix - This ensures the hotfix is validated across all supported Python versions
UP045 suggests using 'X | None' syntax which requires Python 3.10+. Since the project supports Python 3.9+, we should keep using Optional[X] for backwards compatibility.
Since the project supports Python 3.9+ (requires-python = '>=3.9'), we should test Python 3.9 in CI to ensure compatibility.
Test all current stable Python versions to ensure compatibility across the full range of supported versions.
Python 3.14 is still in development and fiona/GDAL dependencies don't have wheels available yet, causing build failures. We'll add it back once it's stable and dependencies are available.
PLC0415 flags imports inside functions, but many of these are intentional lazy imports to avoid circular dependencies or to only import heavy dependencies when needed. These are a common pattern in the codebase and should not be flagged.
Fix 'raises' to 'Raises' in docstring to match Google style convention.
- Fix D405: Capitalize 'returns' to 'Returns' in data.py docstring - Fix RUF022: Sort __all__ list alphabetically in __init__.py - Fix PLW1641: Add noqa comment for DBModelMixin (intentionally unhashable due to mutable dataframes)
- Add prominent YAML Configuration tip at top of wrangler.py docstring - Fix typo: EXISTING_VALUE_CONFLIC -> EXISTING_VALUE_CONFLICT - Update Extended usage section to discourage Python code modifications - Add comprehensive YAML configuration section to how_to.md with examples - Add configuration best practices warning - Update design.md to clarify params.py is not user-configurable - Create example wrangler.config.yml file with all available parameters These changes make YAML/TOML configuration the recommended approach, especially for users less familiar with Python, addressing GitHub issue #428.
- Create wrangler.config.yml.template with all parameters commented out - Include default values and guidance on when to change each parameter - Update how_to.md with step-by-step instructions using the template - Add template details tip explaining what's included in the template - Improve usability by showing users to start with template and customize
- Move Configuration Best Practices warning to top - Restructure Step 3 and Step 4 for better clarity - Add danger box for Python config editing with warning - Consolidate configuration usage examples - Add pymdownx.snippets extension for template file inclusion - Fix typo: 'beetween' -> 'between'
Contributor
|
Contributor
📊 Benchmark ComparisonComparing PR branch to base branch No previous benchmark found in base branch. This will serve as the baseline. |
Collaborator
Author
|
@i-am-sijia This responds to one of your requests. LMK if it satisfies your questions regarding configging! |
Contributor
📊 Benchmark ComparisonComparing PR branch to base branch No previous benchmark found in base branch. This will serve as the baseline. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds comprehensive YAML/TOML configuration documentation and a configuration template file to make Network Wrangler more accessible to users less familiar with Python, addressing GitHub issue #428.
Changes
Documentation Improvements
docs/how_to.md:New Files
examples/wrangler.config.yml.template: Comprehensive configuration template with:examples/stpaul/wrangler.config.yml: Working example configuration fileCode Updates
network_wrangler/configs/wrangler.pydocstring to reference templatedocs/design.mdto clarify thatparams.pyis not user-configurableMkDocs Configuration
pymdownx.snippetsextension to enable file inclusion in documentationcodeincludeplugindocsandexamplesdirectoriesBenefits
Related Issues
Closes #428
Testing