Merge branch 'main' into mirror

Author: asitha-sena

.github/workflows/test-package.yml

@@ -15,7 +15,7 @@ jobs:
     strategy:
       matrix:
         runs-on: [ubuntu-latest, windows-latest, macos-latest]
-        python-version: ['3.9', '3.10', '3.11', '3.12', '3.13']
+        python-version: ['3.9', '3.10', '3.11', '3.12', '3.13', ' 3.14']
 
     runs-on: ${{matrix.runs-on}}
 
@@ -28,7 +28,7 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install pytest toml
+        python -m pip install pytest toml nbval striplog
         python -m pip install .
     - name: Update system path
       run: |
@@ -39,4 +39,6 @@ jobs:
       run: |
         echo $GITHUB_PATH
         echo $PATH
-        python -m pytest -v
+        python -m pytest
+        python -m doctest README.md
+        python -m pytest -v --nbval notebooks/Plot_locations_and_create_strip_log.ipynb

.gitlab-ci.yml

@@ -7,7 +7,7 @@ stages:
   stage: test
   parallel:
     matrix:
-      - PYTHON_VERSION: ["3.9", "3.10", "3.11", "3.12", "3.13"]
+      - PYTHON_VERSION: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
   script:
     - python -m pip install .
     - python -m pip install pytest nbval

changelog.txt

@@ -1,6 +1,20 @@
 Changelog
 =========
 
+1.2.0 (2026-03-18)
+------------------
+- Add support for Standard dicionary v4.2
+- Add support for Python 3.14
+- Update pyproject.toml to PEP621 format
+- Replace bespoke CSV parsing code with CSV module. This allows escaped double
+  quotes to be handle correctly (affects Rules 4 & 5). (Issue #135)
+- Raise exception if GROUP lines are missing (Issues #133 & #137)
+- Fix SyntaxWarning in Python 3.14 about return statement inside 'finally' block
+  (Issue #139)
+- Add feature to choose subset of groups/tables to read from file (Issue #143)
+- Address various warnings related to future changes and deprecations in Python
+
+
 1.1.0 (2025-06-03)
 ------------------
 - Fix minor issues in v4.1 and v4.1.1 standard dictionaries and include latest
@@ -48,7 +62,6 @@ Changelog
 
 0.5.0 (2024-01-27)
 ------------------
-
 - Remove support for Python version 3.7
 - Add support for Pandas version 2.x and drop support for version 1.x.
 - Add functionality to log internal messages

docs/index.md

@@ -29,7 +29,7 @@ Group](https://gitlab.com/ags-data-format-wg)
 ## Project Scope
 For many years the industry has had two checker applications that don't always produce the same results.  This has caused clients to specify that AGS files must pass both checkers before the data is submitted to them, resulting in additional work for clients, contractors and software producers.
 
-The Project has one goal.  To produce an AGS Validator library that can be used by any developer to include AGS validation routine in their application.  By Producing a single code stream that can be incorporated into any application the industry will now know if a file is valid.
+The Project has one goal.  To produce an AGS Validator library that can be used by any developer to include AGS validation routine in their application.  By producing a single code stream that can be incorporated into any application the industry will now know if a file is valid.
 
 The project has also created an EXE file with a simple interface that allows users to validate files.  More information on this project can be found in the Related Projects section of this Wiki.  This library will not visualize or fix AGS data but we welcome developers to start their own related project.
 
@@ -38,28 +38,33 @@ The project has also created an EXE file with a simple interface that allows use
 May 4th 2021 - AGS Validator EXE Beta launched
 This release is for testing by the development team and the AGS committee members.  We believe that it finds all the errors in AGS 4.0.3, 4.0.4 and 4.1 files with the known issue list below. 
 
-Line numbers are not included in all error messages
+September 4th, 2024 - v1.0.0 
+This package has been very stable with no breaking changes to the API since its first release on PyPI. However, finalizing all the validation rules took a lot of testing and refining, so this is the 
+first version that we recommend for validating files in production.
+
+For more details on releases see the _changelog.txt_ file in the Git repo.
 
 ## Adding Test AGS Files
-The project has launched with 55 AGS files that have deliberate errors in them. Each file has a corresponding .check file with it that defines the errors that are in the file.
+The project has 80+ AGS files that have deliberate errors in them. The errors in each file are defined in its TRAN_REM field.
 
-Every AGS file in the /tests/test_files folder is checked before a release is made to ensure that any modifications don't break any checks or introduce new problems.
+Every test file is checked before a release is made to ensure that any modifications don't break any checks or introduce new problems.
 
 If you have an AGS file where the validator either incorrectly identifies an error or does not highlight a known error then we would like to know about it so we can fix any problems and include this file in the checking routine.
 
-To report problems please open an issue, describe the error and attach the file. The Contributor Team will then review the file, create the .check file and include it in the main Repository.
+To report problems please open an issue, describe the error and attach the file. The Contributor Team will then review and address the issue.
 
 ## Credits
 The AGS Python Library is created and maintained by a range of enthusiastic international volunteers (like you)
 
 Beta Release 0.0.1 4th May 2021
+Stable Release 1.0.0 4th September 2024
 
-The initial beta release was made possible due to the considerable work completed by Asitha Senanayake with support and testing by Roger Chandler and Tony Daly.
+This software was made possible due to the considerable work completed by Asitha Senanayake with support and testing by Roger Chandler, Tony Daly, Neil Chadwick.
 
-We welcome push requests from any AGS developer within the bounds of the project, so get coding and create a push request to join the contribution team and see your name add to this Credits Page for the next release.
+We welcome merge requests from any AGS developer within the bounds of the project, so get coding and create a merge request to join the contribution team and see your name add to this Credits Page for the next release.
 
 ## Development Environment
-This project is built in Ubuntu 20.04 and packaged using `poetry` (python-poetry.org). Unit testing is done using `pytest` with the `toml` library as a dependency.
+This project is built in Ubuntu 22.04 and packaged using `poetry` (python-poetry.org). Unit testing is done using `pytest` with the `toml` library as a dependency.
 
 Instructions on how to install `poetry` can be found at https://python-poetry.org/docs/.
 This package does not come with _setup.py_ and _requirements.txt_ files. Instead, all dependency information is provided in the _pyproject.toml_ and _poetry.lock_ files.
@@ -69,10 +74,6 @@ A development environment can be setup by running `poetry install` from within t
 If you want only the dependencies installed, then run `poetry install --no-root` instead.
 Unit tests can be run by running `python -m pytest` from within the root directory of the git repo.
 
-Dev directory is structured as shown below:
-
-![dev_directory_structure](uploads/0ec81e34b20fea36995dcc9342720181/dev_directory_structure.png)
-
 ## Get Involved
 If you are reading this then you are already involved! Thank you for being part of this amazing project. If you share our passion for this project and would like to be involved then we are looking for people to help in any of the roles below. Just introduce yourself to the team via
 
@@ -82,13 +83,12 @@ Managers (Primary role is to merge pull requests from contributors)
 Controllers (Primary role is governance and PR)
 
 ## Related Projects
-The AGS library is an excellent tool for any developers to add AGS validation capabilities to their application but is not user friendly for users.<br><br>
-To help users validate AGS files using this library the following tools should be reviewed.  Please see the links below for the status of each of these projects.<br><br> 
-- [AGS Validator EXE](https://gitlab.com/ags-data-format-wg/ags-checker-desktop-app) (Released by AGS)
-- [AGS Excel Add-In](https://gitlab.com/RogerChandler/ags-validator-excel-add-in) (Developed by Roger Chandler)
-- [AGS Validator Web Interface and API](https://github.com/BritishGeologicalSurvey/AGS-Validator-FastAPI-Web-App ) (Developed by BGS)
-
-
-Use the navigation menu on the left and the search to find what you want.
-
+This library is an excellent tool for developers to add AGS validation capabilities to their application and for end users who like to work with AGS files programmatically using Python.
+However, if you are an end user who wants to validate AGS files without having to write code, then you can check out the following tools.
+
+ - [AGS4 CLI](https://ags-data-format-wg.gitlab.io/ags-python-library/usage/#command-line-interface) (Developed by Asitha Senanayake)
+ - [AGS Validator EXE](https://gitlab.com/ags-data-format-wg/ags-checker-desktop-app) (Released by AGS)
+ - [AGS Excel Add-In](https://www.geotechnicaldata.com/ags-data-toolkit-for-excel) (Developed by Roger Chandler)
+ - [AGS Validator Web Interface and API](https://github.com/BritishGeologicalSurvey/AGS-Validator-FastAPI-Web-App ) (Developed by BGS)
+ - [Web application (Digital Geotechnical)](https://dg-ags-validator.ew.r.appspot.com) (Developed by Neil Chadwick)
 

docs/usage.md

@@ -83,15 +83,15 @@ Tables converted to numeric using the ```AGS4.convert_to_numeric()``` function s
 
 We have created an example Jupyter Notebook which imports an AGS file, plots boreholes on a map and creates a Striplog.
 
-[See here](./notebooks/Plot_locations_and_create_strip_log.ipynb)
+[See here](https://gitlab.com/ags-data-format-wg/ags-python-library/-/tree/main/notebooks)
 
 ### Command Line Interface
 
-A command-line interface (CLI) to access the core functionality of the library
-is available since version 0.2.0. It is automatically installed together with the
-library and can be accessed by typing ```ags4_cli``` in a terminal/shell. If you
-want the CLI to be available globally (i.e. not limited to a single virtual
-environment), then you can install it using ```pipx```.
+An easy-to-use and expressive command-line interface (CLI) to access the core
+functionality of the library is bundled with the library. It can be accessed by
+typing ```ags4_cli``` in a terminal/shell. If you want the CLI to be available
+globally (i.e. not limited to a single virtual environment), then you can
+install it using ```pipx``` or ```uv tool```.
 
 You can do the following operations via the CLI:
 1. Check/validate AGS4 files
@@ -123,7 +123,7 @@ tables, headings = load_test_data()
 gui = show(**tables)
 ```
 
-<img src="./media/pandasgui_screenshot.png" width=800>
+<img src="https://gitlab.com/ags-data-format-wg/ags-python-library/-/blob/main/docs/media/pandasgui_screenshot.png" width=800>
 
 Any edits made in the GUI can be saved and exported back to an AGS4 file as follows:
 
@@ -147,7 +147,9 @@ Senanayake et al., (2022). python-ags4: A Python library to read, write, and val
 
 This library has been used to create
 
+- [Windows/Linux/MacOS Command-Line Utility (included with the library)](https://ags-data-format-wg.gitlab.io/ags-python-library/usage/#command-line-interface)
 - Windows Desktop Application - https://gitlab.com/ags-data-format-wg/ags-checker-desktop-app 
 - Web application and API (pyagsapi) - https://github.com/BritishGeologicalSurvey/pyagsapi 
   - Deployed as https://agsapi.bgs.ac.uk/
 - Excel Add On - https://gitlab.com/RogerChandler/ags-validator-excel-add-in
+- Web application (Digital Geotechnical) - https://dg-ags-validator.ew.r.appspot.com/