Test kaitak hk notebook and set up automated testing with GitHub actions #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

JoostGevaert merged 11 commits into dev from test-kaitak-hk-notebook-bed-26

Apr 29, 2025

.github/workflows/python-tests.yml

-Original file line number
+Diff line change
@@ -0,0 +1,33 @@
+    name: Run Python tests
+    on:
+      push:
+        branches: [ main, dev ]
+      pull_request:
+        branches: [ main, dev ]
+    jobs:
+      test:
+        runs-on: ${{ matrix.os }}
+        strategy:
+          matrix:
+            os: [ubuntu-latest, windows-latest]
+            python-version: ['3.9']
+        steps:
+        - name: Checkout code
+          uses: actions/checkout@v4
+        - name: Install uv
+          uses: astral-sh/setup-uv@v5
+        - name: Set up Python
+          uses: actions/setup-python@v5
+          with:
+            python-version: ${{ matrix.python-version }}
+        - name: Install bedrock-ge with testing dependencies
+          run: uv sync --locked --group tests
+        - name: Run tests
+          run: uv run pytest tests

.python-version

This file was deleted.

.vscode/settings.json

-Original file line number
+Diff line change
@@ -0,0 +1,7 @@
+    {
+      "python.testing.pytestArgs": [
+        "tests"
+      ],
+      "python.testing.unittestEnabled": false,
+      "python.testing.pytestEnabled": true
+    }

RELEASE.md

-Original file line number
+Diff line change
@@ Expand Up / @@ -12,6 +12,7 @@ Update the version number in: @@
     - [`pyproject.toml`](pyproject.toml)
     - [`/src/bedrock/__init__.py`](/src/bedrock/__init__.py)
+    - Inline script dependencies of marimo notebooks in [`examples`](/examples/)
     ## 2. Update the Changelog
@@ Expand Down @@

examples/hk_kaitak_ags3/hk_kaitak_ags3_to_brgi_geodb.py

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -1,7 +1,7 @@
  
    # /// script

    # requires-python = ">=3.12"

    # dependencies = [

    #     "bedrock-ge==0.2.2",

    #     "bedrock-ge==0.2.3",

    #     "chardet==5.2.0",

    #     "folium==0.19.5",

    #     "geopandas==1.0.1",

    @@ -167,7 +167,7 @@ def _(mo):
  
            ### From GI dataframe to `geopandas.GeoDataFrame` 

            In order to construct the 3D simple feature GIS geometry of the `Location`s, `Sample`s and `InSitu_TEST`s, a few more columns have to be calcualated for each of these tables: `elevation_at_top` and `elevation_at_base` if the in-situ test or sample was taken over a depth interval.

            In order to construct the 3D simple feature GIS geometry of the `Location`s, `Sample`s and `InSitu_TEST`s, a few more columns have to be calculated for each of these tables: `elevation_at_top` and `elevation_at_base` if the in-situ test or sample was taken over a depth interval.

            The 3D simple feature GIS geometry as [WKT](https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry) for point tests and samples:  

            `POINT (easting northing elevation_at_top)`

    @@ -341,7 +341,7 @@ def _(mo):
  
            <iframe title="Speckle" src="https://app.speckle.systems/projects/013aaf06e7/models/1cbe68ed69,44c8d1ecae,9535541c2b,a739490298,ff81bfa02b#embed=%7B%22isEnabled%22%3Atrue%7D" width="100%" height="400" frameborder="0"></iframe>

            Additionally, you can load the GI data in other software that Speckle has a connector for, such as Rhino / Grasshopper to enable parameteric geotechnical engineering workflows.

            Additionally, you can load the GI data in other software that Speckle has a connector for, such as Rhino / Grasshopper to enable parametric geotechnical engineering workflows.

            """

        )

        return

    @@ -410,7 +410,7 @@ def zip_of_ags3s_to_bedrock_gi_database(zip, crs):
  
                        )

                        check_no_gis_brgi_database(brgi_db_from_1_ags3_file)

                        print(

                            f"\n✅ Succesfully converted {file_name} to Bedrock GI database and validated!\n"

                            f"\n✅ Successfully converted {file_name} to Bedrock GI database and validated!\n"

                        )

                        print(

                            f"🧵 Concatenating Bedrock GI database for {file_name} to existing Bedrock GI database...\n"

examples/hk_kaitak_ags3/kaitak_gi.gpkg

Binary file not shown.

pyproject.toml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -69,12 +69,9 @@ Source = "https://github.com/bedrock-engineer/bedrock-ge"
  
    Documentation = "https://bedrock.engineer/docs/"

    Tracker = "https://github.com/bedrock-engineer/bedrock-ge/issues"

    [build-system]

    requires = ["hatchling"]

    build-backend = "hatchling.build"

    [tool.uv]

    dev-dependencies = [

    [project.optional-dependencies]

    dev = [

        "duckdb>=1.2.2",

        "folium>=0.17.0",

        "frictionless[excel]>=4.40.8",

    @@ -85,11 +82,18 @@ dev-dependencies = [
  
        "mypy>=1.11.2",

        "nbconvert>=7.16.6",

        "pandas-stubs>=2.2.2.240807",

        "pytest>=8.3.3",

        "ruff>=0.6.7",

        "sqlglot>=26.12.1",

    ]

    tests = [

        "pytest>=8.3.3",

    ]

    [build-system]

    requires = ["hatchling"]

    build-backend = "hatchling.build"

    [tool.ruff]

    line-length = 88

src/bedrock_ge/plot.py

-Original file line number
+Diff line change
@@ -1,2 +1,4 @@
     def hello_plt() -> None:
-        print("Hello from src/bedrock/plot.py!")
+        print(
+            "Hi Bedrock engineer! bedrock_ge.plot is a placeholder module for Geotechnical Engineering plots."
+        )

tests/__init__.py

Empty file.

tests/test_bedrock_ge/__init__.py

Empty file.

tests/test_bedrock_ge/gi/__init__.py

Empty file.

0 tests/fixtures/ags4_sample.ags → ...s/test_bedrock_ge/gi/data/ags4_sample.ags

File renamed without changes.

0 tests/fixtures/asg4_expected.json → ...est_bedrock_ge/gi/data/asg4_expected.json

File renamed without changes.

tests/test_gi.py → tests/test_bedrock_ge/gi/test_ags.py

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -1,22 +1,17 @@
  
    import json

    import sys

    import pandas as pd

    from bedrock_ge.gi.ags.read import ags4_to_dfs

    from pathlib import Path

    src_path = Path(__file__).parent.parent / "src"

    sys.path.insert(0, str(src_path))

    fixtures_dir = Path(__file__).parent / "fixtures"

    import pandas as pd

    from bedrock_ge.gi.ags.read import ags4_to_dfs

    def test_ags3_to_dfs():

        assert True is True

    data_dir = Path(__file__).parent / "data"

    def test_ags4_to_dfs():

        expected_path = fixtures_dir / "asg4_expected.json"

        sample_path = fixtures_dir / "ags4_sample.ags"

        expected_path = data_dir / "asg4_expected.json"

        sample_path = data_dir / "ags4_sample.ags"

        with open(sample_path, "r") as file:

            ags4_sample_data = file.read()

tests/test_bedrock_ge/test_plot.py

-Original file line number
+Diff line change
@@ -0,0 +1,20 @@
+    import io
+    import sys
+    from bedrock_ge.plot import hello_plt
+    def test_hello_plt():
+        # Redirect stdout to capture print output
+        captured_output = io.StringIO()
+        sys.stdout = captured_output
+        # Call the function
+        hello_plt()
+        # Reset stdout
+        sys.stdout = sys.__stdout__
+        # Check that the correct message was printed
+        expected_message = "Hi Bedrock engineer! bedrock_ge.plot is a placeholder module for Geotechnical Engineering plots.\n"
+        assert captured_output.getvalue() == expected_message

tests/test_examples/__init__.py

Empty file.

tests/test_examples/conftest.py

-Original file line number
+Diff line change
@@ -0,0 +1,8 @@
+    from pathlib import Path
+    import pytest
+    @pytest.fixture(scope="session")
+    def examples_dir():
+        return Path(__file__).parent.parent.parent / "examples"

tests/test_examples/test_hk_kaitak_ags3_to_brgi_geodb.py

-Original file line number
+Diff line change
@@ -0,0 +1,59 @@
+    import os
+    import shutil
+    import subprocess
+    import sys
+    from pathlib import Path
+    from tempfile import TemporaryDirectory
+    def test_kaitak_ags3_notebook_runs_and_creates_gpkg(examples_dir):
+        notebook_dir = examples_dir / "hk_kaitak_ags3"
+        notebook_path = notebook_dir / "hk_kaitak_ags3_to_brgi_geodb.py"
+        gpkg_output_path = notebook_dir / "kaitak_gi.gpkg"
+        assert gpkg_output_path.exists(), (
+            f"Expected {gpkg_output_path} to exist, but it doesn't."
+        )
+        # Copy the kaitak_gi.gpkg to a temporary directory for comparing
+        # to the one created when executing the notebook.
+        # And to put back to the original state at the end of the test.
+        with TemporaryDirectory() as temp_dir:
+            temp_gpkg_path = Path(temp_dir) / "temp_kaitak_gi.gpkg"
+            shutil.move(gpkg_output_path, temp_gpkg_path)
+            print(f"Running: `python {notebook_path}`\n")
+            # Run the notebook as a script
+            # TODO: implement logging
+            # NOTE: The env (environment variables) and encoding are required for running
+            # the notebook as a script from both Windows and Linux. Wihtout: UnicodeDecodeError
+            env = os.environ.copy()
+            env["PYTHONIOENCODING"] = "utf-8"
+            result = subprocess.run(
+                ["uvx", "uv", "run", "--no-project", "--no-cache", str(notebook_path)],
+                check=False,
+                capture_output=True,
+                text=True,
+                encoding="utf-8",
+                env=env,
+            )
+            # Check that the script ran successfully
+            assert result.returncode == 0, (
+                f"\n📛 Running `uvx run marimo notebook.py` failed with code {result.returncode}\n"
+                f"📄 STDOUT:\n{result.stdout}\n"
+                f"⚠️ STDERR:\n{result.stderr}"
+            )
+            # Check that the file was created
+            assert gpkg_output_path.exists(), (
+                f"The expected GeoPackage {gpkg_output_path} was not created."
+            )
+            # TODO: write some logic to compare the original and new GeoPackages.
+            # Remove the newly generated kaitak_gi.gpkg
+            os.remove(gpkg_output_path)
+            # Place back the original kaitak_gi.gpkg from the temporary directory
+            # to its original location.
+            shutil.move(temp_gpkg_path, gpkg_output_path)

tests/test_plot.py

This file was deleted.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test kaitak hk notebook and set up automated testing with GitHub actions #30

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!

Uh oh!