bids2table 2.0 (#48)

* Remove all old files

* Initial commit (v2)

* Add entities utilities

- BIDS schema loaded from `bidschematools` following previous impl (thx
  @nx10).
- Add Arrow schema construction
- Otherwise, trying to make much simpler. No dataclass complexity.

* Add back bids-examples submodule

* Add initial impl of indexing

New indexing uses just `Path` operations and string processing.
Generates arrow tables directly with no other dependencies. ~400 LOC.
Supports indexing cloud-hosted datasets via `cloudpathlib`.

Thanks to @nx10 for these suggestions.

* Update README.md

* Move cloudpathlib stuff to another module

* Add dataset and subject ID to progress bar

* Minor changes

- Don't raise error, just warn and return empty table if a dataset is empty.
- Add package-level exports.

* Add `batch_index_dataset` and other misc cleanup

* Add initial CLI

TODO: test

* Accept list of globs in CLI and other misc cleanup

Accepting unexpanded glob patterns in CLI is useful for cloud sources.
May as well accept a list of them. Few other misc cleanup (simplify log
messages, progress bar).

* Logging improvements

- Format large file counts in human readable units.
- Change verbosity order: progress bar -> warnings.
- Filter repeated logging messages.
- Initialize logger state in each process when using
  `ProcessPoolExecutor`.

* Add CI workflows

* Remove `BIDSEntity` enum; add `get_column_names`

A dynamically defined global enum for the column names is a bit awkward
and not that useful. The column names are not available statically in
the editor for example. Remove and replace with a `get_column_names`
function which dynamically defines the column name enum.

* Add docstrings and update README.md

* Clone with submodules in CI

* Add codecov.yaml

* Update codecov.yaml

Make informational, don't fail CI jobs

* Minor change to logging behavior

Disable progress bar by default in python index functions. Enable
progress bar by default in CLI. Add a `--no-progress` flag.

* Minor cleanup

* Add tests to run CLI

* Install with s3 extra in CI

* Remove unnecessary `path.is_dir()` call

* Update README.md before release

Author: clane9

.github/workflows/ci.yaml

@@ -6,29 +6,33 @@ on:
   pull_request:
     branches: [ "main" ]
 
+env:
+  UV_FROZEN: true
+
 jobs:
   test:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
         with:
           submodules: 'true'
-      - name: Set up Python 3.8
-        uses: actions/setup-python@v3
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
         with:
-          python-version: "3.8"
-      - name: Install
-        run: |
-          python -m pip install --upgrade pip
-          pip install .[dev,test]
+          pyproject-file: pyproject.toml
+      - name: Install the project
+        run: uv sync --all-extras
       - name: Check quality
         run: |
-          black --check bids2table tests
-          isort --check-only bids2table tests
-          flake8 bids2table tests
-          pylint --fail-under 9.0 bids2table
-          mypy bids2table 
+          uv run ruff check bids2table tests
+          uv run ruff format --check bids2table tests
       - name: Run tests
-        run: pytest --cov=bids2table tests
+        run: |
+          uv run pytest \
+            --junitxml=pytest.xml \
+            --cov-report=xml:coverage.xml \
+            --cov=bids2table tests
       - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}

.github/workflows/docs.yaml

@@ -0,0 +1,31 @@
+name: release
+
+on:
+  release:
+    types:
+      - published
+
+env:
+  UV_FROZEN: true
+
+jobs:
+  pypi-publish:
+    name: upload release to PyPI
+    runs-on: ubuntu-latest
+    # Specifying a GitHub environment is optional, but strongly encouraged
+    environment: pypi
+    permissions:
+      # IMPORTANT: this permission is mandatory for Trusted Publishing
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          pyproject-file: pyproject.toml
+      - name: Build project
+        run: |
+          uv sync --no-dev --no-install-project
+          uv build
+      - name: Publish package distributions to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

.github/workflows/release.yaml

@@ -25,20 +25,10 @@ htmlcov
 
 # Local data and scratch
 .scratch
-example/bids-examples.b2t
-docs
 
-# Local environment
+# Local virtual environment
 .venv
+.python-version
 
 # Jupyter
 .ipynb_checkpoints
-
-# Example generated tables
-example/tables
-
-# Benchmark data
-benchmark/**/envs
-benchmark/**/results
-benchmark/**/indexes
-slurm-*

.gitignore

@@ -5,33 +5,26 @@ default_language_version:
 
 repos:
 -   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.4.0
+    rev: v5.0.0
     hooks:
     -   id: trailing-whitespace
     -   id: check-ast
     -   id: check-merge-conflict
-    # -   id: no-commit-to-branch
-    #     args: ['--branch=main']
     -   id: end-of-file-fixer
 
--   repo: https://github.com/psf/black
-    rev: 23.3.0 # Replace by any tag/version: https://github.com/psf/black/tags
-    hooks:
-      - id: black
-        language_version: python3 # Should be a command that runs python3.6+
-
--   repo: https://github.com/pycqa/flake8
-    rev: 7.1.1
-    hooks:
-    - id: flake8
-
-- repo: https://github.com/pycqa/isort
-  rev: 5.11.5
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  # Ruff version.
+  rev: v0.9.10
   hooks:
-    - id: isort
-      name: isort (python)
+    # Run the linter.
+    - id: ruff
+      args: [ --fix ]
+    # Run the formatter.
+    - id: ruff-format
 
--   repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.2.0
-    hooks:
-    - id: mypy
+- repo: https://github.com/codespell-project/codespell
+  rev: v2.4.1
+  hooks:
+  - id: codespell
+    additional_dependencies:
+      - tomli