Add CI/CD pipeline, fix all notebooks (#71)

* update deps

* add backoff when downloading assets

* add Makefile

* add link search test file

* add urlextract dependency

* run .py and .ipynb tests in make

* add github workflow

* remove an old notebook

* don't run .ipynb tests on exercise notebooks

* skip existing files when downloading

* use max workers in makefile

* mirdotcom.py changes

* don't run tests on `adtlib.ipynb`

* rename `kmeans_instrument_classification` to make clear that it is an exercise

* pin more recent `mir_eval` version, add `soundfile` dep

* fix all notebooks

* run tests weekly

* retry tests

* Migrate to `JupyterBook` (#73)

* Implement retry mechanism in download functions

* Hardcode assets file paths

* initial commit

* add requirements.in

* add postBuild

* make package installable

* update all `mirdotcom.init()` calls

* add binder badge

* try fix install

* update all binder links

* remove badges, update colab references

* fix line break

* add jupyter-book requirement

* try refactor

* update all headings

* fix test paths

* update cross-references

* add initial jupyter-book build

* try fix

* remove musicinformationretrieval.com URLs and "Back to index"

* update config to fix thebe + binder

* update all notebooks

* actually set working directory

* replace references to "workshop" with "website"

* fix sheet music representations links

* fix midi conversion table

* remove function names as section titles

* axis labels

* remove hardcoded ../../assets, replace with `mirdotcom.get_audio`

* add beginner tutorial

* move "people who use this site" section

* add build target to makefile

* pin dependencies, minor fixes

* add note about .ipynb

* optimize imports

* remove "hide-cell"

* fix titles

* add more unittests for notebooks

* make notebooks pass tests

* add black formatting + deps: run `make fix`

* run black on files

Author: HuwCheston

.github/workflows/deploy_book.yml

@@ -0,0 +1,46 @@
+name: deploy-book
+
+on:
+  push:
+    branches:
+      - gh-pages
+
+env:
+  BASE_URL: /${{ github.event.repository.name }}
+
+concurrency:
+  group: "pages"
+  cancel-in-progress: false
+
+jobs:
+  deploy-book:
+    runs-on: ubuntu-latest
+    permissions:
+      pages: write
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.10'
+          cache: pip
+
+      - name: Install dependencies
+        run: |
+          make install
+
+      - name: Build the book
+        run: |
+          source venv/bin/activate
+          jupyter-book build mirdotcom
+
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: "mirdotcom/_build/html"
+
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

.github/workflows/test_notebooks.yml

@@ -0,0 +1,20 @@
+name: Test notebooks
+
+on:
+  push:
+  pull_request:
+  schedule:
+    - cron: '0 0 * * 0'
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.10'
+      - name: install & run nbmake
+        run: |
+         make install 
+         make test

.gitignore

@@ -58,4 +58,5 @@ coverage.xml
 
 # Sphinx documentation
 docs/_build/
-
+venv/
+mirdotcom/_build

Makefile

@@ -0,0 +1,28 @@
+.PHONY: run install build fix
+
+
+run: venv/bin/jupyter-lab
+	venv/bin/jupyter-lab README.md
+
+install: venv/bin/jupyter-lab
+	venv/bin/pip install -e .
+
+test: venv/bin/jupyter-lab
+	echo "Running .py tests..."
+	venv/bin/pytest -n=auto --ignore-glob='*.ipynb' --reruns 3 --reruns-delay 5 tests
+	echo "Running .ipynb tests..."
+	venv/bin/pytest --nbmake -n=auto --ignore-glob='*.py' --ignore-glob='*exercise*.ipynb' --ignore-glob='*adtlib*.ipynb' --reruns 3 --reruns-delay 5 mirdotcom
+
+venv/:
+	python3 -m venv venv
+	venv/bin/pip install --upgrade pip
+
+venv/bin/jupyter-lab: venv/
+	venv/bin/pip install -r requirements.txt
+	touch venv/bin/jupyter-lab
+
+build:
+	jupyter-book build mirdotcom
+
+fix:
+	black mirdotcom/content --target-version=py310