[Experimental] Use jupyter-book 2.0 for building the lectures

.github/workflows/build-ipynb.yml

@@ -0,0 +1,85 @@
+name: Build ipynb [using QuantEcon/mystmd]
+on:
+  push:
+    branches: [jb2]
+  pull_request:
+  workflow_dispatch:
+jobs:
+  build-ipynb:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+
+      - name: Install Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+          cache: 'pip'
+          cache-dependency-path: 'myst_requirements.txt'
+
+      - name: Install execution requirements
+        run: python -m pip install -r myst_requirements.txt
+
+      - name: Clone QuantEcon/mystmd (myst-to-ipynb branch)
+        run: |
+          git clone --depth 1 --branch myst-to-ipynb https://github.com/QuantEcon/mystmd.git /tmp/mystmd
+
+      - name: Build mystmd from source
+        working-directory: /tmp/mystmd
+        run: |
+          npm install
+          npx turbo run build
+
+      - name: Build HTML with custom mystmd
+        working-directory: ./lectures
+        run: /tmp/mystmd/packages/mystmd/dist/myst.cjs build --html
+
+      - name: Build ipynb exports with custom mystmd
+        working-directory: ./lectures
+        run: /tmp/mystmd/packages/mystmd/dist/myst.cjs build --ipynb
+
+      - name: Audit notebooks for MyST syntax leaks
+        working-directory: ./lectures
+        run: |
+          python3 -c "
+          import json, glob, re, sys
+          PATTERNS = [
+              (r'^\s*:::', 'directive fence'),
+              (r'\{math\}\`', 'math role'),
+              (r'^\+\+\+', 'block marker'),
+              (r'\{doc\}\`', 'doc role'),
+              (r'\{index\}', 'index directive'),
+              (r'\`\`\`\{', 'fenced directive'),
+              (r'^\([a-z_][a-z0-9_-]*\)=\$', 'target label'),
+          ]
+          issues = 0
+          for nb_path in sorted(glob.glob('exports/*.ipynb')):
+              with open(nb_path) as f:
+                  nb = json.load(f)
+              for i, cell in enumerate(nb.get('cells', []), 1):
+                  if cell.get('cell_type') != 'markdown':
+                      continue
+                  source = ''.join(cell.get('source', []))
+                  for line in source.split('\n'):
+                      for pat, desc in PATTERNS:
+                          if re.search(pat, line.strip()):
+                              print(f'{nb_path} cell {i}: [{desc}] {line.strip()[:80]}')
+                              issues += 1
+          if issues:
+              print(f'\n{issues} MyST syntax leaks found!')
+              sys.exit(1)
+          else:
+              print(f'All {len(glob.glob(\"exports/*.ipynb\"))} notebooks clean.')
+          "
+
+      - name: Upload ipynb exports
+        uses: actions/upload-artifact@v4
+        with:
+          name: ipynb-exports
+          path: './lectures/exports/'
+
+

.github/workflows/ci.yml

@@ -5,80 +5,49 @@ jobs:
     runs-on: "runs-on=${{ github.run_id }}/family=g4dn.2xlarge/image=quantecon_ubuntu2404/disk=large"
     steps:
       - uses: actions/checkout@v6
+
+      - name: Setup GitHub Pages
+        uses: actions/configure-pages@v3
+
+      - name: Cache Notebook Execution
+        uses: actions/cache@v3
+        id: cache-execution
         with:
-          fetch-depth: 0  # Fetch full git history for changelog feature
-      - name: Setup Anaconda
-        uses: conda-incubator/setup-miniconda@v3
+          path: ./lectures/_build/execute
+          key: ${{ runner.os }}-execute-cache-${{ hashFiles('lectures/**/*.md') }}
+
+      - uses: actions/setup-node@v4
         with:
-          auto-update-conda: true
-          auto-activate-base: true
-          miniconda-version: 'latest'
-          python-version: "3.13"
-          environment-file: environment.yml
-          activate-environment: quantecon
-      - name: Check nvidia Drivers
-        shell: bash -l {0}
-        run: nvidia-smi
-      - name: Install JAX and Numpyro
-        shell: bash -l {0}
-        run: |
-          pip install -U "jax[cuda13]"
-          pip install numpyro
-          python scripts/test-jax-install.py
-      - name: Install latex dependencies
-        run: |
-          sudo apt-get -qq update
-          sudo apt-get install -y     \
-            texlive-latex-recommended \
-            texlive-latex-extra       \
-            texlive-fonts-recommended \
-            texlive-fonts-extra       \
-            texlive-xetex             \
-            latexmk                   \
-            xindy                     \
-            dvipng                    \
-            cm-super
-      - name: Display Conda Environment Versions
-        shell: bash -l {0}
-        run: conda list
-      - name: Display Pip Versions
-        shell: bash -l {0}
-        run: pip list
-      - name: Download "build" folder (cache)
-        uses: dawidd6/action-download-artifact@v12
+          node-version: 18.x
+
+      - name: Install Python
+        if: steps.cache-execution.outputs.cache-hit != 'true'
+        uses: actions/setup-python@v5
         with:
-          workflow: cache.yml
-          branch: main
-          name: build-cache
-          path: _build
-      # Build Assets (Download Notebooks and PDF via LaTeX)
-      - name: Build Download Notebooks (sphinx-tojupyter)
-        shell: bash -l {0}
-        run: |
-          jb build lectures --path-output ./ --builder=custom --custom-builder=jupyter -n -W --keep-going
-          mkdir -p _build/html/_notebooks
-          cp -u _build/jupyter/*.ipynb _build/html/_notebooks
-      - name: Build PDF from LaTeX
-        shell: bash -l {0}
-        run: |
-          jb build lectures --builder pdflatex --path-output ./ -n -W --keep-going
-          mkdir _build/html/_pdf
-          cp -u _build/latex/*.pdf _build/html/_pdf
-      # Final Build of HTML
+          python-version: '3.12'
+          cache: 'pip'
+          cache-dependency-path: 'myst_requirements.txt'
+
+      - name: Install execution requirements
+        if: steps.cache-execution.outputs.cache-hit != 'true'
+        run: python -m pip install -r myst_requirements.txt
+
+      - name: Install jupyter book 2.0
+        run: pip install "jupyter-book>=2.0.0a0"
+
       - name: Build HTML
-        shell: bash -l {0}
-        run: |
-          jb build lectures --path-output ./ -n -W --keep-going
-      - name: Upload Execution Reports
+        working-directory: ./lectures
+        run: jupyter book build --html --execute
+
+      - name: Upload build output
         uses: actions/upload-artifact@v6
-        if: failure()
         with:
-          name: execution-reports
-          path: _build/html/reports
+          path: './lectures/_build/html'
+
       - name: Preview Deploy to Netlify
         uses: nwtgck/actions-netlify@v3
         with:
-          publish-dir: '_build/html/'
+          publish-dir: './lectures/_build/html'
           production-branch: main
           github-token: ${{ secrets.GITHUB_TOKEN }}
           deploy-message: "Preview Deploy from GitHub Actions"

.gitignore

@@ -11,6 +11,7 @@ lectures/sine_wave.py
 lectures/mathfoo.py
 lectures/mod.py
 lectures/test.py
+lectures/exports/
 lectures/foo.py
 lectures/*.txt
-lectures/*.csv
+lectures/*.csv

lectures/about_py.md

@@ -7,19 +7,18 @@ kernelspec:
   display_name: Python 3
   language: python
   name: python3
+exports:
+  - format: ipynb
+    markdown: commonmark
+    images: attachment
+    output: exports/about_py.ipynb
+downloads:
+  - file: ./about_py.md
+    title: Markdown (md)
+  - file: exports/about_py.ipynb
+    title: IPython (.ipynb)
 ---
 
-(about_py)=
-```{raw} jupyter
-<div id="qe-notebook-header" align="right" style="text-align:right;">
-        <a href="https://quantecon.org/" title="quantecon.org">
-                <img style="width:250px;display:inline;" width="250px" src="https://assets.quantecon.org/img/qe-menubar-logo.svg" alt="QuantEcon">
-        </a>
-</div>
-```
-
-```{index} single: python
-```
 
 # About These Lectures
 
@@ -100,8 +99,7 @@ This is important because it
 
 ### Common Uses
 
-{index}`Python <single: Python; common uses>` is a general-purpose language used
-in almost all application domains, including
+Python is a general-purpose language used in almost all application domains, including
 
 * AI and computer science
 * other scientific computing
@@ -158,8 +156,6 @@ Python is flexible and pragmatic, supporting multiple programming styles (proced
 
 ### Syntax and Design
 
-```{index} single: Python; syntax and design
-```
 
 One reason for Python's popularity is its simple and elegant design.
 
@@ -268,8 +264,6 @@ These lectures will explain how.
 
 ## Scientific Programming with Python
 
-```{index} single: scientific programming
-```
 
 We have already discussed the importance of Python for AI, machine learning and data science
 
@@ -290,9 +284,6 @@ This section briefly showcases some examples of Python for general scientific pr
 
 ### NumPy
 
-```{index} single: scientific programming; numeric
-```
-
 One of the most important parts of scientific computing is working with data.
 
 Data is often stored in matrices, vectors and arrays.
@@ -395,8 +386,6 @@ Later we'll discuss SciPy in more detail.
 
 ### Graphics
 
-```{index} single: Matplotlib
-```
 
 A major strength of Python is data visualization.
 
@@ -409,13 +398,13 @@ The most popular and comprehensive Python library for creating figures and graph
 Example 2D plot with embedded LaTeX annotations
 
 ```{figure} /_static/lecture_specific/about_py/qs.png
-:scale: 75
+:width: 75%
 ```
 
 Example contour plot
 
 ```{figure} /_static/lecture_specific/about_py/bn_density1.png
-:scale: 70
+:width: 70%
 ```
 
 Example 3D plot
@@ -449,8 +438,6 @@ For example, we are interesting in studying
 
 Python has many libraries for studying networks and graphs.
 
-```{index} single: NetworkX
-```
 
 One well-known example is [NetworkX](https://networkx.org/).