Skip to content

Commit d03db7e

Browse files
WillLP-codeWillLP-code
authored
Merge pull request #4 from kws/feature/devcontainer
Feature/devcontainer
2 parents ed8c73b + 61a761e commit d03db7e

15 files changed

Lines changed: 3216 additions & 67 deletions

File tree

‎.devcontainer/Dockerfile‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
FROM python:3.8-bullseye
2+
3+
# System deps for building old numpy/pandas from sdist if needed
4+
RUN apt-get update && apt-get install -y --no-install-recommends \
5+
build-essential \
6+
gfortran \
7+
pkg-config \
8+
git \
9+
git-lfs \
10+
curl \
11+
ca-certificates \
12+
&& rm -rf /var/lib/apt/lists/*
13+
14+
# Keep pip/build tooling conservative (old numpy/pandas often break with very new setuptools/Cython)
15+
RUN python -m pip install --upgrade "pip<24" \
16+
&& python -m pip install \
17+
"setuptools<60" \
18+
"wheel<0.39" \
19+
"Cython<3" \
20+
"virtualenv<21.0.0"
21+
22+
# Install Poetry (pin it if you want full reproducibility)
23+
ENV POETRY_VERSION=1.8.5
24+
RUN curl -sSL https://install.python-poetry.org | python - --version ${POETRY_VERSION} \
25+
&& ln -s /root/.local/bin/poetry /usr/local/bin/poetry
26+
27+
# Poetry config: keep the venv inside the repo for transparency
28+
RUN poetry config virtualenvs.in-project true
29+
30+
WORKDIR /work
31+

‎.devcontainer/devcontainer.json‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{
2+
"name": "legacy-pyodide-parity",
3+
"build": {
4+
"dockerfile": "Dockerfile"
5+
},
6+
"workspaceFolder": "/work",
7+
"mounts": [
8+
"source=${localWorkspaceFolder},target=/work,type=bind,consistency=cached"
9+
],
10+
"customizations": {
11+
"vscode": {
12+
"settings": {
13+
"python.defaultInterpreterPath": "/usr/local/bin/python",
14+
"python.analysis.typeCheckingMode": "basic"
15+
},
16+
"extensions": [
17+
"ms-python.python",
18+
"ms-python.vscode-pylance"
19+
]
20+
}
21+
},
22+
"postCreateCommand": "poetry --version && poetry install",
23+
"remoteUser": "root"
24+
}
25+

‎README.md‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,190 @@ Release naming should follow a pseudo-[semantic versioning][semver] format:
5050

5151
For example, the August 2021 release is named [2021.8][2021.8] with the associated tag [v2021.8][tag-v2021.8].
5252

53+
## Development Environment
54+
55+
This project includes a VS Code devcontainer configuration for a consistent development environment.
56+
57+
### Devcontainer Setup
58+
59+
The `.devcontainer/` directory contains:
60+
- **Dockerfile**: Based on Python 3.8 with conservative build tools (for compatibility with old numpy/pandas versions)
61+
- **devcontainer.json**: VS Code configuration for the container
62+
63+
The devcontainer:
64+
- Uses Python 3.8 (Debian Bullseye)
65+
- Installs Poetry 1.8.5
66+
- Configures Poetry to keep the virtual environment in the project directory (`virtualenvs.in-project true`)
67+
- Automatically runs `poetry install` when the container is created
68+
- Includes VS Code Python extensions (Python and Pylance)
69+
70+
To use the devcontainer:
71+
1. Open the project in VS Code
72+
2. When prompted, click "Reopen in Container" (or use Command Palette: "Dev Containers: Reopen in Container")
73+
3. The container will build and install dependencies automatically
74+
75+
The devcontainer uses conservative versions of build tools (pip<24, setuptools<60, wheel<0.39, Cython<3) to ensure compatibility with older numpy/pandas versions required for Pyodide compatibility.
76+
77+
## Testing in Pyodide/WASM
78+
79+
This project includes examples for testing the library in Pyodide (WebAssembly) environments. Pyodide allows Python code to run in browsers and Node.js using WebAssembly.
80+
81+
### Why Test in Pyodide?
82+
83+
Testing in Pyodide ensures:
84+
1. **WASM Compatibility**: Your code works in WebAssembly environments
85+
2. **Browser Compatibility**: If you plan to run Python in browsers
86+
3. **Cross-Platform**: Verify your code works in JavaScript runtimes
87+
4. **Package Compatibility**: Ensure dependencies work in Pyodide
88+
89+
### Browser Example
90+
91+
The browser example (`examples/pyodide-v0.22.1-browser/`) demonstrates testing the library in a web browser using Pyodide.
92+
93+
**Important**: The browser example requires the wheel to be built first before testing.
94+
95+
#### Setup
96+
97+
1. Build the wheel:
98+
```bash
99+
poetry build
100+
```
101+
This creates the wheel file in the `dist/` directory.
102+
103+
2. Start the Flask server:
104+
```bash
105+
poetry run serve
106+
```
107+
Or run directly:
108+
```bash
109+
python examples/pyodide-v0.22.1-browser/app.py
110+
```
111+
112+
The server will:
113+
- Check if the wheel exists in `dist/` before starting (provides helpful error if missing)
114+
- Start on `http://localhost:8123` (or the port specified by `PORT` environment variable)
115+
- Serve an HTML page that loads Pyodide from CDN
116+
- Automatically install the built wheel in the browser environment
117+
118+
#### Usage
119+
120+
1. Open your browser and navigate to `http://localhost:8123` (or the port shown in the server output)
121+
2. Wait for Pyodide to load (may take a moment on first load)
122+
3. Enter Python code in the left panel
123+
4. Click "Run Code" or press Ctrl+Enter
124+
5. View output in the right panel
125+
126+
The page includes example scripts:
127+
- **Basic**: Check platform and Python version
128+
- **Columns**: Inspect the dataframe structure
129+
- **Load**: Load postcodes for a specific letter
130+
- **Query**: Query and display loaded postcode data
131+
132+
**Note**: The browser example uses insecure mode for testing (no signature verification). The `QLACREF_PC_INSECURE` environment variable is set to `'True'` automatically.
133+
134+
### Node.js Example
135+
136+
The Node.js example (`examples/pyodide-v0.22.1-node/`) uses Vitest to test the library in Pyodide/WASM within a Node.js environment.
137+
138+
#### Setup
139+
140+
1. Navigate to the example directory:
141+
```bash
142+
cd examples/pyodide-v0.22.1-node
143+
```
144+
145+
2. Install dependencies:
146+
```bash
147+
pnpm install
148+
```
149+
150+
#### Running Tests
151+
152+
```bash
153+
# Run all tests
154+
pnpm test
155+
156+
# Run specific test file
157+
pnpm test test.test.js
158+
159+
# Run tests in watch mode
160+
pnpm test --watch
161+
```
162+
163+
#### How It Works
164+
165+
The Node.js example:
166+
- Uses Vitest as the test runner
167+
- Creates a Pyodide instance configured for Node.js
168+
- Copies the `qlacref_postcodes` module to Pyodide's virtual filesystem
169+
- Loads required packages (pandas, rsa) via Pyodide's package system
170+
- Verifies WASM execution by checking `sys.platform == "emscripten"`
171+
172+
The test helper (`pyodide-test-helper.js`) provides utilities:
173+
- `createPyodideInstance()`: Creates a configured Pyodide instance
174+
- `runPythonCode()`: Executes Python code
175+
- `loadPythonFile()`: Loads Python files from disk
176+
- `setupQlacrefPostcodes()`: Sets up the module in Pyodide with proper configuration
177+
178+
#### Testing Your Code
179+
180+
The example includes tests that:
181+
- Verify WASM execution environment
182+
- Test module import and instantiation
183+
- Test dataframe access and column structure
184+
- Test loading postcodes for specific letters
185+
- Test querying loaded postcode data
186+
187+
You can modify the tests in `test-qlacref-postcodes.test.js` to test your specific use cases.
188+
189+
### Key Differences
190+
191+
- **Browser Example**: Interactive testing in a web browser, requires built wheel, uses Flask server
192+
- **Node.js Example**: Automated testing with Vitest, can test source code directly, runs in Node.js environment
193+
194+
Both examples verify that code is running in WASM by checking `sys.platform == "emscripten"`.
195+
196+
### Troubleshooting
197+
198+
#### Browser Example
199+
200+
**Pyodide fails to load**:
201+
- Check your internet connection (Pyodide loads from CDN)
202+
- Check browser console for errors
203+
- Try refreshing the page
204+
205+
**Wheel installation fails**:
206+
- Ensure the wheel file exists in `dist/`
207+
- Check that the Flask server can access the `dist/` directory
208+
- Verify the wheel path in `index.html` matches your actual wheel filename
209+
210+
**Module import errors**:
211+
- Make sure all dependencies (pandas, rsa) are loading correctly
212+
- Check the browser console for detailed error messages
213+
214+
#### Node.js Example
215+
216+
**Module Not Found Errors**:
217+
- Check that `indexURL` is set correctly in the test helper
218+
- Verify the Pyodide package path is correct
219+
- Ensure WASM files are accessible
220+
221+
**Package Loading Issues**:
222+
- Check if the package is available in Pyodide
223+
- Verify package version compatibility
224+
- Check network connectivity (first load downloads from CDN)
225+
226+
**Path Issues**:
227+
- Ensure `indexURL` is set to a directory path (not a URL)
228+
- Use absolute paths when possible
229+
- Check Vitest configuration
230+
231+
### Resources
232+
233+
- [Pyodide Documentation](https://pyodide.org/)
234+
- [Pyodide Packages](https://pyodide.org/en/stable/usage/packages-in-pyodide.html)
235+
- [Vitest Documentation](https://vitest.dev/)
236+
53237
[pypi]: https://pypi.org/project/quality-lac-data-ref-postcodes/
54238
[semver]: https://semver.org/
55239
[2021.8]: https://github.com/SocialFinanceDigitalLabs/quality-lac-data-ref-postcodes/releases/tag/v2021.8

‎examples/pyodide-v0.22.1-browser/app.py‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Flask server for testing Pyodide in the browser.
4+
Serves an HTML page that loads Pyodide and the built wheel.
5+
"""
6+
7+
from flask import Flask, send_from_directory, send_file
8+
from pathlib import Path
9+
import os
10+
import sys
11+
12+
app = Flask(__name__)
13+
14+
# Get the project root directory (three levels up from this file)
15+
PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
16+
DIST_DIR = PROJECT_ROOT / "dist"
17+
18+
# Expected wheel filename based on pyproject.toml
19+
WHEEL_FILENAME = "quality_lac_data_ref_postcodes-2021.8.1-py3-none-any.whl"
20+
WHEEL_PATH = DIST_DIR / WHEEL_FILENAME
21+
22+
@app.route('/')
23+
def index():
24+
"""Serve the main HTML page."""
25+
return send_file(Path(__file__).parent / "index.html")
26+
27+
@app.route('/dist/<path:filename>')
28+
def serve_dist(filename):
29+
"""Serve files from the dist directory."""
30+
return send_from_directory(DIST_DIR, filename)
31+
32+
@app.route('/health')
33+
def health():
34+
"""Health check endpoint."""
35+
return {"status": "ok"}
36+
37+
def check_wheel_exists():
38+
"""Check if the wheel file exists and provide helpful error message if not."""
39+
if not WHEEL_PATH.exists():
40+
print("=" * 70, file=sys.stderr)
41+
print("ERROR: Wheel file not found!", file=sys.stderr)
42+
print("=" * 70, file=sys.stderr)
43+
print(f"\nExpected wheel file: {WHEEL_PATH}", file=sys.stderr)
44+
print(f"Dist directory: {DIST_DIR}", file=sys.stderr)
45+
print("\nTo build the wheel, run:", file=sys.stderr)
46+
print(" poetry build", file=sys.stderr)
47+
print("\nOr if using pip:", file=sys.stderr)
48+
print(" python -m build", file=sys.stderr)
49+
print("=" * 70, file=sys.stderr)
50+
sys.exit(1)
51+
52+
def main():
53+
"""Main entry point for the Flask server."""
54+
check_wheel_exists()
55+
port = os.getenv("PORT", 8123)
56+
print(f"✓ Wheel found: {WHEEL_PATH}")
57+
print(f"Starting Flask server on http://0.0.0.0:{port}")
58+
print(f"Open http://localhost:5000 in your browser")
59+
# Run on all interfaces so it's accessible
60+
app.run(host='0.0.0.0', port=port, debug=True)
61+
62+
if __name__ == '__main__':
63+
main()
64+

0 commit comments

Comments
 (0)