Skip to content

Commit 3f2bd78

Browse files
JacobCoffeeJacobCoffee
committed
docs: add wiki development and contributing guide
1 parent 14bdcab commit 3f2bd78

File tree

3 files changed

+268
-0
lines changed

3 files changed

+268
-0
lines changed

‎contributing.md‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
# Contributing to the Wiki
2+
3+
This is a 4,000+ page Sphinx site built from Markdown files. The original content came from MoinMoin (the wiki engine that powered wiki.python.org for two decades), converted to [MyST Markdown](https://myst-parser.readthedocs.io/) and rendered with the [Shibuya](https://shibuya.lepture.com/) theme.
4+
5+
There are two ways to contribute: edit files directly through GitHub, or use the browser-based CMS. Both end up as pull requests against the `main` branch.
6+
7+
## Editing on GitHub
8+
9+
The most straightforward path. Every wiki page is a `.md` file under one of three directories:
10+
11+
- `python/` -- the main Python wiki (3,274 pages)
12+
- `psf/` -- Python Software Foundation governance and programs (378 pages)
13+
- `jython/` -- Jython, the JVM implementation (433 pages)
14+
15+
Find the file you want to change, edit it, and open a pull request. GitHub's web editor works fine for quick fixes. For anything more involved, clone the repo and work locally (see [Local Development](#local-development) below).
16+
17+
Pages use MyST Markdown, which is standard Markdown plus a handful of Sphinx directives. You can use `{admonition}`, `{toctree}`, `{grid}`, code blocks, tables -- all the usual stuff. The [MyST docs](https://myst-parser.readthedocs.io/en/latest/syntax/roles-and-directives.html) cover the full syntax.
18+
19+
### Adding a new page
20+
21+
1. Create a `.md` file in the right directory. Start it with a top-level heading (`# Page Title`).
22+
2. Add the filename (without `.md`) to the `{toctree}` in the parent `index.md`.
23+
24+
If you're making a new subdirectory, put an `index.md` inside it with its own toctree, then reference `subdir/index` from the parent.
25+
26+
### What about redirects?
27+
28+
The old MoinMoin wiki encoded special characters in URLs in its own way -- spaces became `(20)`, slashes became `(2f)`, German umlauts like `Ä` became `(c384)`. All 5,400+ redirects from old URLs to new paths live in `_redirects.json`.
29+
30+
If you move or rename a page, add an entry there so existing links don't break. You can regenerate the full redirect map with:
31+
32+
```bash
33+
make redirects
34+
```
35+
36+
## Editing with Decap CMS
37+
38+
For people who'd rather not touch Git at all, there's a browser-based editor at `/admin/` on the live site. It's powered by [Decap CMS](https://decapcms.org/) (the maintained fork of Netlify CMS).
39+
40+
You log in with your GitHub account, pick a page from any of the three wiki sections, edit it in a rich-text-ish Markdown editor, and save. Behind the scenes, Decap CMS creates a branch and pull request on your behalf. The editorial workflow means your changes go through review before merging, same as any other PR.
41+
42+
The CMS configuration lives in `_extra/admin/config.yml`. It defines three collections (one per wiki section) and uses `open_authoring`, which means anyone with a GitHub account can propose edits -- they don't need write access to the repo.
43+
44+
:::{admonition} Contributing without a GitHub account
45+
:class: warning
46+
47+
This is a work in progress. We're looking into options for allowing edits without requiring a GitHub account at all. The current CMS flow and the standard PR workflow both need one. If you have content to contribute but no GitHub account, reach out to the PSF or open an issue and we'll figure something out.
48+
:::
49+
50+
## How the CMS authentication works
51+
52+
Decap CMS needs an OAuth handshake with GitHub to get a token for the user. GitHub's OAuth flow requires a server-side component (client secrets can't live in browser JS), so there's a small [Litestar](https://litestar.dev/) app in the `oauth/` directory that handles this.
53+
54+
The flow goes like this:
55+
56+
1. User clicks "Login with GitHub" in the CMS.
57+
2. The CMS redirects to `GET /auth` on the OAuth proxy.
58+
3. The proxy redirects to GitHub's authorize page with the app's client ID.
59+
4. User approves, GitHub redirects back to `GET /callback?code=...`.
60+
5. The proxy exchanges that code for an access token server-side.
61+
6. The proxy returns a small HTML page that `postMessage`s the token back to the CMS window.
62+
63+
That's the entire app. Three routes: `/auth`, `/callback`, and `/_health/` for load balancers. The source is `oauth/app.py` -- it's about 70 lines.
64+
65+
### OAuth proxy stack
66+
67+
The proxy is its own Python project with its own `pyproject.toml` and `uv.lock`, separate from the main wiki's Sphinx dependencies. It uses:
68+
69+
- **Litestar** for the web framework
70+
- **httpx** for the async HTTP call to GitHub's token endpoint
71+
- **uvicorn** as the ASGI server
72+
73+
In production, it runs on Cabotage (the PSF's deployment platform) behind an nginx ingress, binding to a Unix socket at `/var/run/cabotage/cabotage.sock`. The `Dockerfile` and `Procfile` in `oauth/` handle that. It's reachable at `api.wiki.python.org`.
74+
75+
The OpenAPI docs for the proxy are auto-generated by Litestar and served at `/` via the Scalar UI plugin -- hit `http://localhost:8000/` when running locally and you'll see them.
76+
77+
## Local development
78+
79+
You need [uv](https://docs.astral.sh/uv/) and Python 3.14+.
80+
81+
```bash
82+
make install
83+
```
84+
85+
That's the whole setup. No virtualenv juggling, no pip -- uv does it all.
86+
87+
### Previewing the wiki
88+
89+
The wiki has ~3,500 pages total, so building everything takes a while. You almost certainly want to scope your build to whatever section you're working on:
90+
91+
```bash
92+
# Just the PSF wiki (~400 pages, under a minute)
93+
make docs-serve-fast WIKI=psf
94+
95+
# A single subsection (~100 pages, a few seconds)
96+
make docs-serve-fast WIKI=psf SECTION=PackagingWG
97+
98+
# The full Python wiki (~3,400 pages)
99+
make docs-serve-fast WIKI=python
100+
```
101+
102+
These start a live-reload server. Save a file, the browser refreshes.
103+
104+
For a complete build of all sections (what CI runs):
105+
106+
```bash
107+
make docs
108+
```
109+
110+
### Running the OAuth proxy locally
111+
112+
If you need to work on the CMS integration:
113+
114+
```bash
115+
# Set your GitHub OAuth app credentials
116+
export GITHUB_CLIENT_ID=...
117+
export GITHUB_CLIENT_SECRET=...
118+
119+
# Or put them in oauth/.env or .env at the repo root
120+
121+
make oauth-serve
122+
```
123+
124+
This starts the proxy at `http://localhost:8000` with auto-reload. Run the test suite with:
125+
126+
```bash
127+
make oauth-test
128+
```
129+
130+
## CI/CD
131+
132+
Three GitHub Actions workflows keep things running:
133+
134+
**`docs.yml`** -- Builds the full Sphinx site on every push to `main` and deploys to GitHub Pages. Also runs on PRs to catch build errors before merge. Uses the Sphinx doctree cache so incremental builds don't start from scratch.
135+
136+
**`preview.yml`** -- Generates a deploy preview for every pull request. It detects which wiki section(s) changed and scopes the build accordingly -- if you only touched files in `psf/`, it only builds the PSF wiki. The preview URL gets posted as a comment on the PR.
137+
138+
**`linkcheck.yml`** -- Runs weekly (Monday 6am UTC) to scan for dead external links across all pages. Results get uploaded as artifacts.
139+
140+
## Project layout
141+
142+
```
143+
python/ Python wiki content (3,274 pages)
144+
psf/ PSF wiki content (378 pages)
145+
jython/ Jython wiki content (433 pages)
146+
oauth/ GitHub OAuth proxy for Decap CMS
147+
app.py The Litestar application (3 routes)
148+
Dockerfile Production container image
149+
Procfile Cabotage process definition
150+
pyproject.toml Proxy-specific dependencies
151+
tests/ Proxy test suite
152+
k8s/ Kubernetes manifests (ingress)
153+
_extra/admin/ Decap CMS frontend (config.yml + loader HTML)
154+
_static/ CSS and static assets
155+
_templates/ Sphinx HTML template overrides
156+
_redirects.json All 5,400+ redirect mappings
157+
_redirects_html/ Generated static redirect pages
158+
scripts/ Conversion and maintenance tooling
159+
convert.py MoinMoin HTML → MyST Markdown converter
160+
gen_old_wiki_redirects.py Builds the old-URL redirect map
161+
gen_redirect_pages.py Generates static HTML redirect files
162+
strip_attrs.py Strips pandoc attribute cruft from .md files
163+
fix_moin_links.py Fixes remaining MoinMoin-style links
164+
reorganize.py Moves pages into subdirectories
165+
sync.sh Pulls raw HTML from the wiki server
166+
conf.py Sphinx configuration
167+
pyproject.toml Wiki build dependencies (uv-managed)
168+
Makefile All the build/serve/lint commands
169+
```
170+
171+
## Make targets
172+
173+
Run `make help` to see everything, but here's the short version:
174+
175+
::::{tab-set}
176+
177+
:::{tab-item} Setup
178+
```
179+
make install Install all dependencies
180+
```
181+
:::
182+
183+
:::{tab-item} Content Pipeline
184+
```
185+
make sync Sync raw HTML from wiki server
186+
make convert Re-run the HTML → Markdown conversion
187+
```
188+
:::
189+
190+
:::{tab-item} Documentation
191+
```
192+
make docs Build the full Sphinx site
193+
make docs-serve Serve with live reload (all sections)
194+
make docs-serve-fast Scoped build (WIKI=python|psf|jython [SECTION=subdir])
195+
make docs-clean Remove Sphinx build output
196+
```
197+
:::
198+
199+
:::{tab-item} Code Quality
200+
```
201+
make lint Run pre-commit hooks
202+
make redirects Regenerate redirect mapping and static HTML files
203+
```
204+
:::
205+
206+
:::{tab-item} OAuth Proxy
207+
```
208+
make oauth-serve Run OAuth proxy locally (needs GITHUB_CLIENT_ID/SECRET)
209+
make oauth-test Run OAuth proxy tests
210+
```
211+
:::
212+
213+
:::{tab-item} Utility
214+
```
215+
make clean Remove all build artifacts
216+
```
217+
:::
218+
219+
::::

‎index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,4 +187,5 @@ JVM Integration · Java Interop · Guides
187187
python/index
188188
psf/index
189189
jython/index
190+
contributing
190191
```

‎oauth/k8s/ingress.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Wiki OAuth Proxy - Cabotage Ingress
2+
# Apply manually: kubectl apply -f k8s/ingress.yml
3+
---
4+
apiVersion: networking.k8s.io/v1
5+
kind: Ingress
6+
metadata:
7+
name: wiki-wiki-oauth-app-web
8+
namespace: coffee
9+
annotations:
10+
cert-manager.io/cluster-issuer: letsencrypt
11+
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
12+
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
13+
nginx.ingress.kubernetes.io/proxy-connect-timeout: 10s
14+
nginx.ingress.kubernetes.io/proxy-read-timeout: 10s
15+
nginx.ingress.kubernetes.io/proxy-send-timeout: 10s
16+
nginx.ingress.kubernetes.io/service-upstream: "true"
17+
labels:
18+
app: wiki-wiki-oauth-app
19+
process: web
20+
resident-service.cabotage.io: "true"
21+
spec:
22+
ingressClassName: nginx
23+
rules:
24+
- host: coffee-wiki-wiki-oauth-app-web.ingress.us-east-2.psfhosted.computer
25+
http:
26+
paths:
27+
- backend:
28+
service:
29+
name: wiki-wiki-oauth-app-web
30+
port:
31+
number: 8000
32+
path: /
33+
pathType: Prefix
34+
- host: api.wiki.python.org
35+
http:
36+
paths:
37+
- backend:
38+
service:
39+
name: wiki-wiki-oauth-app-web
40+
port:
41+
number: 8000
42+
path: /
43+
pathType: Prefix
44+
tls:
45+
- hosts:
46+
- coffee-wiki-wiki-oauth-app-web.ingress.us-east-2.psfhosted.computer
47+
- api.wiki.python.org
48+
secretName: ingress-wiki-wiki-oauth-app-web

0 commit comments

Comments
 (0)