Skip to content

Commit 69b45d5

Browse files
tballisontballison
authored
Add site update skill (#2963)
1 parent 4d1f7cb commit 69b45d5

5 files changed

Lines changed: 869 additions & 0 deletions

File tree

‎.skills/update-site-for-release/SKILL.md‎

Lines changed: 346 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,346 @@
1+
---
2+
name: update-site-for-release
3+
description: >
4+
Update/publish the Apache Tika website (tika-site SVN repo) for a release —
5+
step 17 of the Release Process. Handles STABLE (full per-version docs +
6+
javadoc) vs PREVIEW/beta (Changes page + Antora docs). Use for "update the
7+
site", "publish the site for X.Y.Z", "the website part of the release".
8+
---
9+
10+
<!--
11+
Licensed to the Apache Software Foundation (ASF) under one or more
12+
contributor license agreements. See the NOTICE file distributed with
13+
this work for additional information regarding copyright ownership.
14+
The ASF licenses this file to You under the Apache License, Version 2.0
15+
(the "License"); you may not use this file except in compliance with
16+
the License. You may obtain a copy of the License at
17+
18+
http://www.apache.org/licenses/LICENSE-2.0
19+
20+
Unless required by applicable law or agreed to in writing, software
21+
distributed under the License is distributed on an "AS IS" BASIS,
22+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23+
See the License for the specific language governing permissions and
24+
limitations under the License.
25+
-->
26+
27+
# Update the Tika website for a release
28+
29+
Step 17 ("Update Tika site") of the Release Process
30+
(<https://cwiki.apache.org/confluence/spaces/TIKA/pages/109454070/Release+Process>).
31+
Assumes the release (tag, artifacts, VOTE, dist promotion) is done; covers the
32+
**website** only.
33+
34+
Set these first:
35+
- **`$SITE`** — `tika-site` **SVN** checkout (not git): `src/site/` (sources) +
36+
`publish/` (generated, SVN-tracked, served; `mvn install` regenerates it and
37+
auto-runs `svn add --force publish` via antrun).
38+
- **`$SCRATCH`** — release working dir: unzipped src release, `CHANGES-<NEW>.txt`,
39+
built javadoc.
40+
41+
Scripts: `./scripts/`. Local paths + toolchain (Maven binary, JDKs) live in a
42+
private companion skill under `~/.claude/skills/`.
43+
44+
> **[HUMAN] gates — never do these yourself:** final `svn commit` (outward-facing,
45+
> irreversible), JIRA "release", s.apache.org shortlink, announce emails. Prepare
46+
> everything, show `svn status`, hand off.
47+
48+
---
49+
50+
## 0. Inputs + release type
51+
52+
| Input | Example | Notes |
53+
|---|---|---|
54+
| `NEW_VERSION` | `3.3.2` | the release |
55+
| `RELEASE_TYPE` | `stable` | `stable` or `preview` (alpha/beta/BETA) |
56+
| `PREV_STABLE` / `PREV_PREVIEW` | `3.3.1` / `4.0.0-beta-1` | current `tika.stable.version` / `tika.preview.version` |
57+
| `PREV_TAG` / `NEW_TAG` | `3.3.1` / `3.3.2` | git tags, for the GitHub contributor query |
58+
| `JIRA fixVersion` | `3.3.2` | **may differ** from the label (betas often use the base version, e.g. `4.0.0`) |
59+
| `CHANGES` file | `$SCRATCH/CHANGES-3.3.2.txt` | notable-changes source |
60+
| src release zip | `tika-<NEW>-src.zip` | **stable only** (javadoc) |
61+
| release date | `2026-07-21` | doap.rdf + news blurb |
62+
63+
Confirm current values in `src/site/pom.xml`.
64+
65+
| Step | STABLE | PREVIEW / beta |
66+
|---|---|---|
67+
| `pom.xml` `<parent><version>` | → `<NEW>` | leave at stable |
68+
| `pom.xml` `tika.stable.version` | → `<NEW>` | leave |
69+
| `pom.xml` `tika.preview.version` | leave | → `<NEW>` |
70+
| `src/site/apt/<NEW>/` | full 8-file set (scaffold from prev) | **only `index.apt`** |
71+
| `site.xml` entry | full sub-menu, expanded | minimal `Changes` item under `docs/<major>-SNAPSHOT` |
72+
| formats.apt | regenerate from `tika-app` jar | n/a (Antora docs) |
73+
| javadoc | `clean install -Pfast` + `javadoc:aggregate` → `publish/<NEW>/api` (step 7) | n/a |
74+
| Antora docs | n/a | `mvn package -pl docs` + `docs/publish-docs.sh` |
75+
| Download page | automatic | automatic |
76+
77+
doap.rdf, index.apt.vm news, verify, publish are common to both.
78+
79+
---
80+
81+
## 1. `src/site/pom.xml` versions [AGENT]
82+
83+
- **STABLE:** bump `<parent><version>` **and** `<tika.stable.version>` to `<NEW>`.
84+
(parent version drives `${project.parent.version}` home-page links → must be
85+
newest stable.)
86+
- **PREVIEW:** bump only `<tika.preview.version>`; leave parent + stable.
87+
88+
Download page auto-reads these — no manual edit.
89+
90+
---
91+
92+
## 2. `src/site/site.xml` menu [AGENT]
93+
94+
Current stable + current preview expanded; older = `collapse="true"`.
95+
96+
- **STABLE:** new expanded block above the previous stable; add `collapse="true"`
97+
to the old stable block:
98+
```xml
99+
<item name="Apache Tika 3.3.2" href="3.3.2/index.html">
100+
<item name="Getting Started" href="3.3.2/gettingstarted.html"/>
101+
<item name="Supported Formats" href="3.3.2/formats.html"/>
102+
<item name="Parser API" href="3.3.2/parser.html"/>
103+
<item name="Parser 5min Quick Start Guide" href="3.3.2/parser_guide.html"/>
104+
<item name="Content and Language Detection" href="3.3.2/detection.html"/>
105+
<item name="Configuring Tika" href="3.3.2/configuring.html"/>
106+
<item name="Usage Examples" href="3.3.2/examples.html"/>
107+
<item name="API Documentation" href="3.3.2/api/"/>
108+
</item>
109+
```
110+
- **PREVIEW:** minimal block above the old preview (collapse it); `href` → Antora
111+
snapshot, not an apt page:
112+
```xml
113+
<item name="Apache Tika 4.0.0-beta-2" href="docs/4.0.0-SNAPSHOT">
114+
<item name="Changes" href="4.0.0-beta-2/index.html"/>
115+
</item>
116+
```
117+
118+
---
119+
120+
## 3. Per-version apt docs `src/site/apt/<NEW>/` [AGENT]
121+
122+
**STABLE** — scaffold (these docs are version-string-identical across 3.x):
123+
```bash
124+
./scripts/scaffold-stable-version.sh $SITE 3.3.1 3.3.2
125+
```
126+
Copies+bumps `configuring/detection/examples/parser/parser_guide/gettingstarted.apt`. Then:
127+
- **formats.apt** — two parts: a hand-written top (license, intro, `%{toc}`, ~25
128+
prose format-family sections) down to the header line
129+
`Full list of Supported Formats in "standard" artifacts`, then a generated flat
130+
list below it. `--list-parser-details-apt` regenerates only the flat part. Copy
131+
prev `formats.apt`, bump versions, replace everything **below** that header:
132+
```bash
133+
java -jar <path>/tika-app-<NEW>.jar --list-parser-details-apt
134+
```
135+
**Do NOT truncate at the `%{toc}` line** — keep the prose sections. (Scaffold
136+
skips `formats.apt` for this reason.)
137+
- **index.apt** — step 4.
138+
139+
**PREVIEW** — create only `index.apt` (copy prev preview's; title `Apache Tika
140+
<NEW>`; fill step 4). No other files.
141+
142+
---
143+
144+
## 4. Per-version `index.apt`: notable changes + contributors [AGENT + HUMAN]
145+
146+
Shape (see `src/site/apt/3.3.1/index.apt`): license+title; "most notable changes…"
147+
bullets; "The following people have contributed…" bullets; "See
148+
{{https://s.apache.org/XXXX}} …".
149+
150+
**Notable changes** — review output, keep only notable items:
151+
```bash
152+
./scripts/extract-tika-issues.py CHANGES-3.3.2.txt out-3.3.2.apt 3.3.2
153+
```
154+
Mirrors CHANGES verbatim; TIKA-####/Github-#### auto-linked; ALL-CAPS headers →
155+
apt sections.
156+
157+
**Contributors** — candidate list, RM curates:
158+
```bash
159+
./scripts/extract-tika-contribs.py 3.3.2 --prev-tag 3.3.1 --tag 3.3.2 > contribs.txt
160+
# beta (fixVersion differs from label):
161+
# ./scripts/extract-tika-contribs.py 4.0.0 --prev-tag 3.3.1 --tag 4.0.0-beta-1
162+
```
163+
Merges JIRA (reporters/assignees/comment authors) + GitHub commit/PR authors
164+
(uses `gh` auth; resolves logins→names; case-insensitive sort; filters bots/AI).
165+
Over-reports drive-by commenters, misses GitHub-issue-only commenters. **[HUMAN]**
166+
prune / normalise / add.
167+
168+
**Shortlink [HUMAN]** — `s.apache.org/XXXX` → the JIRA "issues fixed in <NEW>"
169+
query; needs s.apache.org login. Ask the RM.
170+
171+
---
172+
173+
## 5. `src/site/resources/doap.rdf` [AGENT]
174+
175+
New `<release>` at the top. Ordering is by **date, not version** (a stable point
176+
release can sit above an older-dated preview — 3.3.2/Jul-16 above 4.0.0-beta-1/Jul-3):
177+
```xml
178+
<release>
179+
<Version>
180+
<name>Apache Tika 3.3.2</name>
181+
<created>2026-07-21</created>
182+
<revision>3.3.2</revision>
183+
</Version>
184+
</release>
185+
```
186+
187+
---
188+
189+
## 6. Home page `src/site/apt/index.apt.vm` [AGENT + HUMAN]
190+
191+
1. New **Latest News** block at the top; its CHANGES link uses
192+
`dist.apache.org/repos/dist/release/...` (live mirror):
193+
```
194+
[21 July 2026: Apache Tika Release]
195+
Apache Tika 3.3.2 has been released! <one or two sentence summary>.
196+
Please see the {{{https://dist.apache.org/repos/dist/release/tika/3.3.2/CHANGES-3.3.2.txt}CHANGES.txt}}
197+
file for the full list of changes in the release and have a look at the download page for more information
198+
on how to obtain Apache Tika 3.3.2.
199+
```
200+
2. **Repoint the superseded release's CHANGES link** (its artifacts get `svn rm`'d
201+
from the live mirror at release): `dist.apache.org/repos/dist/release/tika/<PREV>/…`
202+
→ `archive.apache.org/dist/tika/<PREV>/…`. **[HUMAN]** confirm which version was
203+
removed.
204+
205+
---
206+
207+
## 7. Docs / Javadoc
208+
209+
**STABLE — javadoc [AGENT].** NOT the wiki's `javadoc:aggregate-no-fork` (runs
210+
against `tika-parent`, its relative `<sourcepath>` fails → `No source files for
211+
package org.apache.tika`; wrong goal, not a JDK issue). From the unzipped src
212+
release (its `./mvnw` is broken — use system `mvn`):
213+
```bash
214+
unzip tika-3.3.2-src.zip && cd tika-3.3.2
215+
mvn clean install -Pfast # ~4 min; module artifacts + full dep classpath
216+
mvn javadoc:aggregate # FORKING goal (NOT -no-fork)
217+
mkdir -p $SITE/publish/3.3.2
218+
mv target/reports/apidocs $SITE/publish/3.3.2/api
219+
```
220+
Both steps matter: without `install` javadoc dies on `package org.slf4j does not
221+
exist`; the forking `aggregate` (@aggregator) runs once on the root, `-no-fork`
222+
breaks per-`pom`-module. Any modern JDK (11 and 25 verified). (`tika-server`
223+
miredot docs discontinued — skip.)
224+
225+
**PREVIEW (4.x) — Antora docs [AGENT]:**
226+
```bash
227+
cd tika-<NEW> # unzipped src release
228+
./mvnw package -pl docs
229+
./docs/publish-docs.sh $SITE/publish
230+
```
231+
`publish-docs.sh` copies target/site into `publish/docs/`, flattens URLs, rewrites
232+
the search index (has its own guards).
233+
234+
---
235+
236+
## 8. Build + verify [AGENT]
237+
238+
`tika-site` has no `./mvnw` → system `mvn`. **Build with Java 11** — it pins
239+
maven-site-plugin 3.4 (2014), unreliable on newer JDKs; a Doxia error here means
240+
wrong JDK, not a content problem (separate from step 7's JDK-agnostic javadoc).
241+
```bash
242+
cd "$SITE"
243+
mvn clean install
244+
```
245+
> **ALWAYS `clean install`, never bare `install`** — an incremental build leaves
246+
> `publish/css/` stale → pages render with no CSS/sidebar. Fix is a `clean`
247+
> rebuild, not a CSS edit.
248+
249+
Build auto-copies target/site → `publish/`, strips timestamps, `svn add --force
250+
publish`. Check: new version in the menu; news + download versions right; **pages
251+
styled (CSS + sidebar)**; per-version pages + javadoc/Antora resolve. Preview:
252+
`mvn site:run` → <http://localhost:8080>.
253+
254+
---
255+
256+
## 9. Stage + hand off the commit [HUMAN]
257+
258+
```bash
259+
cd "$SITE"
260+
svn status
261+
svn add src/site/apt/<NEW> # + any other new files
262+
# hand to the RM — do NOT run yourself:
263+
# svn commit -m "Update website for <NEW> release."
264+
```
265+
266+
**Big-commit caveat (stable):** `publish/<NEW>/api` is ~3,000 files / ~55 MB; a
267+
single commit often **times out / `E000104 Connection reset by peer`** — this is
268+
size, NOT auth (bad password = `Authentication failed`/403, and cached creds won't
269+
re-prompt). Fixes:
270+
- `http-timeout = 1800` in `~/.subversion/servers` `[global]`.
271+
- Else commit the api in chunks, then the rest:
272+
```bash
273+
svn commit --depth=empty publish/<NEW> publish/<NEW>/api \
274+
publish/<NEW>/api/org publish/<NEW>/api/org/apache \
275+
publish/<NEW>/api/org/apache/tika -m "<NEW> site: api dir skeleton"
276+
for d in publish/<NEW>/api/org/apache/tika/*/; do
277+
svn commit "$d" -m "<NEW> javadoc: $(basename "$d")" || break # parser/ ~1,300 files
278+
done
279+
svn commit publish/<NEW>/api -m "<NEW> javadoc: remaining api files"
280+
svn commit -m "Update website for <NEW> release."
281+
```
282+
- Atomic per invocation → a failed commit rolls back; retry. Locked (`E155004`) →
283+
`svn cleanup`.
284+
285+
---
286+
287+
## 10. Confirm published + re-kick [HUMAN]
288+
289+
svnwcsub maps `/www/tika.apache.org ← %(ASF)s/tika/site/publish`: **only a commit
290+
touching `publish/` triggers a republish**, and it publishes the whole tree at
291+
HEAD. Verify (cache-buster hits the origin, not Varnish):
292+
```bash
293+
curl -s -o /dev/null -w "%{http_code}\n" "https://tika.apache.org/<NEW>/index.html?cb=$(date +%s)"
294+
```
295+
Want `200`. Still `404`/old minutes later → the web-node `svn up` choked on the
296+
big commit. **Re-kick** with a trivial whitespace commit to a file **under
297+
`publish/`** (e.g. a blank line in `publish/index.html`):
298+
```bash
299+
svn commit publish/index.html -m "Nudge svnwcsub to republish."
300+
```
301+
Re-fires svnwcsub → `svn up` to HEAD (harmless; next build regenerates it). A
302+
commit outside `publish/` won't trigger. Still stuck ~30 min → ping `#asfinfra`.
303+
304+
---
305+
306+
## 11. Post-site [HUMAN] (context)
307+
308+
- JIRA: "release" `<NEW>`; move stragglers to the next version.
309+
- Announce to `user@`, `dev@`, `announce@apache.org`.
310+
- Log at <https://reporter.apache.org/addrelease.html?tika>.
311+
- CVEs fixed → update `security.apt` / per-version security notes, republish.
312+
313+
---
314+
315+
## Checklist
316+
317+
- [ ] stable vs preview decided
318+
- [ ] `pom.xml` versions (parent+stable for stable; preview only for preview)
319+
- [ ] `site.xml`: new entry added, previous same-track entry collapsed
320+
- [ ] per-version apt docs (full set for stable / `index.apt` only for preview)
321+
- [ ] `formats.apt` regenerated (stable)
322+
- [ ] `index.apt`: notable changes + curated contributors + shortlink
323+
- [ ] `doap.rdf` entry
324+
- [ ] `index.apt.vm`: news block + superseded CHANGES link → archive
325+
- [ ] javadoc → `publish/<NEW>/api` (stable) / Antora docs (preview)
326+
- [ ] `mvn clean install` (never bare `install`); pages styled (CSS + sidebar)
327+
- [ ] `svn status`/`svn add` done, commit handed to RM (chunk `api/` if it resets)
328+
- [ ] live site 200: `https://tika.apache.org/<NEW>/index.html?cb=…` — else re-kick
329+
330+
---
331+
332+
## Troubleshooting
333+
334+
| Symptom | Cause | Fix |
335+
|---|---|---|
336+
| `aggregate-no-fork` → `No source files for package org.apache.tika` | runs against `tika-parent`; relative `<sourcepath>` can't resolve | use forking `javadoc:aggregate` after `clean install -Pfast` (step 7) |
337+
| javadoc → `package org.slf4j does not exist` etc. | aggregate without a prior build → empty classpath | `mvn clean install -Pfast` first |
338+
| pages unstyled (no CSS/sidebar) | incremental `install` left `publish/css/` stale | `mvn clean install` (never bare `install`) |
339+
| site-plugin / Doxia error on `mvn install` | maven-site-plugin 3.4 on too-new a JDK | build with **Java 11** |
340+
| notable-changes bullet split on a version number | old numeric heuristic (removed) | use the bundled script; re-run |
341+
| contributors have bots/AI, or surname order | old behavior (fixed): filter + `str.casefold` sort | use bundled `extract-tika-contribs.py`; RM curates |
342+
| commit `E175012 timed out` / `E000104 Connection reset` | ~55 MB api tree too big for one transaction | `http-timeout=1800`; chunk the `api/` (step 9). Size, not auth. |
343+
| commit `Authentication failed` / 403 | genuinely bad/expired credential | `svn commit --username <you>` to re-cache |
344+
| `svn: E155004 working copy locked` | prior commit died mid-transaction | `svn cleanup`, retry |
345+
| committed, site still old even with `?cb=` (origin 404s/old) | web-node `svn up` choked on the big commit | re-kick: whitespace commit under `publish/` (step 10); stuck ~30 min → `#asfinfra` |
346+
| home-page CHANGES link 404s for the previous release | it was `svn rm`'d from the live dist mirror | repoint to `archive.apache.org/dist/tika/<prev>/CHANGES-<prev>.txt` (step 6) |

0 commit comments

Comments
 (0)