Skip to content

Commit 3143565

Browse files
dshkolclaude
andcommitted
Add documentation: tests, chart guide, workflow updates
Documentation improvements: - Add comprehensive test suite for fix-paths.js (22 tests) - Add npm test script to package.json - Create CHART-STYLE-GUIDE.md with Observable Plot conventions - Update WORKFLOW.md for Observable Framework workflow - Add troubleshooting section to WORKFLOW.md - Refactor fix-paths.js to export testable functions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 2fdd374 commit 3143565

5 files changed

Lines changed: 673 additions & 76 deletions

File tree

.claude/skills/the-daily-generator/WORKFLOW.md

Lines changed: 187 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -74,74 +74,154 @@ Rscript r-tools/fetch_cansim_enhanced.R <table-number> output
7474
}
7575
```
7676

77-
## Step 2: Article Generation
77+
## Step 2: Article Generation (Observable Framework)
7878

79-
**Command:**
80-
```bash
81-
python3 generate_article_enhanced.py <data.json> <output.html>
82-
```
79+
Create markdown files directly in the Observable Framework structure:
80+
81+
**English article:** `docs/en/<slug>/index.md`
82+
**French article:** `docs/fr/<slug-fr>/index.md`
8383

8484
**Naming convention for slugs:**
85-
- CPI: `cpi-<month>-<year>.html` (e.g., `cpi-november-2025.html`)
86-
- Retail: `retail-<month>-<year>.html`
87-
- Labour: `employment-<month>-<year>.html`
88-
- Generic: `<short-title>-<month>-<year>.html`
89-
90-
**What it generates:**
91-
1. **Headline** - Max 15 words, leads with key statistic
92-
2. **Highlights** - 3-5 bullet points with key findings
93-
3. **Narrative sections** - Lede, analysis, comparisons
94-
4. **Main chart** - Time series line chart (last 24 months)
95-
5. **Data tables** - Component and provincial breakdowns
96-
6. **Bar chart** - YoY changes by component
97-
7. **Source section** - StatCan links, release date, table number
98-
99-
## Step 3: Quality Review
100-
101-
**Open in browser:**
85+
86+
| Type | English Slug | French Slug |
87+
|------|-------------|-------------|
88+
| CPI | `cpi-november-2025` | `ipc-novembre-2025` |
89+
| Retail | `retail-trade-october-2025` | `commerce-detail-octobre-2025` |
90+
| Labour | `lfs-november-2025` | `epa-novembre-2025` |
91+
| GDP | `gdp-october-2025` | `pib-octobre-2025` |
92+
| Generic | `<indicator>-<month>-<year>` | `<indicateur>-<mois>-<année>` |
93+
94+
**Article template structure:**
95+
96+
```markdown
97+
---
98+
title: Consumer prices up 2.2% year over year in November 2025
99+
toc: false
100+
---
101+
102+
# Consumer prices up 2.2% year over year in November 2025
103+
104+
<p class="release-date">Released: December 22, 2025 <span class="article-type-tag release">New Release</span></p>
105+
106+
<div class="highlights">
107+
108+
**Highlights**
109+
110+
- Key finding 1 with specific number
111+
- Key finding 2 with comparison
112+
- Regional highlight
113+
- Secondary finding
114+
115+
</div>
116+
117+
[Body paragraphs with analysis]
118+
119+
\`\`\`js
120+
import * as Plot from "npm:@observablehq/plot";
121+
122+
// Real data from Statistics Canada Table XX-XX-XXXX
123+
const data = [...];
124+
125+
display(Plot.plot({...}));
126+
\`\`\`
127+
128+
[More sections, charts, tables]
129+
130+
<div class="note-to-readers">
131+
132+
## Note to readers
133+
134+
[Methodology explanation]
135+
136+
</div>
137+
138+
<div class="source-info">
139+
140+
**Source:** Statistics Canada, [Table XX-XX-XXXX](https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=XXXXXXXXXX)
141+
**Survey:** Survey Name
142+
**Reference period:** Month Year
143+
**DOI:** [https://doi.org/10.25318/XXXXXXXXXX-eng](https://doi.org/10.25318/XXXXXXXXXX-eng)
144+
145+
</div>
146+
```
147+
148+
See [CHART-STYLE-GUIDE.md](/CHART-STYLE-GUIDE) for chart conventions.
149+
150+
## Step 3: Update Language Mapping
151+
152+
**Critical step!** The language switcher won't work without this.
153+
154+
**File:** `src/lang-map.js`
155+
156+
Add the EN/FR slug pair:
157+
158+
```js
159+
export const slugMap = {
160+
// ... existing mappings
161+
'new-article-slug': 'nouvel-article-slug',
162+
};
163+
```
164+
165+
**Example mappings:**
166+
```js
167+
'cpi-november-2025': 'ipc-novembre-2025',
168+
'lfs-november-2025': 'epa-novembre-2025',
169+
'gdp-october-2025': 'pib-octobre-2025',
170+
```
171+
172+
## Step 4: Update Index Pages
173+
174+
Add the new article to both feed pages:
175+
176+
**English:** `docs/en/index.md`
177+
**French:** `docs/fr/index.md`
178+
179+
Entry format:
180+
```markdown
181+
### [Consumer prices up 2.2% year over year in November 2025](./cpi-november-2025/)
182+
183+
Released: 2025-12-22 · Table 18-10-0004
184+
185+
The Consumer Price Index rose 2.2% year over year in November 2025...
186+
```
187+
188+
## Step 5: Quality Review
189+
190+
**Preview locally:**
102191
```bash
103-
open output/articles/<slug>.html
104-
# or use browser automation to display
192+
npm run dev
193+
# Opens at http://localhost:3000
105194
```
106195

107196
**Checklist:**
108-
- [ ] Headline is accurate and leads with key number
109-
- [ ] Highlights capture the most newsworthy points
110-
- [ ] YoY and MoM percentages match the data
111-
- [ ] Charts render correctly
197+
- [ ] Headline leads with key statistic
198+
- [ ] Highlights capture most newsworthy points
199+
- [ ] YoY and MoM percentages match JSON data
200+
- [ ] Charts render with correct colors (#AF3C43)
112201
- [ ] Data tables are sorted and formatted properly
113202
- [ ] Source links point to correct StatCan pages
203+
- [ ] Language switcher toggles between EN/FR versions
114204
- [ ] Voice is neutral (no "surged", "plummeted", etc.)
205+
- [ ] French uses comma decimals (2,2 % not 2.2%)
115206

116-
**If issues found:**
117-
- Edit `generate_article_enhanced.py` for systematic fixes
118-
- Or manually adjust the HTML for one-off corrections
119-
- Regenerate if needed
207+
## Step 6: Build and Deploy
120208

121-
## Step 4: Site Build
122-
123-
**Command:**
209+
**Build the site:**
124210
```bash
125-
python3 build_site.py
211+
npm run build
126212
```
127213

128-
**What it does:**
129-
- Scans `output/articles/` for all HTML files
130-
- Extracts metadata (title, date, table number) from each
131-
- Generates `site/index.html` with latest 5 articles
132-
- Generates `site/archive.html` with all articles by month
133-
- Copies articles to `site/articles/`
134-
- Saves `site/articles.json` with full metadata
135-
136-
## Step 5: Publish
214+
This runs Observable Framework build plus the Safari path fix script.
137215

138-
**Commands:**
216+
**Commit and push:**
139217
```bash
140-
git add output/ site/
218+
git add docs/ src/
141219
git commit -m "Add: <headline from article>"
142220
git push
143221
```
144222

223+
GitHub Actions will auto-deploy to Netlify.
224+
145225
**Commit message format:**
146226
```
147227
Add: Consumer prices up 2.2% year over year in November 2025
@@ -207,22 +287,76 @@ Always verify fetched data against:
207287
2. Recent StatCan Daily releases (via web search)
208288
3. Third-party sources when available (CMHC, industry reports)
209289

210-
## Common Issues
290+
## Troubleshooting
291+
292+
### R Script Issues
211293

212294
**R script fails to download:**
213295
- Check internet connection
214296
- Verify table number format (XX-XX-XXXX)
215297
- Table may be deprecated; check StatCan for replacement
298+
- Try `--refresh` flag to force re-download: `Rscript r-tools/fetch_cansim_enhanced.R 18-10-0004 output --refresh`
299+
300+
**"Error: object 'X' not found":**
301+
- Column name may differ from expected
302+
- Run `names(df)` in R to see actual column names
303+
- Table structure may have changed; update detection logic
216304

217305
**Missing subseries data:**
218306
- Table may not have component breakdowns
219307
- Check `breakdown_dimension` in JSON output
308+
- Some tables only have Canada-level aggregates
309+
310+
### Observable Framework Issues
220311

221312
**Charts not rendering:**
222-
- Ensure browser has JavaScript enabled
223-
- Check console for errors
224-
- Verify `time_series` array has data points
313+
- Check browser console for JavaScript errors
314+
- Verify import statement is at top of first code block only
315+
- Ensure data array has values (not empty)
316+
- Check for syntax errors in Plot.plot() call
317+
318+
**CSS not loading (Safari):**
319+
- Run `npm run build` - the post-build script fixes paths
320+
- Clear browser cache
321+
- Check that paths start with `/thedaily/` not `./`
322+
323+
**Language switcher broken:**
324+
- Verify slug is added to `src/lang-map.js`
325+
- Check slug spelling matches folder name exactly
326+
- Rebuild with `npm run build`
327+
328+
**"Module not found" error:**
329+
- Run `npm install` to ensure dependencies installed
330+
- Check import path is `"npm:@observablehq/plot"` not a file path
331+
332+
### Article Content Issues
225333

226334
**Wrong date extracted:**
227-
- Check filename matches expected pattern
228-
- Verify `REF_DATE` in JSON output
335+
- Check REF_DATE in JSON output
336+
- Verify date format is YYYY-MM
337+
- Some tables have different date granularity
338+
339+
**Numbers don't match StatCan website:**
340+
- Check if data is seasonally adjusted vs not
341+
- Verify you're comparing same reference period
342+
- Some tables have revisions; refetch with `--refresh`
343+
344+
**French decimals showing period instead of comma:**
345+
- Use `.replace(".", ",")` on formatted numbers
346+
- Check CHART-STYLE-GUIDE.md for French formatting
347+
348+
### Build/Deploy Issues
349+
350+
**Build fails:**
351+
- Run `npm run clean` then `npm run build`
352+
- Check for syntax errors in markdown files
353+
- Verify all Observable code blocks are valid JS
354+
355+
**Changes not appearing on live site:**
356+
- Wait 1-2 minutes for Netlify deploy
357+
- Check GitHub Actions for deploy status
358+
- Hard refresh browser (Cmd+Shift+R)
359+
360+
**Tests failing:**
361+
- Run `npm test` to see specific failures
362+
- Check that fix-paths.js hasn't been modified incorrectly

0 commit comments

Comments
 (0)