Skip to content

Commit 306c0c9

Browse files
wwwadxAr9av
authored andcommitted
feat(skills): add opt-in Paper Deep-Dive template for academic papers
Academic PDFs (arXiv/conference) carry their substance in figures, equations, and results tables, but the generic page template flattens everything to a terse Key Ideas bullet list and the PDF reader drops the text layer's figures. ML/AI/LLM/VLM paper pages end up text-only and hard to use. Add an opt-in path, scoped to academic papers landing in references/, that leaves the terse default untouched for every other source: - llm-wiki: new Paper Deep-Dive Template (TL;DR callout, Problem, Method/Architecture with an embedded real figure + a Mermaid schematic, Key Equations in LaTeX, Results table, Limitations). - wiki-ingest: an Academic papers sub-step that re-reads figure/equation pages with vision, embeds the paper's real architecture figure when it can be extracted (PyMuPDF: get_image_info -> extract_image, or render the bbox for vector figures) and always draws a Mermaid schematic; Step 5 branches to the new template. - ingest-prompts: a Paper Extraction Frame reading checklist. Mermaid stays the zero-dependency default; real-figure extraction is an opt-in enhancement. No behavior change for non-paper sources.
1 parent 0f8c63e commit 306c0c9

3 files changed

Lines changed: 95 additions & 3 deletions

File tree

.skills/llm-wiki/SKILL.md

Lines changed: 66 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ Organize pages into these default categories (customizable in `.env`):
4848
| `concepts/` | Ideas, theories, mental models | `concepts/transformer-architecture.md` |
4949
| `entities/` | People, orgs, tools, projects | `entities/andrej-karpathy.md` |
5050
| `skills/` | How-to knowledge, procedures | `skills/fine-tuning-llms.md` |
51-
| `references/` | Summaries of specific sources | `references/attention-is-all-you-need.md` |
51+
| `references/` | Summaries of specific sources; academic papers use the Paper Deep-Dive Template (below) | `references/attention-is-all-you-need.md` |
5252
| `synthesis/` | Cross-cutting analysis across sources | `synthesis/scaling-laws-debate.md` |
5353
| `journal/` | Timestamped observations, session logs | `journal/2024-03-15.md` |
5454

@@ -201,6 +201,71 @@ Things that are unresolved or need more sources.
201201
- [[references/attention-is-all-you-need]] — Original paper
202202
```
203203

204+
## Paper Deep-Dive Template
205+
206+
The generic template suits most sources. **Academic papers are the exception.** For ML/AI/LLM/VLM (and similar) papers landing in `references/`, the substance lives in the architecture, the equations, and the results table — exactly what a terse "Key Ideas" list flattens away. For these, use the richer template below. This is the one place where *"compile, don't retrieve"* yields to a thorough, self-contained walkthrough a reader could study instead of the paper.
207+
208+
Obsidian renders the needed primitives natively, so no extra tooling is required: Mermaid fenced diagrams, `$$…$$` LaTeX (MathJax), markdown tables, and `![[image]]` / `![[paper.pdf#page=N]]` embeds.
209+
210+
Use this template only when the source is an academic paper (arXiv/conference) with load-bearing figures or equations. Everything else uses the generic Page Template above. Frontmatter, provenance markers, confidence, lifecycle, and `relationships:` are unchanged — only the body sections differ.
211+
212+
````markdown
213+
---
214+
# ...required frontmatter, same as the generic template; category: references...
215+
---
216+
217+
# Paper Title
218+
219+
> [!tldr] One sentence: what's new, plus the headline result.
220+
221+
## Problem & Motivation
222+
223+
What's broken or missing that this paper addresses.
224+
225+
## Method / Architecture
226+
227+
Prose walkthrough. Embed the paper's real architecture figure as the primary
228+
visual (see *Academic papers* in `wiki-ingest` for the PyMuPDF extraction recipe).
229+
Fall back to a Mermaid flowchart only when no figure can be extracted.
230+
231+
![[attachments/<slug>-fig1.png]]
232+
*Figure N (Author Year): one-line caption.*
233+
234+
## Key Equations
235+
236+
The 1–3 core equations as display math, not backtick code:
237+
238+
$$ \mathcal{L} = \mathbb{E}_{x}\!\left[-\log p_\theta(y \mid z)\right] $$
239+
240+
## Results
241+
242+
Headline numbers as a table, not a comma-separated blob — and embed a key
243+
results/motivating figure (scaling plot, benchmark chart, capability collage)
244+
when the paper has one:
245+
246+
| Method | Benchmark | Metric | Cost |
247+
|---|---|---|---|
248+
| Baseline ||||
249+
| **This paper** ||||
250+
251+
![[attachments/<slug>-resultsN.png]]
252+
*Figure N (Author Year): one-line caption.*
253+
254+
## Limitations
255+
256+
What the paper concedes or sidesteps. Mark reading-between-the-lines as ^[inferred].
257+
258+
## Related
259+
260+
Typed `[[wikilinks]]` to neighbouring work.
261+
262+
## Sources
263+
264+
- Clickable canonical link, e.g. <https://arxiv.org/abs/XXXX.XXXXX>
265+
````
266+
267+
A Mermaid diagram reconstructed from the paper's prose is a synthesis, not a transcription — treat it as `^[inferred]` when the interpretation is non-trivial.
268+
204269
## Provenance Markers
205270

206271
Every claim on a wiki page has one of three provenance states. Mark them inline so the reader (and future ingest passes) can tell signal from synthesis.

.skills/wiki-ingest/SKILL.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ In raw mode, each file in `OBSIDIAN_VAULT_PATH/_raw/` (or `OBSIDIAN_RAW_DIR`) is
8585
Read the source(s) the user wants to ingest. In append mode, skip files the manifest says are already ingested and unchanged. Supported formats:
8686
- Markdown (`.md`) — read directly
8787
- Text (`.txt`) — read directly
88-
- PDF (`.pdf`) — use the Read tool with page ranges
88+
- PDF (`.pdf`) — use the Read tool with page ranges. For **academic papers** (arXiv/conference), see *Academic papers* below — re-read figure- and equation-dense pages with vision so the architecture diagram, key equations, and results tables aren't lost.
8989
- Web clippings — markdown files from Obsidian Web Clipper
9090
- **Structured data** (`.json`, `.jsonl`, `.csv`, `.tsv`, `.html`) — parse the structure first, then distill the knowledge it carries. See *Unstructured & conversational sources* below.
9191
- **Chat / conversation exports** — ChatGPT `conversations.json`, Slack/Discord channel JSON, timestamped chat logs, meeting transcripts. See *Unstructured & conversational sources* below.
@@ -130,6 +130,21 @@ Vision is interpretive by nature, so image-derived pages will skew heavily towar
130130

131131
For PDFs that are mostly images (scanned docs, slide decks exported to PDF), use `Read pages: "N"` to pull specific pages and treat each page as an image source.
132132

133+
### Academic papers
134+
135+
Research papers (arXiv/conference PDFs) carry their substance in figures, equations, and results tables — exactly what plain text extraction drops. A normal arXiv PDF has a text layer, so the image branch above never fires and its diagrams are skipped by default. When a source is an academic paper, override that:
136+
137+
1. **Read the text layer** for the narrative (problem, method, claims), then **re-read the figure- and equation-dense pages with vision** (`Read pages: "N"`) — the architecture/method figure (often Figure 1) and the main results table rarely live in the text layer.
138+
2. **Capture the method visually — prefer the paper's real figures.**
139+
- **Embed the paper's own architecture/method figure as the primary visual.** Most arXiv figures are a single embedded raster. With PyMuPDF (`fitz`): use `page.get_image_info(xrefs=True)` to find the figure's `xref` and bbox — it is usually the wide image sitting just above its caption (locate the caption with `page.search_for("Figure N")`) — then `img = doc.extract_image(xref)` and save `img["image"]` to `attachments/<slug>-figN.<ext>` using the native `img["ext"]` (it may be JPEG, not PNG — don't hardcode the extension; downscale oversized figures, e.g. `sips -Z 1800 <file>`). If the figure is vector rather than raster (`extract_image` returns nothing and `page.get_drawings()` is non-empty), render the bbox region instead: `page.get_pixmap(clip=rect, matrix=fitz.Matrix(4, 4))` — compute `rect` by unioning `get_drawings()` rects (drawings-only; text blocks pull in body text) within one column above the caption, and in multi-column papers bound the window below the previous element so adjacent tables/text aren't caught; verify the render and re-crop if needed. Embed with `![[<slug>-figN.<ext>]]` plus an italic caption.
140+
- **Also embed a key results / motivating figure** when the paper has one — a scaling plot, a benchmark chart, or a capability collage — in the Results section alongside the table.
141+
- **Mermaid is the dependency-free fallback.** If PyMuPDF/poppler isn't available or a figure can't be extracted, draw the architecture as a Mermaid diagram instead — Obsidian renders Mermaid fenced code blocks natively with no dependencies. `![[<source>.pdf#page=N]]` (the whole source page) is another no-extract option.
142+
3. **Keep the math as math.** Set the 1–3 core equations as `$$…$$` display LaTeX, not backtick code.
143+
4. **Tabulate results.** Render headline benchmark numbers as a markdown table, not a comma-separated blob.
144+
5. **Write the page with the Paper Deep-Dive Template** (`llm-wiki/SKILL.md`) into `references/`, in addition to the distilled concept/entity cross-links. This is the deliberate exception to "aim for 10–15 small pages" (Step 4) — a paper earns one rich, self-contained page.
145+
146+
See the *Paper Extraction Frame* in `references/ingest-prompts.md` for the reading checklist.
147+
133148
### Step 1b: QMD Source Discovery (optional — requires `QMD_PAPERS_COLLECTION` in `.env`)
134149

135150
**GUARD: If `$QMD_PAPERS_COLLECTION` is empty or unset, skip this entire step and proceed to Step 2.**
@@ -264,7 +279,7 @@ For each page in your plan:
264279
**If `WIKI_STAGED_WRITES` is not set or is `false` (default):**
265280

266281
**If creating a new page:**
267-
- Use the page template from the llm-wiki skill (frontmatter + sections)
282+
- Use the page template from the llm-wiki skill (frontmatter + sections). **For academic papers landing in `references/`, use the Paper Deep-Dive Template** from `llm-wiki/SKILL.md` instead of the generic one (see *Academic papers* in Step 1).
268283
- Place in the correct category directory
269284
- Add `[[wikilinks]]` to at least 2-3 existing pages
270285
- Include the source in the `sources` frontmatter field. In raw mode: derive from `capture_source` + `sources` frontmatter of the `_raw/` file — never use the `_raw/` path itself (see Raw Mode section)

.skills/wiki-ingest/references/ingest-prompts.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,18 @@ When reading a source document, ask yourself:
2121
5. **How does this connect to what the wiki already knows?**
2222
This is the most important question. The value of the wiki compounds through connections.
2323

24+
## Paper Extraction Frame
25+
26+
For academic papers (ML/AI/LLM/VLM and similar), the generic frame above misses what makes a paper legible. Add these questions:
27+
28+
1. **What problem does it solve, and what's new?** The one-sentence thesis + the single most important result.
29+
2. **What is the method?** Which figure shows the architecture/pipeline? Sketch it as a Mermaid flowchart — capture the data flow, not just the component names.
30+
3. **What are the core equations?** The 1–3 that define the mechanism — keep them as math (`$$…$$`), not prose.
31+
4. **What's the experimental setup and the headline numbers?** Datasets, baselines, and the metric table the paper is judged on.
32+
5. **What are the ablations and limitations?** What did they vary, and what does the method *not* do?
33+
34+
These map onto the Paper Deep-Dive Template in `llm-wiki/SKILL.md`. The goal is a page a reader could study instead of the PDF — figures, equations, and results included.
35+
2436
## Synthesis Frame
2537

2638
When a new source covers ground that existing pages already cover:

0 commit comments

Comments
 (0)