You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(skills): add opt-in Paper Deep-Dive template for academic papers
Academic PDFs (arXiv/conference) carry their substance in figures,
equations, and results tables, but the generic page template flattens
everything to a terse Key Ideas bullet list and the PDF reader drops
the text layer's figures. ML/AI/LLM/VLM paper pages end up text-only
and hard to use.
Add an opt-in path, scoped to academic papers landing in references/,
that leaves the terse default untouched for every other source:
- llm-wiki: new Paper Deep-Dive Template (TL;DR callout, Problem,
Method/Architecture with an embedded real figure + a Mermaid
schematic, Key Equations in LaTeX, Results table, Limitations).
- wiki-ingest: an Academic papers sub-step that re-reads figure/equation
pages with vision, embeds the paper's real architecture figure when it
can be extracted (PyMuPDF: get_image_info -> extract_image, or render
the bbox for vector figures) and always draws a Mermaid schematic;
Step 5 branches to the new template.
- ingest-prompts: a Paper Extraction Frame reading checklist.
Mermaid stays the zero-dependency default; real-figure extraction is an
opt-in enhancement. No behavior change for non-paper sources.
@@ -201,6 +201,71 @@ Things that are unresolved or need more sources.
201
201
-[[references/attention-is-all-you-need]] — Original paper
202
202
```
203
203
204
+
## Paper Deep-Dive Template
205
+
206
+
The generic template suits most sources. **Academic papers are the exception.** For ML/AI/LLM/VLM (and similar) papers landing in `references/`, the substance lives in the architecture, the equations, and the results table — exactly what a terse "Key Ideas" list flattens away. For these, use the richer template below. This is the one place where *"compile, don't retrieve"* yields to a thorough, self-contained walkthrough a reader could study instead of the paper.
207
+
208
+
Obsidian renders the needed primitives natively, so no extra tooling is required: Mermaid fenced diagrams, `$$…$$` LaTeX (MathJax), markdown tables, and `![[image]]` / `![[paper.pdf#page=N]]` embeds.
209
+
210
+
Use this template only when the source is an academic paper (arXiv/conference) with load-bearing figures or equations. Everything else uses the generic Page Template above. Frontmatter, provenance markers, confidence, lifecycle, and `relationships:` are unchanged — only the body sections differ.
211
+
212
+
````markdown
213
+
---
214
+
# ...required frontmatter, same as the generic template; category: references...
215
+
---
216
+
217
+
# Paper Title
218
+
219
+
> [!tldr] One sentence: what's new, plus the headline result.
220
+
221
+
## Problem & Motivation
222
+
223
+
What's broken or missing that this paper addresses.
224
+
225
+
## Method / Architecture
226
+
227
+
Prose walkthrough. Embed the paper's real architecture figure as the primary
228
+
visual (see *Academic papers* in `wiki-ingest` for the PyMuPDF extraction recipe).
229
+
Fall back to a Mermaid flowchart only when no figure can be extracted.
230
+
231
+
![[attachments/<slug>-fig1.png]]
232
+
*Figure N (Author Year): one-line caption.*
233
+
234
+
## Key Equations
235
+
236
+
The 1–3 core equations as display math, not backtick code:
What the paper concedes or sidesteps. Mark reading-between-the-lines as ^[inferred].
257
+
258
+
## Related
259
+
260
+
Typed `[[wikilinks]]` to neighbouring work.
261
+
262
+
## Sources
263
+
264
+
- Clickable canonical link, e.g. <https://arxiv.org/abs/XXXX.XXXXX>
265
+
````
266
+
267
+
A Mermaid diagram reconstructed from the paper's prose is a synthesis, not a transcription — treat it as `^[inferred]` when the interpretation is non-trivial.
268
+
204
269
## Provenance Markers
205
270
206
271
Every claim on a wiki page has one of three provenance states. Mark them inline so the reader (and future ingest passes) can tell signal from synthesis.
Copy file name to clipboardExpand all lines: .skills/wiki-ingest/SKILL.md
+17-2Lines changed: 17 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -85,7 +85,7 @@ In raw mode, each file in `OBSIDIAN_VAULT_PATH/_raw/` (or `OBSIDIAN_RAW_DIR`) is
85
85
Read the source(s) the user wants to ingest. In append mode, skip files the manifest says are already ingested and unchanged. Supported formats:
86
86
- Markdown (`.md`) — read directly
87
87
- Text (`.txt`) — read directly
88
-
- PDF (`.pdf`) — use the Read tool with page ranges
88
+
- PDF (`.pdf`) — use the Read tool with page ranges. For **academic papers** (arXiv/conference), see *Academic papers* below — re-read figure- and equation-dense pages with vision so the architecture diagram, key equations, and results tables aren't lost.
89
89
- Web clippings — markdown files from Obsidian Web Clipper
90
90
-**Structured data** (`.json`, `.jsonl`, `.csv`, `.tsv`, `.html`) — parse the structure first, then distill the knowledge it carries. See *Unstructured & conversational sources* below.
@@ -130,6 +130,21 @@ Vision is interpretive by nature, so image-derived pages will skew heavily towar
130
130
131
131
For PDFs that are mostly images (scanned docs, slide decks exported to PDF), use `Read pages: "N"` to pull specific pages and treat each page as an image source.
132
132
133
+
### Academic papers
134
+
135
+
Research papers (arXiv/conference PDFs) carry their substance in figures, equations, and results tables — exactly what plain text extraction drops. A normal arXiv PDF has a text layer, so the image branch above never fires and its diagrams are skipped by default. When a source is an academic paper, override that:
136
+
137
+
1.**Read the text layer** for the narrative (problem, method, claims), then **re-read the figure- and equation-dense pages with vision** (`Read pages: "N"`) — the architecture/method figure (often Figure 1) and the main results table rarely live in the text layer.
138
+
2.**Capture the method visually — prefer the paper's real figures.**
139
+
- **Embed the paper's own architecture/method figure as the primary visual.** Most arXiv figures are a single embedded raster. With PyMuPDF (`fitz`): use `page.get_image_info(xrefs=True)` to find the figure's `xref` and bbox — it is usually the wide image sitting just above its caption (locate the caption with `page.search_for("Figure N")`) — then `img = doc.extract_image(xref)` and save `img["image"]` to `attachments/<slug>-figN.<ext>` using the native `img["ext"]` (it may be JPEG, not PNG — don't hardcode the extension; downscale oversized figures, e.g. `sips -Z 1800 <file>`). If the figure is vector rather than raster (`extract_image` returns nothing and `page.get_drawings()` is non-empty), render the bbox region instead: `page.get_pixmap(clip=rect, matrix=fitz.Matrix(4, 4))` — compute `rect` by unioning `get_drawings()` rects (drawings-only; text blocks pull in body text) within one column above the caption, and in multi-column papers bound the window below the previous element so adjacent tables/text aren't caught; verify the render and re-crop if needed. Embed with `![[<slug>-figN.<ext>]]` plus an italic caption.
140
+
-**Also embed a key results / motivating figure** when the paper has one — a scaling plot, a benchmark chart, or a capability collage — in the Results section alongside the table.
141
+
-**Mermaid is the dependency-free fallback.** If PyMuPDF/poppler isn't available or a figure can't be extracted, draw the architecture as a Mermaid diagram instead — Obsidian renders Mermaid fenced code blocks natively with no dependencies. `![[<source>.pdf#page=N]]` (the whole source page) is another no-extract option.
142
+
3.**Keep the math as math.** Set the 1–3 core equations as `$$…$$` display LaTeX, not backtick code.
143
+
4.**Tabulate results.** Render headline benchmark numbers as a markdown table, not a comma-separated blob.
144
+
5.**Write the page with the Paper Deep-Dive Template** (`llm-wiki/SKILL.md`) into `references/`, in addition to the distilled concept/entity cross-links. This is the deliberate exception to "aim for 10–15 small pages" (Step 4) — a paper earns one rich, self-contained page.
145
+
146
+
See the *Paper Extraction Frame* in `references/ingest-prompts.md` for the reading checklist.
**GUARD: If `$QMD_PAPERS_COLLECTION` is empty or unset, skip this entire step and proceed to Step 2.**
@@ -264,7 +279,7 @@ For each page in your plan:
264
279
**If `WIKI_STAGED_WRITES` is not set or is `false` (default):**
265
280
266
281
**If creating a new page:**
267
-
- Use the page template from the llm-wiki skill (frontmatter + sections)
282
+
- Use the page template from the llm-wiki skill (frontmatter + sections). **For academic papers landing in `references/`, use the Paper Deep-Dive Template** from `llm-wiki/SKILL.md` instead of the generic one (see *Academic papers* in Step 1).
268
283
- Place in the correct category directory
269
284
- Add `[[wikilinks]]` to at least 2-3 existing pages
270
285
- Include the source in the `sources` frontmatter field. In raw mode: derive from `capture_source` + `sources` frontmatter of the `_raw/` file — never use the `_raw/` path itself (see Raw Mode section)
Copy file name to clipboardExpand all lines: .skills/wiki-ingest/references/ingest-prompts.md
+12Lines changed: 12 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,6 +21,18 @@ When reading a source document, ask yourself:
21
21
5.**How does this connect to what the wiki already knows?**
22
22
This is the most important question. The value of the wiki compounds through connections.
23
23
24
+
## Paper Extraction Frame
25
+
26
+
For academic papers (ML/AI/LLM/VLM and similar), the generic frame above misses what makes a paper legible. Add these questions:
27
+
28
+
1.**What problem does it solve, and what's new?** The one-sentence thesis + the single most important result.
29
+
2.**What is the method?** Which figure shows the architecture/pipeline? Sketch it as a Mermaid flowchart — capture the data flow, not just the component names.
30
+
3.**What are the core equations?** The 1–3 that define the mechanism — keep them as math (`$$…$$`), not prose.
31
+
4.**What's the experimental setup and the headline numbers?** Datasets, baselines, and the metric table the paper is judged on.
32
+
5.**What are the ablations and limitations?** What did they vary, and what does the method *not* do?
33
+
34
+
These map onto the Paper Deep-Dive Template in `llm-wiki/SKILL.md`. The goal is a page a reader could study instead of the PDF — figures, equations, and results included.
35
+
24
36
## Synthesis Frame
25
37
26
38
When a new source covers ground that existing pages already cover:
0 commit comments