Commit 9556690
authored
feat: multimodal documentation, examples, and changelog for v0.4.1 (#464)
* docs(08): capture phase context
* docs(state): record phase 8 context session
* docs(08): research phase domain
* docs(phase-8): add validation strategy
* docs(08): create phase plan
* fix(08): revise plans based on checker feedback
* docs(08-01): update Gemini and VoyageAI sections in embeddings.md
- Add VoyageAI option functions list (11 options)
- Add VoyageAI Multimodal (Content API) subsection with image and video examples
- Update Gemini default model to gemini-embedding-2-preview
- Add WithMaxFileSize to Gemini option list
- Add Gemini Multimodal (Content API) subsection with image and video examples
- Both multimodal subsections cross-reference embeddings/multimodal.md
* docs(08): capture phase context
* docs(state): record phase 8 context session
* docs(08): research phase domain
* docs(phase-8): add validation strategy
* docs(08): create phase plan
* fix(08): revise plans based on checker feedback
* docs(08-02): update README with multimodal capability mentions and example rows
- Add Content API feature bullet in additional support features
- Update Gemini line with multimodal modalities (text, images, audio, video, PDF)
- Update VoyageAI line with multimodal modalities (text, images, video)
- Add gemini_multimodal and voyage_multimodal example table rows
* feat(08-01): add runnable multimodal example programs
- Add examples/v2/gemini_multimodal/main.go with EmbedContent and EmbedContents
- Add examples/v2/voyage_multimodal/main.go with EmbedContent and EmbedContents
- Both examples demonstrate image and video modalities via Content API
- Follow established example pattern with log.Fatalf error handling
* docs(08-02): create CHANGELOG.md and correct ROADMAP.md naming
- Create CHANGELOG.md with v0.4.1 release notes in Keep a Changelog format
- Remove Nemotron parenthetical from Phase 7 description in ROADMAP.md
- Mark Phase 7 as complete in ROADMAP.md
- Reword Phase 8 success criteria to remove stale Nemotron reference
* docs(08-01): complete provider documentation plan
- Add 08-01-SUMMARY.md with execution results
- Update STATE.md with plan progress and decisions
- Update ROADMAP.md with plan completion status
* docs(08-02): complete README/CHANGELOG/ROADMAP updates plan
- Add 08-02-SUMMARY.md with execution results
- Update STATE.md with progress, metrics, and session info
- Update ROADMAP.md Phase 7 completion and naming correction
* fix(08): use Embedding.Len() instead of non-existent ArrayOfFloat32 field
* refactor(08): share testdata across providers and add VoyageAI integration tests
- Move test assets from pkg/embeddings/gemini/testdata/ to shared
pkg/embeddings/testdata/ for reuse by all multimodal providers
- Add VoyageAI Content API integration tests mirroring Gemini's coverage
(text, image, video, mixed parts, batch, intent)
- Align example programs to use identical URLs and descriptions
- Align VoyageAI doc section to match Gemini's content examples
* docs(08): add phase verification report
* docs(phase-08): complete phase execution
* docs(phase-08): evolve PROJECT.md after phase completion
* docs(08): rewrite multimodal.md as concept-first guide
Replace reference-style docs with a guide that explains Content, Part,
BinarySource, and Intent in plain terms before showing code. Add ASCII
concept diagram, intent decision table, common recipes section, and
provider support matrix. Use Gemini as primary example provider.
* docs(09): add Phase 9 — convenience constructors and documentation polish
* fix(08): use absolute paths in integration tests to avoid path traversal rejection
The shared testdata at ../testdata/ triggers the containsDotDot path
traversal check in resolveBytes. Using filepath.Abs resolves the
relative path to an absolute one before it reaches the security check.
* fix(08): replace panic with require.NoError in testdataPath helper
Pass *testing.T to testdataPath so filepath.Abs errors use
require.NoError instead of panic, consistent with project guidelines.
* fix(08): skip VoyageAI video integration test — asset exceeds context window
The 5.3MB test video base64-encodes to ~7MB which exceeds VoyageAI's
32K token context window. Video conversion logic is covered by unit
tests with mock server.
* fix(08): add small video asset for VoyageAI integration test
VoyageAI tokenizes video at 1120 pixels/token with a 32K token limit.
The original 1280x720 8s video (~197K tokens) far exceeds this. Create
a 480x480 2s 15fps copy (~6K tokens) that fits within the limit.
Original asset preserved for Gemini which handles large files natively.
* fix(08): address PR review — remove placeholder URLs, add ef.Close()
- Remove fake example.com/lecture.mp4 video URL from batch examples
(both Gemini and VoyageAI) — only use real Wikimedia image URLs
- Add explicit ef.Close() to Gemini example for proper resource cleanup
* fix(08): use local testdata assets in examples instead of external URLs
Examples should be fully self-contained and not rely on external
resources. Replace Wikimedia image URLs and example.com placeholders
with local testdata files (lioness.png, the_pounce.mp4). VoyageAI
example uses the_pounce_small.mp4 to fit within 32K token limit.
* fix(08): address PR review — run() pattern, defer Close(), add webp/gif MIME support
- Refactor examples to use main() → run() error pattern for proper defer cleanup
- Add .webp and .gif to Gemini extToMIME map (API supports them, map was missing)
- Add run-from-repo-root comment to both multimodal examples
* fix(08): resolve CWD dependency in examples, use placeholder paths in docs
Examples now locate repo root via go.mod lookup so they run from any
directory. Doc snippets use generic /path/to/ placeholders instead of
internal testdata paths.1 parent 4532be7 commit 9556690
28 files changed
Lines changed: 2539 additions & 93 deletions
File tree
- .planning
- phases
- 08-document-gemini-and-nemotron-multimodal-embedding-functions
- 09-convenience-constructors-and-documentation-polish
- docs/docs
- embeddings
- examples/v2
- gemini_multimodal
- voyage_multimodal
- pkg/embeddings
- gemini
- testdata
- voyage
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
26 | | - | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
27 | 31 | | |
28 | 32 | | |
29 | 33 | | |
30 | 34 | | |
31 | | - | |
| 35 | + | |
32 | 36 | | |
33 | 37 | | |
34 | 38 | | |
| |||
57 | 61 | | |
58 | 62 | | |
59 | 63 | | |
60 | | - | |
| 64 | + | |
61 | 65 | | |
62 | 66 | | |
63 | | - | |
| 67 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
24 | 26 | | |
25 | 27 | | |
26 | 28 | | |
| |||
132 | 134 | | |
133 | 135 | | |
134 | 136 | | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
135 | 155 | | |
136 | 156 | | |
137 | 157 | | |
| |||
143 | 163 | | |
144 | 164 | | |
145 | 165 | | |
| 166 | + | |
146 | 167 | | |
147 | | - | |
| 168 | + | |
148 | 169 | | |
149 | | - | |
| 170 | + | |
150 | 171 | | |
151 | | - | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
152 | 178 | | |
153 | 179 | | |
154 | 180 | | |
155 | | - | |
| 181 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
6 | | - | |
7 | | - | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
11 | | - | |
12 | | - | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
| 66 | + | |
65 | 67 | | |
66 | 68 | | |
67 | 69 | | |
| |||
103 | 105 | | |
104 | 106 | | |
105 | 107 | | |
| 108 | + | |
| 109 | + | |
106 | 110 | | |
107 | 111 | | |
108 | 112 | | |
| |||
111 | 115 | | |
112 | 116 | | |
113 | 117 | | |
| 118 | + | |
114 | 119 | | |
115 | 120 | | |
116 | 121 | | |
| |||
136 | 141 | | |
137 | 142 | | |
138 | 143 | | |
139 | | - | |
140 | | - | |
| 144 | + | |
| 145 | + | |
141 | 146 | | |
0 commit comments