Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/skills-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18'
node-version: '20'

- name: Run skills add command
run: npx skills add antvis/chart-visualization-skills
37 changes: 14 additions & 23 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,24 +57,19 @@ Skills are the **single source of truth** for chart generation knowledge.

### 2. CLI Tool (`src/`)

The build script (`src/scripts/build.ts`) parses all skill markdown files and generates JSON index files (`src/index/*.index.json`). Each index stores two things: the `skills[]` array (reference docs) and `info` (SKILL.md metadata including `constraintsContent` — the core constraints section up to `<!-- CONSTRAINTS:END -->`).
The build script (`src/scripts/build.ts`) reads all skill markdown files directly from `src/content/` and generates zvec vector collections (`src/index/*.zvec/`) with full FTS indexes on title, description, tags, content, use_cases, and anti_patterns. All metadata (including frontmatter `id`, `category`, `tags`, etc.) is stored in zvec fields via `@antv/context`, so no intermediate JSON index is needed.

A second build step (`src/scripts/build-zvec.ts`) embeds all skill texts and writes them into zvec vector collections (`src/index/*.zvec/`) with full FTS indexes on title, description, tags, content, use_cases, and anti_patterns.
The CLI (`antv` command) provides one command:

The CLI (`antv` command) provides four commands:

- `antv retrieve <query>` - zvec hybrid search (FTS + vector + RRF fusion); `--content` returns reference doc markdown and auto-prepends the core constraints block as the first result
- `antv get <id>` - Get a single skill by exact ID
- `antv list` - List/filter available skills
- `antv info <library>` - Show library core constraints (SKILL.md Section 1-2)
- `antv retrieve <query>` - zvec hybrid search (FTS + vector + RRF fusion); `--content` returns reference doc markdown; constraints docs are indexed as regular skill documents and appear naturally in search results

The retrieval engine uses **zvec** (in-process vector database) with two strategies:
- **`hybrid`** (default): zvec native multiQuery — FTS (jieba tokenizer, raw text) + Vector (HNSW ANN, 512d) + RRF fusion
- **`vector`**: Pure ANN vector similarity search

Embedding is handled by `SimpleEmbedder` (tokenizer + multi-hash, 512d, synchronous). No model download required.
Embedding is handled by `@antv/context`'s built-in embedder (tokenizer + multi-hash, 512d, synchronous).

Public API (`src/api.ts`) exports `retrieve()`, `info()`, `getSkillById()`, `libraries()`, and `listSkills()` for programmatic use.
Public API (`src/api.ts`) exports `retrieve()` and `libraries()` for programmatic use.

### HTTP Server (`http-server/`)

Expand All @@ -86,10 +81,7 @@ cd http-server && npm run dev # starts on http://localhost:3100 (configurable

| Endpoint | Method | Description |
|---|---|---|
| `/retrieve` | POST | Retrieve skills by query (hybrid/vector search). Body: `{query, library, topK, content, includeConstraints, strategy, maxTokens, progressiveLevel}` |
| `/info` | POST | Get library core constraints. Body: `{library}` |
| `/get` | POST | Get single skill by exact ID. Body: `{id, library}` |
| `/list` | POST | List/filter skills. Body: `{library, category, tags}` |
| `/retrieve` | POST | Retrieve skills by query (hybrid/vector search). Body: `{query, library, topK, content, strategy, maxTokens, progressiveLevel}` |
| `/libraries` | GET | List available library names |

### 3. Evaluation (`eval/`)
Expand Down Expand Up @@ -136,11 +128,11 @@ Two retrieval modes:
├── src/ # Core library: CLI, API, zvec retriever, build scripts
│ ├── index.ts # CLI entry point (Commander.js)
│ ├── api.ts # Public Node.js API
│ ├── commands/ # CLI commands (retrieve, list, info, get)
│ ├── core/ # Types, zvec retriever, embedder
│ └── retrieval/ # zvec-store, embedder
│ ├── scripts/ # Build scripts (markdown → JSON index + zvec index)
│ └── index/ # Generated JSON + zvec index files
│ ├── commands/ # CLI commands (retrieve)
│ ├── core/ # Types, zvec retriever, synonyms, token-budget
── scripts/ # Build script (build.ts only)
│ ├── content/ # Skill markdown source files (with frontmatter)
│ └── index/ # Generated zvec index files only
├── http-server/ # Standalone HTTP API deployment
├── skills/ # Skill definitions (markdown + YAML frontmatter)
├── eval/ # Evaluation framework and test datasets
Expand All @@ -153,12 +145,11 @@ Two retrieval modes:
## Key Commands

```bash
# Build: parse skills → generate JSON index → build zvec index → compile TS
# Build: build zvec index → compile TS → copy index
pnpm build

# Build individual steps
pnpm build:index:json # JSON index only
pnpm build:index:zvec # zvec vector index only
# Build individual step
pnpm build:index # zvec vector index only

# Test
pnpm test
Expand Down
61 changes: 8 additions & 53 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,32 +90,17 @@ We also provide a CLI tool named `antv` for easy usage in your terminal, Install
npm install -g @antv/chart-visualization-skills
```

**Retrieve or list skills by query**:
**Retrieve skills by query**:

```bash
# Retrieve skills by query (metadata only)
antv retrieve "bar chart" --library g2 --topk 10

# Retrieve skills with full markdown content (core constraints auto-prepended)
# Retrieve skills with full markdown content
antv retrieve "bar chart" --library g2 --content

# Retrieve skills and output as JSON
antv retrieve "bar chart" --library g2 --output json

# Get a skill by its exact ID
antv get g2-mark-interval-basic --library g2

# List all available skills
antv list --library g2 --category core

# List skills and output as JSON
antv list --output json

# Show skill info (core constraints from SKILL.md)
antv info --library g2

# Show skill info as JSON
antv info --library g2 --output json
```

**Usage for the command**:
Expand All @@ -131,35 +116,29 @@ Options:

Commands:
retrieve [options] <query> Search for skills matching a query
get [options] <id> Get a skill by its exact ID
list [options] List all available skills
info [options] Show skill info from SKILL.md
help [command] display help for command

Options for retrieve:
--library <lib> Filter by library (e.g. g2, g6, x6)
--topk <n> Number of results to return (default: 7)
--strategy <s> Retrieval strategy: hybrid | vector (default: hybrid)
--content Include markdown content body in results; core constraints (SKILL.md Section 1-2) are always prepended as the first result
--content Include markdown content body in results
--output <format> Output format: json | text (default: "text")
```

> Note: `--content` always prepends the library's core constraints (Section 1 & 2 of SKILL.md, up to the `<!-- CONSTRAINTS:END -->` marker) as the first result, ensuring the model receives essential rules alongside the reference documents.
> Note: Constraints docs (category `__constraints__`) are indexed as regular skill documents and appear naturally in search results when the query matches their content.

### API Usage

```typescript
import { retrieve } from '@antv/chart-visualization-skills';

// With full markdown content and constraints (defaults)
// With full markdown content (defaults)
const skills = retrieve('bar chart', { library: 'g2', topK: 5 });

// Metadata only (no content body or constraints)
// Metadata only (no content body)
const skills = retrieve('bar chart', { library: 'g2', topK: 5, content: false });

// Explicit constraints control
const skills = retrieve('bar chart', { library: 'g2', content: true, includeConstraints: false });

// With token budget — content trimmed to fit 4000 tokens, summary + code only
const skills = retrieve('bar chart', {
library: 'g2',
Expand All @@ -169,13 +148,12 @@ const skills = retrieve('bar chart', {
```

```typescript
retrieve(query: string, options?: RetrieveOptions): Promise<Skill[]>
retrieve(query: string, options?: RetrieveOptions): Promise<Doc[]>

interface RetrieveOptions {
library?: string; // Library filter, e.g. 'g2' or 'g6'
topK?: number; // Number of results (default: 7)
content?: boolean; // Include markdown content body (default: true)
includeConstraints?: boolean; // Prepend constraints as first result (default: same as content)
strategy?: 'hybrid' | 'vector'; // Retrieval strategy (default: 'hybrid')
maxTokens?: number; // Token budget — content trimmed to fit when set
progressiveLevel?: 0 | 1 | 2; // 0=full, 1=summary+code, 2=summary (default: 1)
Expand All @@ -187,7 +165,6 @@ interface RetrieveOptions {
| `library` | `string` | all | Library filter (`g2` / `g6` / `x6`) |
| `topK` | `number` | `7` | Number of results |
| `content` | `boolean` | `true` | Include markdown content body |
| `includeConstraints` | `boolean` | same as `content` | Prepend constraints as first result |
| `strategy` | `'hybrid' \| 'vector'` | `'hybrid'` | Retrieval strategy |
| `maxTokens` | `number` | — | Token budget; content trimmed to fit when set |
| `progressiveLevel` | `0 \| 1 \| 2` | `1` | 0=full, 1=summary+code, 2=summary only |
Expand All @@ -196,31 +173,9 @@ interface RetrieveOptions {
> - Default retrieval uses **hybrid** strategy: zvec native FTS (jieba) + Vector (HNSW ANN) + RRF fusion.
> - `strategy: 'vector'` uses pure ANN vector similarity search.
> - `content: true` returns markdown content body (frontmatter metadata is excluded).
> - When `includeConstraints` is true, the core constraints block is injected as the first element (id prefixed with `__info__`).
> - Constraints docs (category `__constraints__`) are indexed as regular skill documents and appear naturally in search results.
> - When `maxTokens` is set, skill content is formatted and trimmed to fit the budget according to `progressiveLevel`.

```typescript
import { info } from '@antv/chart-visualization-skills';

const skillInfo = info('g2');
// => { name: 'antv-g2-chart', description: '...', content: '...', constraintsContent: '...' }
```

```typescript
info(library?: string): SkillInfo | undefined

interface SkillInfo {
name: string;
description: string;
content: string; // Full SKILL.md body (after frontmatter)
constraintsContent: string; // SKILL.md body up to <!-- CONSTRAINTS:END --> marker; injected by retrieve when includeConstraints: true
}
```

| Parameter | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `library` | `string` | `'g2'` | Library to get info for (`g2` or `g6`) |

## License

MIT License - see the [LICENSE](LICENSE) file for details.
120 changes: 0 additions & 120 deletions __tests__/embedder.test.ts

This file was deleted.

26 changes: 0 additions & 26 deletions __tests__/info.test.ts

This file was deleted.

Loading
Loading