refactor: skill cli#37
Conversation
There was a problem hiding this comment.
Code Review
This pull request undertakes a significant refactoring of the AntV Skills project, migrating the entire codebase from JavaScript to TypeScript. The changes introduce a new modular structure, a command-line interface built with commander.js, and a programmatic API for skill retrieval. Core functionalities like BM25 search, skill retrieval, and index building have been re-implemented in TypeScript, and project dependencies have been updated accordingly. Feedback highlights a high-severity bug in the custom YAML parser in src/scripts/build.ts that incorrectly handles multi-line strings, a logical issue with the englishPattern regex in src/core/bm25.ts due to lowercasing, and a recommendation to relocate build artifacts from src/index to dist/index for improved project organization.
| } else if (currentKey && line.startsWith(' ') && currentArray) { | ||
| if (Array.isArray(meta[currentKey])) { | ||
| meta[currentKey] = meta[currentKey].join(' ') + ' ' + line.trim(); | ||
| } | ||
| } |
There was a problem hiding this comment.
The manual YAML parser fails to correctly handle multi-line strings (block scalars). After the first line of a multi-line string is processed, meta[currentKey] is reassigned from an array to a string. On subsequent lines, the Array.isArray(meta[currentKey]) check will fail, causing all remaining lines of the multi-line string to be ignored. Consider using a robust YAML parsing library like js-yaml or fixing the accumulation logic to keep the lines in an array until the entire block is parsed.
| const normalized = text.toLowerCase(); | ||
| const tokens: string[] = []; | ||
|
|
||
| const englishPattern = /[a-z][a-z0-9]*(?:[A-Z][a-z0-9]*)*/gi; |
There was a problem hiding this comment.
The englishPattern regex includes a character class for uppercase letters [A-Z], but it is applied to normalized, which has already been converted to lowercase. This makes the camelCase matching logic ineffective and the regex redundant. If camelCase splitting is intended, the regex should be applied to the original text before lowercasing, or the regex should be simplified to avoid confusion.
| const normalized = text.toLowerCase(); | |
| const tokens: string[] = []; | |
| const englishPattern = /[a-z][a-z0-9]*(?:[A-Z][a-z0-9]*)*/gi; | |
| const normalized = text.toLowerCase(); | |
| const tokens: string[] = []; | |
| const englishPattern = /[a-z][a-z0-9]*/gi; |
|
|
||
| const PKG_ROOT = path.resolve(__dirname, '../..'); | ||
| const SKILLS_DIR = path.join(PKG_ROOT, 'skills'); | ||
| const INDEX_DIR = path.join(PKG_ROOT, 'src', 'index'); |
There was a problem hiding this comment.
Generating build artifacts into the src/ directory is generally discouraged as it pollutes the source tree. It is better to generate the index files directly into the dist/ directory or a dedicated build/ directory. This would also allow simplifying the build script in package.json by removing the cp command.
| const INDEX_DIR = path.join(PKG_ROOT, 'src', 'index'); | |
| const INDEX_DIR = path.join(PKG_ROOT, 'dist', 'index'); |
* chore: 新增 g2 skills * chore: add performance part * fix: test * fix: Ai CR * chore: user invocable * chore: AI SDK 优化 * chore: add harness * fix: path error * chore: 补充参考文档 * chore: 新增 grep 兜底检索 * chore: eval 自检更新 * chore: 删除无效scripts * chore: skill 优化 * chore: add worktree * chore: add kimi model * chore: override env * fix: 修复 data 格式错误 * chore: 目录结构调整 * chore: 补充评测数据 * chore: 删除 common 模块 * chore: 优化 scale skill * chore: skill tool 移动到 cli 下 * chore: add playground * refactor: skill cli (#37) Co-authored-by: 逍为 <xiaowei.wzw@antgroup.com> Co-authored-by: Joel Alan <31396322+lxfu1@users.noreply.github.com> * test: fix ci workflow (#38) Co-authored-by: 逍为 <xiaowei.wzw@antgroup.com> * docs: add readme for cli usage (#39) * docs: add readme for cli usage * docs: fix skill typo * docs: update * test: fix ci --------- Co-authored-by: 逍为 <xiaowei.wzw@antgroup.com> * refactor: playground for cli and skill (#40) * chore: 补全 playground libs * fix: ts * fix: resolve conversation * chore: 代码优化 * fix: 修复由于 nextjs 加载源码导致 __dirname 异常 (#44) * chore: 添加 claude plugin (#43) * refactor: eval logic (#42) * chore: 优化 harness 逻辑 (#48) * chore: 优化 harness 逻辑 * chore: 优化 harness 流程 * fix: result count * feat: 新增 retrieveWithContent (#49) * feat: 新增 retrieveWithContent * fix: test * chore: loadSkillContent 目录约束 * fix: 修复 work tree 模式下 __dirname 不一致导致的优化异常 * chore: 新增 harness 优化 skill 后的基线对比逻辑 (#51) * chore: cli 重命名 (#52) * chore: cli 重命名 * fix: tests * refactor: antv cli (#53) * 精简索引构建输出并优化检索链路(移除 indexDir、使用 gray-matter 解析元数据、简化 includeContent,并统一 index 到 src/index→dist/index) (#56) * Refactor playground AI flow to Vercel AI SDK with Skill/CLI mode tool assembly and token usage visibility (#54) * Initial plan * refactor(playground): migrate chat flow to vercel ai sdk Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/60eb740b-946b-4dbc-8065-b2143da12fa4 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * fix(playground): harden skill path handling and restore temperature Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/60eb740b-946b-4dbc-8065-b2143da12fa4 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * refactor(playground): split skill tools into separate modules Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/bcdb3e77-18a5-4b73-8c9d-4ae97f87510a Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * feat(playground): redesign dual-mode ai tools workflow Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/dd7b7fe2-54a4-4e8c-ac4d-531df443ff31 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * fix(playground): normalize library naming in prompts Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/dd7b7fe2-54a4-4e8c-ac4d-531df443ff31 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * fix(playground): tighten tool input limits and constants Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/dd7b7fe2-54a4-4e8c-ac4d-531df443ff31 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * fix(playground): simplify read-file validation fallback naming Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/dd7b7fe2-54a4-4e8c-ac4d-531df443ff31 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * refactor(playground): use explicit step constants and schemas Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/dd7b7fe2-54a4-4e8c-ac4d-531df443ff31 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * refactor(playground): rename modes to skill/cli and unify tools directory Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/69d26ddf-26a6-43dd-bdca-32bb851ea746 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * fix(playground): harden tools path handling and naming Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/69d26ddf-26a6-43dd-bdca-32bb851ea746 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * fix(playground): tighten tool path validation guards Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/69d26ddf-26a6-43dd-bdca-32bb851ea746 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * refactor(playground): move mode entry files above tools directory Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/e4438102-2cd2-42ac-b772-a29fa11c5a20 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * fix(playground): adapt retrieve tool to updated retrieve api Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/8d505619-d6d9-48dd-85a5-af7c42bc73b7 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * refactor(playground): remove retrieve content fallback path Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/c8187137-e11b-480a-9abf-ed20f30bb6f7 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * refactor(playground): use gray-matter in list references tool Agent-Logs-Url: https://github.com/antvis/chart-visualization-skills/sessions/23696116-a1cb-402e-954e-06b426ddc4b2 Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> * chore: opt cli command * refactor: playground * feat: fix style * feat: update website meta info * fix: tool call error --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> Co-authored-by: hustcc <i@hust.cc> Co-authored-by: 逍为 <xiaowei.wzw@antgroup.com> * refactor: 重构 g2-chart skill (#57) * refactor: 重构 g2-chart skill * chore: resolve skill * chore: 去掉 eval 中 list refercnces 逻辑 * chore: playground 模块支持加载外层 env (#58) * chore: playground 模块支持加载外层 env * Apply suggestions from code review Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: hustcc <i@hust.cc> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * docs: add agent.md (#60) * docs: add agent.md * chore: fix typo --------- Co-authored-by: 逍为 <xiaowei.wzw@antgroup.com> * chore: 修改 env 示例 (#59) * chore: 修改 env 示例 * chore: 新增 VL Model 配置示例 * refactor: playground (#61) * refactor: playground * Update playground/src/libs/cli-mode.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update playground/src/libs/skill-mode.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update playground/src/app/api/generate/route.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: 逍为 <xiaowei.wzw@antgroup.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: hustcc <i@hust.cc> Co-authored-by: 逍为 <xiaowei.wzw@antgroup.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: hustcc <7856674+hustcc@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
package.json 变更:
编译和所有命令均验证通过。