Research: Streamline sync with upstream repos (pygraphistry, graphistry-js, etc.)

[lmeyerov]

Summary

Research and design a system to keep skills in sync with upstream Graphistry repositories, catching regressions and new features automatically.

Problem

Skills can drift from upstream repos:

• New features: APIs added to pygraphistry/graphistry-js that skills don't cover
• Breaking changes: API signatures change, skills become stale
• Deprecations: Skills reference deprecated patterns
• Bug fixes: Upstream fixes that skills should reflect

Currently no automated way to detect when upstream changes require skill updates.

Upstream Repos to Track

• pygraphistry - Python client
• graphistry-js - JS client + REST API
• graphistry-cli - CLI tools
• ReadTheDocs sites - documentation changes

Research Questions

1. Change detection: How to detect relevant upstream changes?

   • GitHub webhooks / Actions on upstream repos?
   • Periodic diffing of API surfaces?
   • Release tag monitoring?

2. Impact analysis: How to map upstream changes to affected skills?

   • Static analysis of skill references?
   • Test-based detection (run evals, see what breaks)?

3. Notification/action: What happens when drift is detected?

   • Auto-file issues in graphistry-skills?
   • Slack/email alerts?
   • Auto-PR with suggested updates?

4. Coverage tracking: How to know what % of upstream API is covered by skills?

   • API surface enumeration
   • Skill-to-API mapping

Potential Approaches

1. GitHub Actions cross-repo triggers

• Upstream repos trigger graphistry-skills CI on release
• Run evals against new version, detect regressions

2. Scheduled API diffing

• Nightly job compares current API surface to skill coverage
• Flags new/changed/removed APIs

3. Release notes parsing

• Parse CHANGELOG.md from upstream repos
• Extract feature/breaking change mentions
• Map to affected skills

4. Monorepo or submodule approach

• Pin upstream versions explicitly
• Bump + test in controlled way

Deliverables

• Research doc comparing approaches
• Prototype of recommended approach
• Integration plan with existing CI
• Documentation

References

• Skills repo: graphistry-skills
• Eval harness: scripts/agent_eval_loop.py
• Related: Evals: Evergreen / self-regenerating eval journeys #8 (evergreen evals)

Notes

This is a research/design issue. Start with investigation and a written proposal before implementing.

---

Help wanted! Community contributions welcome.