You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Feb 11, 2026. It is now read-only.
During documentation audit, we discovered several documented CLI commands that don't actually exist or work. We need automated testing to prevent this.
What Happened
Documentation claimed these commands existed:
❌ get-paper-citations - doesn't exist
❌ get-paper-references - doesn't exist
❌ get-citation-network - doesn't exist
❌ get-all-identifiers - wrong name (should be get-all-identifiers-from-europepmc)
These were documented for months without anyone noticing.
Root Cause
No automated testing of documentation examples
CLI and MCP tools aren't in sync
No CI/CD validation that documented commands actually work
Proposed Solution
1. Create Documentation Test Suite
Add tests/test_documentation_examples.py:
"""Test that all examples in README.md and USERS.md actually work."""deftest_readme_cli_examples():
"""Test every CLI command shown in README.md."""# Extract code blocks from README# Run each command# Verify it doesn't errordeftest_users_md_cli_examples():
"""Test every CLI command shown in USERS.md."""deftest_mcp_tools_listed_in_readme():
"""Verify MCP tool list matches actual registered tools."""# Parse list from README# Get actual registered tools from main.py# Assert they match
2. Add CI Check
Add to .github/workflows/:
- name: Test documentation examplesrun: | pytest tests/test_documentation_examples.py
Problem
During documentation audit, we discovered several documented CLI commands that don't actually exist or work. We need automated testing to prevent this.
What Happened
Documentation claimed these commands existed:
get-paper-citations- doesn't existget-paper-references- doesn't existget-citation-network- doesn't existget-all-identifiers- wrong name (should beget-all-identifiers-from-europepmc)These were documented for months without anyone noticing.
Root Cause
Proposed Solution
1. Create Documentation Test Suite
Add
tests/test_documentation_examples.py:2. Add CI Check
Add to
.github/workflows/:3. Add Pre-commit Hook
Optionally add hook that:
artl-cli --help4. Create Command Inventory
Maintain
commands.json:{ "mcp_tools": ["search_europepmc_papers", ...], "cli_commands": ["get-doi-metadata", ...], "both": ["search-papers-by-keyword", ...] }Update this automatically and verify docs match.
Testing Levels
Benefits
References