OpenSpec Instructions

These instructions are for AI assistants working in this project.

Always open @/openspec/AGENTS.md when the request:

Mentions planning or proposals (words like proposal, spec, change, plan)
Introduces new capabilities, breaking changes, architecture shifts, or big performance/security work
Sounds ambiguous and you need the authoritative spec before coding

Use @/openspec/AGENTS.md to learn:

How to create and apply change proposals
Spec format and conventions
Project structure and guidelines

Keep this managed block so 'openspec update' can refresh the instructions.

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Important: This project uses English as its primary language. All documentation, code comments, and commit messages should be in English.

Project Overview

CKAN MCP Server - MCP (Model Context Protocol) server for interacting with CKAN-based open data portals (dati.gov.it, data.gov, demo.ckan.org, etc.).

The server exposes MCP tools for:

Advanced dataset search with Solr syntax
DataStore queries for tabular data analysis
Organization and group exploration
Complete metadata access

Build and Development

Main Commands

# Build project (uses esbuild - fast and lightweight)
npm run build

# Run test suite (191 tests - unit + integration)
npm test

# Watch mode for tests during development
npm run test:watch

# Test coverage report
npm run test:coverage

# Start server in stdio mode (default for local integration)
npm start

# Start server in HTTP mode (for remote access)
TRANSPORT=http PORT=3000 npm start

# Watch mode for development
npm run watch

# Build + run
npm run dev

# Cloudflare Workers deployment (v0.4.0+)
npm run build:worker      # Build for Workers
npm run dev:worker        # Test locally with Wrangler
npm run deploy            # Deploy to Cloudflare Workers

Local MCP Client Build Test

Before deploying, you can test the current dev build by pointing your MCP client at the Node entrypoint in dist/:

Build: npm run build
Example absolute path: /home/aborruso/git/idee/ckan-mcp-server/dist/index.js (adjust to your local checkout)

GitHub CLI Notes

When creating issues with multi-line bodies, avoid literal \n in --body. Use a here-doc or -F - / --body-file - to preserve newlines:

cat <<'EOF' | gh issue create --title "Title" --body-file - --repo ondata/ckan-mcp-server
Line 1
Line 2
EOF

When writing GitHub release notes, avoid literal \n in --notes. Always use a here-doc and --notes-file - so line breaks render correctly:

cat <<'EOF' | gh release create v0.X.Y --title "v0.X.Y - Title" --notes-file -
## What's New

### Changes
- Item 1
- Item 2

### No Breaking Changes
- All existing functionality preserved

**Full Changelog**: https://github.com/ondata/ckan-mcp-server/compare/v0.X-1.Y...v0.X.Y
EOF

Build System

The project uses esbuild for compilation and vitest for testing:

Build: Ultra-fast builds (milliseconds instead of minutes)
Tests: 191 tests (unit + integration) with 100% success rate
Coverage: ~39% overall (utils: 98%, tools: 15-20%) - available via vitest with v8 coverage engine

The build:tsc script is available as a fallback but can cause memory issues in some environments (particularly WSL). Always use npm run build which uses esbuild.

The esbuild build bundles all internal modules but keeps external dependencies (@modelcontextprotocol/sdk, axios, express, zod) as external, so they must be present in node_modules.

Testing

The project has a comprehensive test suite using Vitest:

tests/
├── unit/
│   ├── formatting.test.ts    # Utility functions (19 tests)
│   ├── http.test.ts           # HTTP client (11 tests)
│   └── uri.test.ts            # URI parsing (11 tests)
├── integration/
│   ├── package.test.ts        # Package tools (31 tests)
│   ├── organization.test.ts   # Organization tools (6 tests)
│   ├── datastore.test.ts      # DataStore tools (19 tests)
│   ├── resources.test.ts      # MCP Resources (11 tests)
│   └── status.test.ts         # Status tools (2 tests)
└── fixtures/
    ├── responses/             # Success response mocks
    └── errors/                # Error scenario mocks

Test Coverage: 191 tests total (117 unit + 74 integration)

When making changes:

Run tests before committing: npm test
Ensure all tests pass
Add tests for new features or bug fixes
Follow existing test patterns in tests/ directory

Test target rule: Never use demo.ckan.org for tests. Always use https://www.dati.gov.it/opendata.

Before committing and pushing: For any locally testable change, always:

Build locally: npm run build
Run automated tests: npm test
Run real HTTP server tests to verify end-to-end behavior:
- Start: TRANSPORT=http PORT=3001 node dist/index.js & disown
- Call each affected tool via curl against a real CKAN portal
- Stop: kill $(lsof -ti:3001)

Portal selection for HTTP tests:

General search / organization tools: https://www.dati.gov.it/opendata
DataStore tools: https://dati.comune.messina.it or https://open.canada.ca/data — dati.gov.it does NOT have DataStore
To find a valid datastore resource_id: curl -s "https://dati.comune.messina.it/api/3/action/package_search?q=*:*&rows=20" | jq -r '[.result.results[].resources[] | select(.datastore_active==true) | .id][0]'

Architecture

Code Structure

The server is implemented with a modular structure to improve maintainability and testability:

src/
├── index.ts              # Entry point Node.js (42 lines)
├── worker.ts             # Entry point Cloudflare Workers (95 lines) [v0.4.0+]
├── server.ts             # MCP server setup (12 lines)
├── types.ts              # Types & schemas (16 lines)
├── utils/
│   ├── http.ts           # CKAN API client (51 lines)
│   └── formatting.ts     # Output formatting (37 lines)
├── tools/
│   ├── package.ts        # Package tools (350 lines)
│   ├── organization.ts   # Organization tools (341 lines)
│   ├── datastore.ts      # DataStore tools (146 lines)
│   └── status.ts         # Status tools (66 lines)
├── resources/            # MCP Resource Templates
│   ├── index.ts          # Resource registration (19 lines)
│   ├── uri.ts            # URI parsing utilities (50 lines)
│   ├── dataset.ts        # Dataset resource (56 lines)
│   ├── resource.ts       # Resource resource (56 lines)
│   └── organization.ts   # Organization resource (58 lines)
└── transport/
    ├── stdio.ts          # Stdio transport (12 lines)
    └── http.ts           # HTTP transport (27 lines)

Total: ~1445 lines (including Workers deployment)

Note: worker.ts (v0.4.0+) is an alternative entry point for Cloudflare Workers deployment. Tool handlers (tools/*) are shared between Node.js and Workers runtimes.

The server (src/index.ts):

Entry Point (index.ts)
- Imports and registers all tools
- Chooses transport (stdio/http) from environment variable
- Handles startup and error handling
Registered Tools (in separate modules)
- tools/package.ts: ckan_package_search, ckan_package_show
- tools/organization.ts: ckan_organization_list, ckan_organization_show, ckan_organization_search
- tools/datastore.ts: ckan_datastore_search
- tools/status.ts: ckan_status_show
MCP Resource Templates (resources/)
- ckan://{server}/dataset/{id} - Dataset metadata
- ckan://{server}/resource/{id} - Resource metadata
- ckan://{server}/organization/{name} - Organization metadata
Utility Functions (utils/)
- http.ts: makeCkanRequest<T>() - HTTP client for CKAN API v3
- formatting.ts: truncateText(), formatDate(), formatBytes()
Type Definitions (types.ts)
- ResponseFormat enum (MARKDOWN, JSON)
- ResponseFormatSchema Zod validator
- CHARACTER_LIMIT constant
Transport Layer (transport/)
- stdio.ts: Standard input/output (Claude Desktop)
- http.ts: HTTP server (remote access)
Validation Schema
- Uses Zod to validate all tool inputs
- Each tool has a strict schema that rejects extra parameters
Output Formatting
- All tools support two formats: markdown (default) and json
- Markdown format optimized for human readability
- JSON format returns compact objects with only essential fields (~70% token reduction vs raw CKAN API)
- JSON truncation is safe: shrinks arrays instead of cutting mid-string (always valid JSON)
- See docs/JSON-OUTPUT.md for complete field schemas

Transport Modes

The server supports three transport modes:

stdio (default): for integration with Claude Desktop and other local MCP clients
http: exposes POST /mcp endpoint on configurable port (default 3000)
Cloudflare Workers (v0.4.0+): global edge deployment via src/worker.ts

Cloudflare Workers Deployment

Added in v0.4.0. The server can be deployed to Cloudflare Workers for global HTTP access.

Key files:

src/worker.ts - Workers entry point using Web Standards transport
wrangler.toml - Cloudflare configuration

Deployment workflow:

npm run dev:worker - Test locally (http://localhost:8787)
npm run deploy - Deploy to Cloudflare
Access at: https://ckan-mcp-server.<account>.workers.dev

Architecture:

Uses WebStandardStreamableHTTPServerTransport from MCP SDK
Compatible with Workers runtime (no Node.js APIs)
Stateless mode (no session management)
All 7 tools and 3 resource templates work identically to Node.js version

See docs/DEPLOYMENT.md for complete deployment guide.

CKAN API Integration

The server uses CKAN API v3 available on any CKAN portal. All requests:

Use axios with 30 second timeout
Send User-Agent CKAN-MCP-Server/1.0
Handle HTTP errors, timeouts, and server not found
Normalize server URL (removing trailing slash)
Validate that response.success === true

Solr Queries

CKAN uses Apache Solr for search. The ckan_package_search tool supports:

q (query): complete Solr syntax (field:value, AND/OR/NOT, wildcard, range)
fq (filter query): additional filters without affecting score
facet_field: aggregations for statistical analysis
sort: result ordering
start/rows: pagination

Common query examples are documented in EXAMPLES.md.

TypeScript Configuration

The project uses ES2022 as target and module system.

Note: tsconfig.json is present mainly for editor support (IDE, LSP). The actual compilation uses esbuild which ignores most TypeScript options to maximize speed.

TypeScript configuration (for IDE):

Output in dist/ directory
Strict mode enabled
Strict type checking with noUnusedLocals, noUnusedParameters, noImplicitReturns
Skip lib check to reduce overhead
Declaration and source map disabled

Dependencies

Runtime:

@modelcontextprotocol/sdk - Official MCP SDK
axios - HTTP client for CKAN API
zod - Schema validation
express - HTTP server (only for http mode)

Dev:

esbuild - Build tool (bundler and compiler)
typescript - Only for type checking and editor support
@types/node, @types/express - Type definitions

Supported CKAN Portals

The server can connect to any CKAN instance. Some main portals:

🇮🇹 https://dati.gov.it (Italy)
🇺🇸 https://catalog.data.gov (United States)
🇨🇦 https://open.canada.ca/data (Canada)
🇬🇧 https://data.gov.uk (United Kingdom)
🌍 https://demo.ckan.org (Official CKAN Demo)

Each portal may have different configurations for:

DataStore availability
Custom dataset fields
Available organizations and tags
Supported resource formats

The project uses Vitest for automated testing:

# Run all tests
npm test

# Run tests in watch mode
npm run test:watch

# Run tests with coverage report
npm run test:coverage

Current test coverage: ~39% overall (utility modules: 98%, tool handlers: 15-20%).

Test suite includes:

Unit tests for utility functions (formatting, HTTP, URI parsing, URL generation, search)
Integration tests for MCP tools with mocked CKAN API responses
Mock fixtures for CKAN API success and error scenarios

Coverage is strong for utility modules but needs improvement for tool handlers. See tests/README.md for detailed testing guidelines and fixture structure.

Manual Testing

For manual testing, use HTTP transport with curl:

# Terminal 1 — start server
npm run build
TRANSPORT=http PORT=3001 node dist/index.js

# Terminal 2 — call a tool
curl -s -X POST http://localhost:3001/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"ckan_package_search",
      "arguments":{
        "server_url":"https://www.dati.gov.it/opendata",
        "q":"ambiente",
        "page":1,
        "page_size":3
      }
    },
    "id":1
  }'

Key points:

Always include both Content-Type: application/json and Accept: application/json, text/event-stream headers
Use node dist/index.js directly (not npm start) to avoid npm buffering
Use port 3001 to avoid conflicts with other services on 3000
Check output for correct field values, pagination hints, and Showing: N results (from X) line

To test with Claude Desktop, add MCP configuration to config file.

Development Notes

Version History

v0.4.0 (2026-01-10): Cloudflare Workers deployment

Added src/worker.ts for Workers runtime
Global edge deployment support
Web Standards transport integration
See docs/DEPLOYMENT.md for deployment guide

v0.3.0 (2026-01-08): MCP Resource Templates

Added ckan:// URI scheme support
Direct data access for datasets, resources, organizations

v0.2.0 (2026-01-08): Comprehensive test suite

191 tests (unit + integration)
~39% code coverage (utils well-tested, tools improving)

v0.1.0 (2026-01-08): Modular refactoring

Restructured from monolithic file to 11 modules
Improved maintainability and testability

Known Limitations

Output limit: 50,000 characters hardcoded in types.ts (could be configurable)
Date formatting: Uses fixed ISO YYYY-MM-DD in utils/formatting.ts (could be configurable)
Read-only: All tools are read-only (no data modification on CKAN)
Caching: Read-through cache in makeCkanRequest. Action-based TTL (metadata 300s, datastore 60s, status 3600s). Backend: Cloudflare Cache API on Workers, in-memory LRU on Node. Disable with CKAN_CACHE_ENABLED=false. Env vars: CKAN_CACHE_TTL_DEFAULT, CKAN_CACHE_MAX_ENTRIES, CKAN_CACHE_MAX_ENTRY_BYTES.
No authentication: Uses only public CKAN endpoints
No WebSocket: MCP over HTTP uses JSON responses (not SSE streaming in Workers)
Domain allowlist: Optional SSRF hardening via CKAN_ALLOWED_DOMAINS=domain1.com,domain2.org. If set, requests to unlisted domains are blocked. Default: no restriction (all public domains allowed). Enforced in validateServerUrl().
Audit logging: Every makeCkanRequest call writes a JSON line to stderr (Node modes only; Workers use console.log). Fields: ts, server, action, cache_hit, plus relevant query params (q, fq, sql truncated to 200 chars, id, rows, limit).

Adding New Tools

Create new file in src/tools/
Export registerXxxTools(server: McpServer) function
Add to registerAll() in src/server.ts
Add tests in tests/integration/
Build and test: npm run build && npm test

README Authoring Rule

The project has two README files:

README.md — full README, shown on GitHub
.readme-npm.md — short README published to npm (intro + essentials + link to full README)

prepack/postpack hooks in package.json swap them automatically during npm publish. Never run npm publish manually without building first (npm run build).

Always use absolute GitHub URLs in both README files — never relative paths. npm cannot resolve relative paths from the tarball.

Images/GIFs: https://raw.githubusercontent.com/ondata/ckan-mcp-server/main/...
Links to files: https://github.com/ondata/ckan-mcp-server/blob/main/...

Release Workflow

When releasing a new version:

Update version: Edit package.json version field and manifest.json version field
Update LOG.md: Add entry with date and changes
Commit changes: git add . && git commit -m "..."
Push to GitHub: git push origin main
Create tag: git tag -a v0.x.0 -m "..." && git push origin v0.x.0
Build DXT: npm run pack:dxt → produces ckan-mcp-server.dxt
Build skill: npm run pack:skill → produces tmp/ckan-mcp.skill
Attach to release: gh release upload v0.x.0 ckan-mcp-server.dxt tmp/ckan-mcp.skill
Publish to npm (optional): npm publish
Deploy to Cloudflare (if code changed): npm run deploy

See docs/DEPLOYMENT.md for detailed Cloudflare deployment instructions.

CSV Data Exploration

For exploring CSV resources from datasets, use duckdb CLI (already installed) with direct HTTP URL:

duckdb -jsonlines -c "DESCRIBE SELECT * FROM read_csv('http://url/file.csv')"
duckdb -jsonlines -c "SUMMARIZE SELECT * FROM read_csv('http://url/file.csv')"
duckdb -jsonlines -c "SELECT * FROM read_csv('http://url/file.csv') USING SAMPLE 10"

Use direct resource URLs (http/https), not GitHub view/blob URLs. The -jsonlines parameter outputs in JSONL format, easier for AI to parse.

For random sampling, use USING SAMPLE N syntax (where N is the number of rows):

duckdb -jsonlines -c "SELECT * FROM read_csv('http://url/file.csv') USING SAMPLE 10"

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenSpec Instructions

CLAUDE.md

Project Overview

Build and Development

Main Commands

Local MCP Client Build Test

GitHub CLI Notes

Build System

Testing

Architecture

Code Structure

Transport Modes

Cloudflare Workers Deployment

CKAN API Integration

Solr Queries

TypeScript Configuration

Dependencies

Supported CKAN Portals

Manual Testing

Development Notes

Version History

Known Limitations

Adding New Tools

README Authoring Rule

Release Workflow

CSV Data Exploration

FilesExpand file tree

CLAUDE.md

Latest commit

History

CLAUDE.md

File metadata and controls

OpenSpec Instructions

CLAUDE.md

Project Overview

Build and Development

Main Commands

Local MCP Client Build Test

GitHub CLI Notes

Build System

Testing

Architecture

Code Structure

Transport Modes

Cloudflare Workers Deployment

CKAN API Integration

Solr Queries

TypeScript Configuration

Dependencies

Supported CKAN Portals

Manual Testing

Development Notes

Version History

Known Limitations

Adding New Tools

README Authoring Rule

Release Workflow

CSV Data Exploration