xk6-ai

xk6-ai is a k6 extension for testing AI tools, specifically AI agents. The extension provides a JavaScript/TypeScript API that enables automated testing scenarios to measure AI agent performance, reliability, and quality.

❗ This is a proof of concept

This extension is not supported by Grafana. The API can break in the future without notice. USE AT YOUR OWN RISK!

Features

• LLM Integration: Support for multiple LLM providers (OpenAI, Ollama, Anthropic, etc.)
• A2A Protocol: Communication with AI agents using the official A2A SDK
• Response Validation: Evaluate agent responses using validator LLMs
• User Interaction: Generate tasks based on user prompts
• Performance Testing: Measure AI agent performance under load

Prerequisites

• Access to LLM providers (OpenAI, Anthropic, etc.)
• A2A-compatible agent endpoint

Installation

Using xk6

Prerequisites

• Go 1.21+ installed
• xk6 tool installed (go install go.k6.io/xk6/cmd/xk6@latest)

# Install xk6
go install go.k6.io/xk6/cmd/xk6@latest

# Build k6 with xk6-ai extension
xk6 build --with github.com/grafana/xk6-ai@latest

# Run your test
./k6 run your-test.js

Quick start

QUICKSTART.md

Implementation plan

• Core implementation (llms, basic support for a2a protocol, evaluation)
• JS API implementation
• Integration tests
• E2e tests
• Test with real agents
• Generate custom metrics
• Async API
• Support for asynchronous agent request

Contributing

See CONTRIBUTING.md for development setup, guidelines, and contribution instructions.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

• Issues: GitHub Issues