Skip to content

Htunn/simple-cicd-translator

BranchesTags

Repository files navigation

CI/CD Translator

Agentic service that translates CI/CD pipeline definitions between Jenkins, GitLab CI, and GitHub Actions using a pluggable LLM (Gemini, OpenAI, Azure OpenAI, GitHub Models, Anthropic, or Ollama — chosen via LLM_PROVIDER), LangGraph memory, and an MCP (Model Context Protocol) tool surface.

python fastapi langgraph license

Table of contents

  1. Highlights
  2. Architecture
  3. Sequence diagrams
  4. Quick start (Makefile)
  5. How to use
  6. Configuration
  7. API reference
  8. MCP server
  9. Adding a new provider
  10. Production deployment
  11. Testing
  12. Security
  13. Project layout
  14. Further docs
  15. License

Highlights

  • Three-step agentic workflow (analyse → translate → validate) implemented in LangGraph.
  • Jenkins Declarative and Scripted pipelines — the agent classifies the style during the analyse step and faithfully translates both pipeline { ... } blocks and free-form Groovy (@Library, def helpers, parallel(...), try/catch/finally, properties([...]), withCredentials, currentBuild.result, etc.).
  • Pluggable provider registry — add Bitbucket Pipelines, CircleCI, Azure DevOps, etc., without touching core code (entry-point discovery supported).
  • Dual transports — FastAPI REST API and an MCP server exposing the same translation tools to Claude, Copilot, Cursor, etc.
  • Pluggable persistence — SQLite for local dev, PostgreSQL for production (LangGraph checkpoints).
  • RBAC — JWT auth with admin / translator / viewer roles.
  • Production-ready container — multi-stage Dockerfile, non-root user, healthcheck, resource limits.
  • One-command local dev via Makefile.

Architecture

flowchart LR
    subgraph Clients
        UI[Web / CLI]
        MCPCLI[MCP Client<br/>Claude · Cursor · Copilot]
    end

    subgraph App["cicd-translator (FastAPI + MCP)"]
        AUTH[/Auth Router<br/>JWT + RBAC/]
        TRANS[/Translation Router/]
        MCP[/MCP Server<br/>/mcp/]
        WF[LangGraph Workflow<br/>analyse → translate → validate]
        REG[Provider Registry]
    end

    subgraph Providers
        JEN[Jenkins REST]
        GL[GitLab API]
        GH[GitHub API]
        EXT[3rd-party<br/>entry-points]
    end

    subgraph Infra
        PG[(PostgreSQL<br/>LangGraph<br/>checkpoints)]
        GEM[LLM API<br/>Gemini · OpenAI · Azure · GH Models · Anthropic · Ollama]
    end

    UI -->|HTTPS + JWT| AUTH
    UI -->|HTTPS + JWT| TRANS
    MCPCLI -->|MCP/SSE| MCP

    TRANS --> WF
    MCP --> WF
    WF -->|LLM calls| GEM
    WF -->|persist state| PG

    TRANS --> REG
    MCP --> REG
    REG --> JEN
    REG --> GL
    REG --> GH
    REG --> EXT
Loading

Component overview

Layer Module Responsibility
API api/auth_router.py Login, user CRUD
API api/translation_router.py Translate inline / fetch-and-translate / push
MCP mcp_server.py translate_pipeline, fetch_pipeline, fetch_and_translate tools
Agent agent/workflow.py 3-node LangGraph: analyse → translate → validate
Agent agent/checkpointer.py SQLite / Postgres checkpoint factory
Providers providers/registry.py Pluggable provider registry (+ entry-point discovery)
Providers providers/{jenkins,gitlab_provider,github_provider}.py Built-in connectors
Auth auth/security.py JWT, password hashing, RBAC dependencies
Config config.py pydantic-settings environment loader

Sequence diagrams

1. Authentication

sequenceDiagram
    autonumber
    participant C as Client
    participant API as FastAPI /auth/token
    participant SEC as auth.security
    C->>API: POST username + password (form)
    API->>SEC: authenticate_user()
    SEC-->>API: user dict (or None)
    alt invalid credentials
        API-->>C: 401 Unauthorized
    else valid
        API->>SEC: create_access_token(role)
        SEC-->>API: signed JWT
        API-->>C: 200 { access_token, token_type }
    end
Loading

2. Inline translation (POST /api/v1/translate/)

sequenceDiagram
    autonumber
    participant C as Client
    participant API as Translation Router
    participant WF as LangGraph Workflow
    participant CP as Checkpointer<br/>(SQLite / Postgres)
    participant LLM as LLM (chosen via LLM_PROVIDER)

    C->>API: POST pipeline_content + source/target + JWT
    API->>API: require_translator (RBAC)
    API->>WF: run_translation(request)
    WF->>CP: open checkpoint store
    WF->>LLM: analyse_node(source pipeline)
    LLM-->>WF: structured analysis
    WF->>LLM: translate_node(messages)
    LLM-->>WF: candidate translation
    WF->>LLM: validate_node(candidate)
    LLM-->>WF: LGTM or corrected output
    WF->>CP: persist final state (thread_id)
    WF-->>API: TranslationResponse
    API-->>C: 200 { thread_id, translated_pipeline, notes }
Loading

3. Fetch-translate-push (POST /api/v1/translate/push)

sequenceDiagram
    autonumber
    participant C as Client
    participant API as Translation Router
    participant REG as Provider Registry
    participant SRC as Source Provider
    participant WF as LangGraph Workflow
    participant TGT as Target Provider

    C->>API: POST source+target+repo+JWT
    API->>REG: get_provider(source)
    REG-->>API: SourceProvider
    API->>SRC: fetch_pipeline(repo, ref, path)
    SRC-->>API: raw pipeline file
    API->>WF: run_translation(...)
    WF-->>API: translated_pipeline
    API->>REG: get_provider(target)
    REG-->>API: TargetProvider
    API->>TGT: push_pipeline(repo, branch, path, content)
    TGT-->>API: commit URL
    API-->>C: 200 { thread_id, pushed_to, notes }
Loading

4. MCP tool invocation

sequenceDiagram
    autonumber
    participant M as MCP Client
    participant SRV as FastApiMCP (/mcp)
    participant T as POST /api/v1/translate (auto-exposed MCP tool)
    participant WF as LangGraph Workflow
    participant LLM as LLM (chosen via LLM_PROVIDER)

    M->>SRV: tools/call translate_pipeline_api_v1_translate__post { args }
    SRV->>T: dispatch
    T->>WF: run_translation(request)
    WF->>LLM: 3-step agentic loop
    LLM-->>WF: validated output
    WF-->>T: TranslationResponse
    T-->>SRV: dict
    SRV-->>M: MCP result
Loading

Quick start (Makefile)

# 1. Create .env from template and pick an LLM
make env
$EDITOR .env   # set LLM_PROVIDER + the matching API key + APP_SECRET_KEY

# 2. Install (creates .venv, installs project + dev deps)
make install

# 3. Run unit tests
make test

# 4. One-shot end-to-end test against a live LLM call (SQLite checkpoints)
make smoke

# 5. Or: end-to-end test against a Postgres checkpoint store
make smoke-postgres

Other useful targets:

Target Description
make run Run API in dev mode with hot reload
make run-prod Run API in production mode (2 workers)
make serve-bg / make stop Background server + stop
make e2e Run the live-LLM E2E script against $BASE_URL
make db-up / make db-down / make db-psql Manage local Postgres via docker compose
make docker-up / make docker-down Dev stack in containers (API + Postgres)
make docker-prod-up / make docker-prod-down Production stack
make lint ruff + mypy
make clean Remove caches

Run make help for the complete list.

How to use

This project is an agentic CI/CD pipeline translator. The typical flow is:

  1. You point it at (or send it the content of) a Jenkinsfile / .gitlab-ci.yml / GitHub Actions workflow.
  2. A 3-step LangGraph agent (analyse → translate → validate) executes against the configured LLM provider (Gemini by default) with persistent memory.
  3. You get back an idiomatic target-platform pipeline, plus reviewer notes.

Three usage modes:

Mode Best for Entry point
REST API CI scripts, CLI, web UI POST /api/v1/translate/*
MCP Claude · Cursor · Copilot · custom agents http://localhost:8000/mcp
End-to-end smoke (declarative) Quick demo / regression make smoke (uses sample/Jenkinsfile)
End-to-end smoke (scripted) Verify scripted Groovy support make smoke-scripted (uses sample/Jenkinsfile.scripted)

A realistic Jenkins pipeline is included at sample/Jenkinsfile. After make smoke, the translated GitHub Actions workflow appears at:

sample/out/translated.github.yml

For full step-by-step recipes (REST, MCP, iterative thread_id conversations, Docker) see docs/USAGE.md.

For an in-depth explanation of the agentic graph and how to extend it see docs/AGENTIC_FLOW.md.

Configuration

Configuration is loaded from environment variables (or .env for local dev) by pydantic-settings. See .env.example.

Variable Default Notes
APP_ENV development development enables CORS *, reload
APP_SECRET_KEY — (required, ≥32 chars) JWT signing key
ACCESS_TOKEN_EXPIRE_MINUTES 60
CORS_ALLOW_ORIGINS empty Comma-separated, used when APP_ENV != development
LLM_PROVIDER gemini gemini · openai · azure_openai · github_models · anthropic · ollama
GEMINI_API_KEY / GEMINI_MODEL — / gemini-2.5-pro Required when LLM_PROVIDER=gemini
OPENAI_API_KEY / OPENAI_MODEL / OPENAI_BASE_URL — / gpt-4o / empty Required when LLM_PROVIDER=openai; set OPENAI_BASE_URL for OpenAI-compatible servers
AZURE_OPENAI_API_KEY / AZURE_OPENAI_ENDPOINT / AZURE_OPENAI_DEPLOYMENT / AZURE_OPENAI_API_VERSION — / — / — / 2024-10-21 Required when LLM_PROVIDER=azure_openai
GITHUB_MODELS_TOKEN / GITHUB_MODELS_MODEL / GITHUB_MODELS_BASE_URL — / openai/gpt-4o-mini / https://models.github.ai/inference GitHub Models marketplace; falls back to GITHUB_TOKEN
ANTHROPIC_API_KEY / ANTHROPIC_MODEL — / claude-sonnet-4-5-20250929 Required when LLM_PROVIDER=anthropic
OLLAMA_BASE_URL / OLLAMA_MODEL http://localhost:11434 / llama3.1 Local / self-hosted
JENKINS_URL / JENKINS_USERNAME / JENKINS_TOKEN empty Only needed when using Jenkins fetch
GITLAB_URL / GITLAB_TOKEN https://gitlab.com / empty
GITHUB_TOKEN empty
CHECKPOINT_BACKEND sqlite sqlite or postgres
CHECKPOINT_DB_PATH ./data/checkpoints.db Used when backend = sqlite
CHECKPOINT_POSTGRES_URL empty e.g. postgresql://cicd:cicd@localhost:5432/cicd_translator
DATABASE_URL sqlite Reserved for future app-data storage
POSTGRES_USER / POSTGRES_PASSWORD / POSTGRES_DB / POSTGRES_PORT cicd / — / cicd_translator / 5432 Used by docker compose
HOST / PORT / UVICORN_WORKERS / LOG_LEVEL 0.0.0.0 / 8000 / 1 / info Server runtime

API reference

OpenAPI/Swagger UI: http://localhost:8000/docs ReDoc: http://localhost:8000/redoc MCP endpoint: http://localhost:8000/mcp Healthcheck: GET /health

Auth

Method Path Body Returns
POST /api/v1/auth/token form: username, password { access_token, token_type }
POST /api/v1/auth/users { username, password, role } (admin) UserOut
GET /api/v1/auth/me — UserOut

Default user: admin / changeme123! (change immediately).

Translation

Method Path Purpose
POST /api/v1/translate/ Translate inline pipeline content
POST /api/v1/translate/from-provider Fetch from source provider, then translate
POST /api/v1/translate/push Translate, then commit translated file to target provider

Example

TOKEN=$(curl -sX POST http://localhost:8000/api/v1/auth/token \
  -d 'username=admin&password=changeme123!' | jq -r .access_token)

curl -sX POST http://localhost:8000/api/v1/translate/ \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
        "source_provider": "jenkins",
        "target_provider": "github",
        "pipeline_content": "pipeline { agent any; stages { stage(\"build\") { steps { sh \"make\" } } } }"
      }' | jq .

MCP server

The same translation capabilities are exposed over the Model Context Protocol at /mcp using fastapi-mcp. Three tools are available:

Tool Inputs Description
translate_pipeline source_provider, target_provider, pipeline_content, thread_id? Translate inline content
fetch_pipeline provider, repo_or_job, ref?, pipeline_path? Return raw pipeline file
fetch_and_translate as above + target_provider Fetch + translate in one call

Point an MCP client (Claude Desktop, Cursor, etc.) at http://<host>:8000/mcp.

Adding a new provider

The provider system is fully pluggable. To add (for example) Bitbucket Pipelines:

  1. Implement BaseProvider:

    from cicd_translator.providers.base import BaseProvider

class BitbucketProvider(BaseProvider):
    async def fetch_pipeline(self, repo_or_job, ref, pipeline_path): ...
    async def push_pipeline(self, repo, branch, pipeline_path, content, message): ...

  2. Register it — either in-process:

    from cicd_translator.providers.registry import register_provider
register_provider("bitbucket", BitbucketProvider)

    …or via a pyproject.toml entry point in your own package (auto-discovered):

    [project.entry-points."cicd_translator.providers"]
bitbucket = "my_pkg.bitbucket:BitbucketProvider"

  3. Extend CICDProvider only if you want strict Pydantic validation for the new value — the registry itself accepts any string name.

That's it: the new provider becomes usable through both the REST API and MCP tools.

Production deployment

Recommended: docker compose (API + Postgres)

cp .env.example .env
# Edit .env — at minimum set:
#   APP_SECRET_KEY  (>= 32 random chars)
#   LLM_PROVIDER + the matching API key (e.g. GEMINI_API_KEY / OPENAI_API_KEY / ...)
#   POSTGRES_PASSWORD
#   CORS_ALLOW_ORIGINS

docker compose -f docker-compose.prod.yml --env-file .env up -d
docker compose -f docker-compose.prod.yml logs -f cicd-translator

The production stack provides:

  • Multi-stage build → small image, no build toolchain at runtime
  • Non-root app user (UID 10001)
  • HTTP healthcheck on /health
  • 4 uvicorn workers, proxy_headers=true (place behind nginx/Traefik/ALB for TLS)
  • Restart policy always, CPU/memory limits, rotated JSON logs
  • Persistent Postgres volume for LangGraph checkpoints

Behind a reverse proxy

Terminate TLS upstream and forward to port 8000. Recommended headers: X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Host.

Kubernetes

The Docker image is plain python -m cicd_translator. A minimal Deployment + Service + Secret (env) + managed Postgres (RDS / Cloud SQL / Neon) is sufficient. Configure readinessProbe and livenessProbe at /health.

Do I need docker compose for production?

Not strictly — any container runtime (Kubernetes, ECS, Cloud Run, Fly.io, Render, App Service) works because the image is a self-contained ASGI app. Docker compose is the recommended baseline because it co-provisions Postgres, sets sensible defaults (healthchecks, restart policy, log rotation, resource limits), and is reproducible across environments.

Testing

make test            # unit tests (httpx ASGI transport, no external calls)
make smoke           # e2e — live LLM call via SQLite checkpoint
make smoke-postgres  # e2e — live LLM call via Postgres checkpoint

The scripts/e2e_test.py script performs:

  1. Boots the FastAPI app in-process via the httpx ASGI transport (no real HTTP server required).
  2. Authenticates as the bootstrapped admin to obtain a JWT.
  3. POSTs the contents of a Jenkins pipeline (sample/Jenkinsfile for declarative, sample/Jenkinsfile.scripted for scripted) to /api/v1/translate.
  4. Asserts the response is a valid GitHub Actions workflow (name: + jobs: keys) and writes it to sample/out/.
  5. Exits non-zero on any assertion or upstream LLM failure, so it is safe to run in CI.

Project layout

.
├── README.md
├── LICENSE
├── Dockerfile
├── docker-compose.yml          Local dev stack (API + Postgres)
├── docker-compose.prod.yml     Production stack (hardened)
├── Makefile                    Dev + e2e workflow
├── pyproject.toml              Project metadata + deps
├── .env.example                Reference configuration
├── sample/                     Reference pipelines used by `make smoke`
│   ├── Jenkinsfile             Mid-complexity Jenkins declarative pipeline
│   ├── Jenkinsfile.scripted    Scripted Groovy pipeline
│   ├── .gitlab-ci.yml          GitLab CI equivalent
│   └── README.md
├── docs/                       Long-form documentation
│   ├── HLD.md                  High-level design (C4 context + containers + sequences)
│   ├── AGENTIC_FLOW.md         How the LangGraph workflow works
│   ├── PROJECT_LOGIC.md        LangGraph + LLM + Postgres deep dive
│   └── USAGE.md                Step-by-step usage recipes
├── scripts/
│   └── e2e_test.py             Live end-to-end smoke test
├── src/cicd_translator/
│   ├── __main__.py             `python -m cicd_translator` entrypoint
│   ├── main.py                 FastAPI app factory + MCP mount
│   ├── config.py               pydantic-settings configuration
│   ├── schemas.py              Pydantic models + enums
│   ├── mcp_server.py           Standalone FastMCP tool definitions (legacy)
│   ├── agent/
│   │   ├── workflow.py         3-node LangGraph workflow
│   │   ├── llm.py              Pluggable LLM factory (LLM_PROVIDER)
│   │   └── checkpointer.py     SQLite / Postgres checkpointer factory
│   ├── api/
│   │   ├── auth_router.py      JWT login + user CRUD
│   │   └── translation_router.py
│   ├── auth/security.py        JWT + RBAC
│   └── providers/
│       ├── base.py             BaseProvider ABC
│       ├── registry.py         Pluggable provider registry
│       ├── factory.py          Backwards-compatible factory
│       ├── jenkins.py          Jenkins (declarative + scripted) fetcher
│       ├── gitlab_provider.py
│       └── github_provider.py
└── tests/
    └── test_api.py

Further docs

  • docs/HLD.md — system context, container view, deployment topology, runtime sequence diagrams.
  • docs/AGENTIC_FLOW.md — the LangGraph workflow explained (state, nodes, prompt, tuning, extension).
  • docs/PROJECT_LOGIC.md — deep dive into how LangGraph, the LLM layer, and Postgres interact.
  • docs/USAGE.md — install, REST API, MCP, sample translation, iterative thread_id usage, Docker, troubleshooting.
  • sample/README.md — pipeline fixtures used by make smoke and make smoke-scripted.

Contributing

Contributions are welcome. To propose a change:

  1. Fork the repository and create a feature branch: git checkout -b feat/<short-description>.
  2. Set up the dev environment: python -m venv .venv && source .venv/bin/activate && pip install -e ..
  3. Make your change following the existing layout (src/cicd_translator/...). Keep public APIs typed; add or update a provider in providers/ via the registry, not by editing the factory.
  4. Verify locally before pushing:
    • pytest -q — unit tests must pass.
    • make smoke and/or make smoke-scripted — end-to-end translation against a configured LLM_PROVIDER.
    • docker compose -f docker-compose.prod.yml config -q if you touched the prod compose file.
  5. Commit with a clear, imperative subject line (e.g. feat(providers): add CircleCI provider). Reference an issue when one exists.
  6. Open a PR describing the motivation, the change, and how you validated it. Do not commit secrets — .env is gitignored; use .env.example to document new settings.

Bug reports and feature requests are tracked on GitHub Issues.

License

This project is licensed under the MIT License — see LICENSE for the full text.

About

Agentic service that translates CI/CD pipeline definitions between Jenkins, GitLab CI, and GitHub Actions using a pluggable LLM (Gemini, OpenAI, Azure OpenAI, GitHub Models, Anthropic, or Ollama), LangGraph memory, and an MCP (Model Context Protocol) tool surface.

Topics

gemini openai cicd claude fastapi llm agentic-workflow agentic-ai

Resources

Readme

License

MIT license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 1

v0.0.1 — Initial release Latest
May 31, 2026

Packages

 
 
 

Contributors

Languages