Skip to content

Commit 11b7eda

Browse files
add blog on how to use starlark in Woodpecker CI
Signed-off-by: prashant-gurung899 <prasantgrg777@gmail.com>
1 parent 0c1007c commit 11b7eda

File tree

1 file changed

+137
-0
lines changed

1 file changed

+137
-0
lines changed
Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
---
2+
title: Starlark in woodpecker CI
3+
authorName: Prashant Gurung
4+
authorAvatar: https://avatars.githubusercontent.com/u/53248463?v=4
5+
authorLink: https://github.com/prashant-gurung899
6+
createdAt: July 3, 2025
7+
tags: CI/CD
8+
banner: https://raw.githubusercontent.com/JankariTech/blog/master/src/imgs/fallback_banner.png
9+
---
10+
11+
## Background
12+
Continuous Integration (CI) tools are vital for automating the testing and deployment of modern software. One such open-source tool is Woodpecker CI, a lightweight CI/CD system. Traditionally, Woodpecker pipelines are written in YAML. But with the rise of programmable pipelines, we now have the flexibility to define our CI configuration using Starlark — a Python-like configuration language.
13+
14+
In this blog, I’ll walk you through how I set up Woodpecker CI with a dynamic Starlark-to-YAML conversion service (WCCS), enabling us to write pipelines in .drone.star files. Here's a high-level view of what we'll cover:
15+
16+
- Spinning up a Woodpecker server
17+
18+
- Authenticating with GitHub
19+
20+
- Enabling a repository
21+
22+
- Setting up WCCS (Woodpecker Config Conversion Service)
23+
24+
- Connecting WCCS with the Woodpecker server
25+
26+
By the end, you’ll be able to write CI pipelines in Starlark and dynamically convert them to YAML during runtime.
27+
28+
29+
## Setting Up Woodpecker Server
30+
We started by running the Woodpecker server using Docker. Here's the essential setup:
31+
32+
```yml
33+
version: '3'
34+
services:
35+
woodpecker-server:
36+
image: woodpeckerci/woodpecker-server:latest
37+
ports:
38+
- 8000:8000
39+
environment:
40+
- WOODPECKER_OPEN=true
41+
- WOODPECKER_HOST=https://<ngrok-or-your-domain>
42+
- WOODPECKER_GITHUB=true
43+
- WOODPECKER_GITHUB_CLIENT=<client-id>
44+
- WOODPECKER_GITHUB_SECRET=<client-secret>
45+
- WOODPECKER_AGENT_SECRET=supersecret
46+
- WOODPECKER_CONFIG_ENDPOINT=http://wccs:3000/ciconfig
47+
volumes:
48+
- woodpecker-data:/var/lib/woodpecker
49+
50+
woodpecker-agent:
51+
image: woodpeckerci/woodpecker-agent:latest
52+
depends_on:
53+
- woodpecker-server
54+
environment:
55+
- WOODPECKER_SERVER=http://woodpecker-server:8000
56+
- WOODPECKER_AGENT_SECRET=supersecret
57+
volumes:
58+
- /var/run/docker.sock:/var/run/docker.sock
59+
60+
volumes:
61+
woodpecker-data:
62+
```
63+
64+
## Authenticating with GitHub
65+
66+
To connect GitHub with Woodpecker:
67+
68+
1. Register a new OAuth application in your GitHub developer settings.
69+
70+
2. Set the callback URL to https://<your-ngrok-url>/authorize
71+
72+
3. Copy the generated Client ID and Secret.
73+
74+
4. Use these values in the WOODPECKER_GITHUB_CLIENT and WOODPECKER_GITHUB_SECRET env vars.
75+
76+
After that, you can log into the Woodpecker web UI using your GitHub account.
77+
78+
## Enabling a Repository
79+
From the Woodpecker UI:
80+
81+
- Select your GitHub repo.
82+
83+
- Enable it for CI.
84+
85+
Woodpecker will automatically add the necessary webhooks to the repo.
86+
87+
## Setting Up WCCS-Woodpecker CI Config Service (Starlark Conversion Service)
88+
WCCS is a small web service that receives a signed POST request from Woodpecker and returns a valid YAML pipeline. We deployed it with Docker:
89+
```yml
90+
wccs:
91+
image: opencloudeu/wccs:latest
92+
ports:
93+
- 3000:3000
94+
command: server
95+
```
96+
97+
## Generating the Public Key
98+
We started the Woodpecker server first, let it generate a key pair, and then copied the public key from the container or volume path:
99+
```console
100+
docker cp woodpecker-server:/var/lib/woodpecker/keys/public.pem ./woodpecker.pub
101+
```
102+
103+
## Connecting WCCS with Woodpecker
104+
To allow Woodpecker to fetch pipeline configs from WCCS, we added this to the server env:
105+
```console
106+
WOODPECKER_CONFIG_ENDPOINT=http://wccs:3000/ciconfig
107+
```
108+
Woodpecker now sends a signed JSON payload to WCCS whenever a build is triggered. WCCS verifies the signature using the public key and responds with a YAML pipeline based on your .woodpecker.star file.
109+
110+
## Testing WCCS Manually
111+
112+
To debug or test WCCS manually, we used curl:
113+
```console
114+
curl -v -X POST http://localhost:3000/ciconfig \
115+
-H "Content-Type: application/json" \
116+
-d '{"repo": {"name": "demo"}, "build": {"event": "push", "branch": "main"}}'
117+
```
118+
119+
## Sample Starlark Pipeline
120+
Here's a simple .woodpecker.star example:
121+
```console
122+
def main(ctx):
123+
return {
124+
"steps": [
125+
{
126+
"name": "hello",
127+
"image": "alpine",
128+
"commands": ["echo Hello from Starlark"]
129+
}
130+
]
131+
}
132+
```
133+
This gets converted by WCCS into a valid Woodpecker pipeline YAML.
134+
135+
Now, we can write pipelines in Starlark and let WCCS handle the conversion on the fly. This approach brings flexibility, structure, and the power of logic-based configurations to our CI pipelines.
136+
137+
Want to try it yourself? Check out Woodpecker's config service example and start building smarter pipelines today!

0 commit comments

Comments
 (0)