Skip to content

Commit efe3848

Browse files
add blog on how to use starlark in Woodpecker CI
Signed-off-by: prashant-gurung899 <prasantgrg777@gmail.com>
1 parent 0c1007c commit efe3848

File tree

2 files changed

+138
-0
lines changed

2 files changed

+138
-0
lines changed
37.7 KB
Loading
Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
---
2+
title: Starlark in woodpecker CI
3+
authorName: Prashant Gurung
4+
authorAvatar: https://avatars.githubusercontent.com/u/53248463?v=4
5+
authorLink: https://github.com/prashant-gurung899
6+
createdAt: July 3, 2025
7+
tags: CI/CD
8+
banner: https://raw.githubusercontent.com/JankariTech/blog/woodpecker-ci-starlark/src/assets/woodpeckerCI/images/woodpecker.png
9+
---
10+
11+
## Background
12+
Continuous Integration (CI) tools are vital for automating the testing and deployment of modern software. One such open-source tool is Woodpecker CI, a lightweight CI/CD system. Woodpecker CI is an open-source continuous integration and delivery platform that helps developers automate building, testing, and deploying their code. It is a community-driven fork and successor of the popular Drone CI project, sharing many design principles and compatibility with Drone pipelines. Traditionally, Woodpecker pipelines are written in YAML. But with the rise of programmable pipelines, we now have the flexibility to define our CI configuration using Starlark — a Python-like configuration language.
13+
14+
In this blog, I’ll walk you through how I set up Woodpecker CI with a dynamic Starlark-to-YAML conversion service (WCCS), enabling us to write pipelines in .woodpecker.star files. Here's a high-level view of what we'll cover:
15+
16+
- Spinning up a Woodpecker server
17+
18+
- Authenticating with GitHub
19+
20+
- Enabling a repository
21+
22+
- Setting up WCCS (Woodpecker Config Conversion Service)
23+
24+
- Connecting WCCS with the Woodpecker server
25+
26+
By the end, you’ll be able to write CI pipelines in Starlark and dynamically convert them to YAML during runtime.
27+
28+
29+
## Setting Up Woodpecker Server
30+
We started by running the Woodpecker server using Docker. Here's the essential setup:
31+
32+
```yml
33+
version: '3'
34+
services:
35+
woodpecker-server:
36+
image: woodpeckerci/woodpecker-server:latest
37+
ports:
38+
- 8000:8000
39+
environment:
40+
- WOODPECKER_OPEN=true
41+
- WOODPECKER_HOST=https://<ngrok-or-your-domain>
42+
- WOODPECKER_GITHUB=true
43+
- WOODPECKER_GITHUB_CLIENT=<client-id>
44+
- WOODPECKER_GITHUB_SECRET=<client-secret>
45+
- WOODPECKER_AGENT_SECRET=supersecret
46+
- WOODPECKER_CONFIG_ENDPOINT=http://wccs:3000/ciconfig
47+
volumes:
48+
- woodpecker-data:/var/lib/woodpecker
49+
50+
woodpecker-agent:
51+
image: woodpeckerci/woodpecker-agent:latest
52+
depends_on:
53+
- woodpecker-server
54+
environment:
55+
- WOODPECKER_SERVER=http://woodpecker-server:8000
56+
- WOODPECKER_AGENT_SECRET=supersecret
57+
volumes:
58+
- /var/run/docker.sock:/var/run/docker.sock
59+
60+
volumes:
61+
woodpecker-data:
62+
```
63+
64+
## Authenticating with GitHub
65+
66+
To connect GitHub with Woodpecker:
67+
68+
1. Register a new OAuth application in your GitHub developer settings.
69+
70+
2. Set the callback URL to https://<your-ngrok-url>/authorize
71+
72+
3. Copy the generated Client ID and Secret.
73+
74+
4. Use these values in the WOODPECKER_GITHUB_CLIENT and WOODPECKER_GITHUB_SECRET env vars.
75+
76+
After that, you can log into the Woodpecker web UI using your GitHub account.
77+
78+
## Enabling a Repository
79+
From the Woodpecker UI:
80+
81+
- Select your GitHub repo.
82+
83+
- Enable it for CI.
84+
85+
Woodpecker will automatically add the necessary webhooks to the repo.
86+
87+
## Setting Up WCCS-Woodpecker CI Config Service (Starlark Conversion Service)
88+
The Woodpecker Config Conversion Service (WCCS) is a lightweight web service created and maintained by [Opencloud-eu](https://opencloud.eu/en). It enables Woodpecker CI to convert pipeline definitions written in Starlark into standard YAML on the fly by receiving a signed POST request from Woodpecker.
89+
You can easily deploy WCCS using their official Docker image available on Docker Hub: [opencloudeu/wccs](https://hub.docker.com/r/opencloudeu/wccs).
90+
```yml
91+
wccs:
92+
image: opencloudeu/wccs:latest
93+
ports:
94+
- 3000:3000
95+
command: server
96+
```
97+
98+
## Generating the Public Key
99+
Every request sent by Woodpecker is signed using a http-signature by a private key (ed25519) generated on the first start of the Woodpecker server. You can get the public key for the verification of the http-signature from:
100+
```console
101+
http(s)://your-woodpecker-server/api/signature/public-key
102+
```
103+
104+
## Connecting WCCS with Woodpecker
105+
To allow Woodpecker to fetch pipeline configs from WCCS, we added this to the server env:
106+
```console
107+
WOODPECKER_CONFIG_ENDPOINT=http://wccs:3000/ciconfig
108+
```
109+
Woodpecker now sends a signed JSON payload to WCCS whenever a build is triggered. WCCS verifies the signature using the public key and responds with a YAML pipeline based on your .woodpecker.star file.
110+
111+
## Testing WCCS Manually
112+
113+
To debug or test WCCS manually, we used curl:
114+
```console
115+
curl -v -X POST http://<wccs-server>:3000/ciconfig \
116+
-H "Content-Type: application/json" \
117+
-d '{}'
118+
```
119+
120+
## Sample Starlark Pipeline
121+
Here's a simple .woodpecker.star example:
122+
```console
123+
def main(ctx):
124+
return {
125+
"steps": [
126+
{
127+
"name": "hello",
128+
"image": "alpine",
129+
"commands": ["echo Hello from Starlark"]
130+
}
131+
]
132+
}
133+
```
134+
This gets converted by WCCS into a valid Woodpecker pipeline YAML.
135+
136+
Now, we can write pipelines in Starlark and let WCCS handle the conversion on the fly. This approach brings flexibility, structure, and the power of logic-based configurations to our CI pipelines.
137+
138+
Want to try it yourself? Check out Woodpecker's config service example and start building smarter pipelines today!

0 commit comments

Comments
 (0)