diff --git a/neon-branching-tutorial/.gitignore b/neon-branching-tutorial/.gitignore new file mode 100644 index 0000000..092d53f --- /dev/null +++ b/neon-branching-tutorial/.gitignore @@ -0,0 +1,40 @@ +# Dependencies +node_modules/ + +# Environment variables +.env +.env.local +.env.*.local + +# Logs +logs/ +*.log +npm-debug.log* + +# Runtime data +pids/ +*.pid +*.seed +*.pid.lock + +# Coverage directory +coverage/ + +# Build output +dist/ +build/ + +# IDE and editor files +.idea/ +.vscode/ +*.swp +*.swo +*~ +.DS_Store + +# Kubernetes secrets (never commit these!) +*-secret.yaml +db-secret.yaml + +# Neon credentials +.neonctl \ No newline at end of file diff --git a/neon-branching-tutorial/Makefile b/neon-branching-tutorial/Makefile new file mode 100644 index 0000000..fb2fdfb --- /dev/null +++ b/neon-branching-tutorial/Makefile @@ -0,0 +1,29 @@ +.PHONY: build deploy clean setup-neon-secret + +IMAGE_NAME := signadot/neon-demo-users:latest + +build: + docker build -t $(IMAGE_NAME) -f ./docker/users-service.Dockerfile ./pkg/users-service + +deploy: + kubectl apply -f ./k8s + +clean: + kubectl delete -f ./k8s --ignore-not-found + +setup-neon-secret: + @echo "Creating Neon API credentials secret in signadot namespace..." + @read -p "Enter your Neon API key: " NEON_API_KEY; \ + kubectl create secret generic neon-api-credentials \ + --namespace=signadot \ + --from-literal=NEON_API_KEY=$$NEON_API_KEY \ + --dry-run=client -o yaml | kubectl apply -f - + +setup-db-secret: + @echo "Creating database credentials secret..." + @read -p "Enter your Neon connection string: " DATABASE_URL; \ + kubectl create secret generic users-db-credentials \ + --from-literal=DATABASE_URL="$$DATABASE_URL" \ + --dry-run=client -o yaml | kubectl apply -f - + +all: build deploy \ No newline at end of file diff --git a/neon-branching-tutorial/README.md b/neon-branching-tutorial/README.md new file mode 100644 index 0000000..5ab53c6 --- /dev/null +++ b/neon-branching-tutorial/README.md @@ -0,0 +1,347 @@ +# True "Branch-Based Environments": Combining Signadot Sandboxes with Neon DB Branching + +Ephemeral sandbox environments solve many problems for microservices teams. You can spin up an isolated copy of your service, test your changes, and tear it down. No conflicts with other developers. No waiting for a shared staging slot. + +But here's the catch: your sandbox service still connects to the same staging database as everyone else. One developer's test writes pollute another's queries. Schema migrations break active tests. Seed data disappears mid-run. The application layer is isolated, but the data layer is not. + +This guide shows you how to fix that problem. You will combine Signadot Sandboxes with Neon's database branching to create true full-stack isolation. Every sandbox gets its own application fork and its own database branch. When the sandbox dies, the database branch dies with it. + +## What You Will Build + +The end-to-end system works as follows: + +1. A developer creates a Signadot Sandbox +2. A Resource Plugin automatically creates a Neon database branch and exposes the connection string as an output +3. The sandbox pod starts with a connection string pointing to the isolated branch +4. The developer runs tests against isolated data +5. The developer deletes the sandbox +6. The Resource Plugin deletes the Neon branch automatically + +No shared state. No test pollution. No manual cleanup scripts. + +## How It Works + +The architecture relies on two key technologies working together. + +**Neon Database Branching**: Neon uses copy-on-write storage to create instant database branches. A branch inherits all schema and data from its parent but operates independently. Writes to a branch don't affect the parent, and branches can be created or deleted in seconds with minimal storage overhead. + +**Signadot Resource Plugins**: Resource Plugins extend Signadot's sandbox lifecycle with custom provisioning logic. When a sandbox starts, the plugin runs a create workflow. When the sandbox terminates, the plugin runs a delete workflow. Outputs from the create workflow (like connection strings) can be injected directly into sandbox pods. + + +## Prerequisites + +Before you begin, ensure you have: + +- `kubectl` and `minikube` installed +- A [Neon account](https://neon.tech) with an API key +- The `neonctl` CLI installed and authenticated +- A [Signadot account](https://www.signadot.com/) with the operator installed in your cluster +- The `signadot` CLI installed and authenticated + +## Baseline Environment + +We'll set up a users microservice connected to a Neon database, then demonstrate how sandboxes can get isolated database branches. + +### Step 1: Clone the Example Repository + +The example repository contains a pre-built users microservice and all necessary Kubernetes manifests: + +```bash +mkdir -p ~/git/signadot/ +cd ~/git/signadot/ +git clone https://github.com/signadot/examples.git +cd examples/neon-branching-tutorial +``` + +### Step 2: Set Up the Neon Database + +Create a Neon project: + +```bash +neonctl projects create --name users-demo +``` +![Create a Neon project](./images/img-001.png) + +Note the project ID from the output (e.g., `sparkling-queen-66410086`). You'll need it throughout this tutorial. + +Retrieve the connection string and create the schema: + +```bash +neonctl connection-string main \ + --project-id \ + --database-name neondb +``` + +Connect to the database and run the schema file: + +```bash +psql "" -f schema.sql +``` + +The `schema.sql` file creates a `users` table and inserts three seed records. Every sandbox branch will inherit this data. + +Generate an API key for the Resource Plugin: + +1. Go to the [Neon Console](https://console.neon.tech/) +2. Navigate to **Account Settings > Personal API keys** +3. Click **Create new API key** and save it securely + +### Step 3: Deploy to Minikube + +Start minikube and build the demo image: + +```bash +minikube start + +eval $(minikube docker-env) +make build +``` + +Create the required secrets: + +```bash +make setup-db-secret # Enter your Neon connection string when prompted +make setup-neon-secret # Enter your Neon API key when prompted +``` + +Deploy the baseline service: + +```bash +make deploy +``` + +Verify the deployment: + +```bash +kubectl get pods -l app=users-service +``` +![Verify the deployment](./images/img-002.png) + +You should see pods in `Running` state with `2/2` containers (the service plus the Signadot routing sidecar). + +### Step 4: Install the Resource Plugin + +The Resource Plugin bridges Signadot and Neon. Take a look at `neon-branch-plugin.yaml`: + +```yaml +name: neon-branch +spec: + description: Creates and deletes Neon database branches for sandbox isolation + + runner: + image: node:20-alpine + namespace: signadot + podTemplateOverlay: | + spec: + containers: + - name: main + env: + - name: NEON_API_KEY + valueFrom: + secretKeyRef: + name: neon-api-credentials + key: NEON_API_KEY + + create: + - name: createbranch + inputs: + - name: project-id + valueFromSandbox: true + as: + env: NEON_PROJECT_ID + - name: parent-branch + valueFromSandbox: true + as: + env: PARENT_BRANCH + - name: database-name + valueFromSandbox: true + as: + env: DATABASE_NAME + script: | + #!/bin/sh + set -e + npm install -g neonctl + + SAFE_NAME=$(echo "${SIGNADOT_SANDBOX_NAME}" | tr -d '-') + BRANCH_NAME="sandbox${SAFE_NAME}" + + neonctl branches create \ + --project-id "${NEON_PROJECT_ID}" \ + --name "${BRANCH_NAME}" \ + --parent "${PARENT_BRANCH}" \ + --output json > /tmp/branch-output.json + + BRANCH_ID=$(cat /tmp/branch-output.json | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4) + + CONNECTION_STRING=$(neonctl connection-string "${BRANCH_NAME}" \ + --project-id "${NEON_PROJECT_ID}" \ + --database-name "${DATABASE_NAME}") + + mkdir -p /outputs + echo -n "${BRANCH_NAME}" > /outputs/branch-name + echo -n "${BRANCH_ID}" > /outputs/branch-id + echo -n "${CONNECTION_STRING}" > /outputs/connection-string + + outputs: + - name: branch-name + valueFromPath: /outputs/branch-name + - name: branch-id + valueFromPath: /outputs/branch-id + - name: connection-string + valueFromPath: /outputs/connection-string + + delete: + - name: deletebranch + inputs: + - name: project-id + valueFromSandbox: true + as: + env: NEON_PROJECT_ID + - name: branch-name + valueFromStep: + name: createbranch + output: branch-name + as: + env: BRANCH_NAME + script: | + #!/bin/sh + set -e + npm install -g neonctl + neonctl branches delete "${BRANCH_NAME}" --project-id "${NEON_PROJECT_ID}" +``` + +The plugin has three main sections: + +- **runner**: Uses `node:20-alpine` with the Neon API key injected via `podTemplateOverlay`. The runner executes in the `signadot` namespace where the API key secret exists. +- **create**: Installs `neonctl`, creates a branch named after the sandbox, retrieves the connection string, and exposes it as an output. The script sanitizes the sandbox name by removing hyphens since Neon branch names work best with alphanumeric characters. +- **delete**: Reads the branch name from the create step's output (using `valueFromStep`) and deletes it. + +Apply the plugin: + +```bash +signadot resourceplugin apply -f neon-branch-plugin.yaml +``` + +### Step 5: Configure the Sandbox Specification + +The sandbox spec ties everything together. Review `users-sandbox.yaml`: + +```yaml +name: "@{sandbox-name}" +spec: + description: "Users service sandbox with isolated Neon database branch" + cluster: "@{cluster}" + + resources: + - name: usersDb + plugin: neon-branch + params: + project-id: "@{neon-project-id}" + parent-branch: "main" + database-name: "neondb" + + forks: + - forkOf: + kind: Deployment + namespace: default + name: users-service + customizations: + env: + - name: DATABASE_URL + valueFrom: + resource: + name: usersDb + outputKey: createbranch.connection-string + + defaultRouteGroup: + endpoints: + - name: users-api + target: http://users-service.default.svc:3000 +``` + +The key sections: + +- **resources**: Invokes the `neon-branch` plugin with project parameters passed at apply time. +- **forks**: Creates a copy of the `users-service` Deployment with the `DATABASE_URL` overridden. The `valueFrom.resource` field references the plugin output directly using the format `.`. No intermediate Kubernetes Secret is required. +- **defaultRouteGroup**: Creates a preview URL for accessing the sandboxed service. + +## Using Sandboxes + +Create a sandbox with an isolated database branch: + +```bash +signadot sandbox apply -f users-sandbox.yaml \ + --set sandbox-name=my-feature \ + --set cluster= \ + --set neon-project-id= +``` + +### Verify Branch Creation + +Check the Neon branches: + +```bash +neonctl branches list --project-id +``` + +![Check the Neon branches](./images/img-003.png) + +You should see both `main` and `sandboxmyfeature` branches. + +### Test Data Isolation + +Query the sandbox endpoint to see the inherited seed data: + +```bash +curl -H "signadot-api-key: " \ + "https://users-api--my-feature.preview.signadot.com/users" +``` + +![Query the sandbox](./images/img-004.png) + +Create a test user in the sandbox: + +```bash +curl -X POST \ + -H "signadot-api-key: " \ + -H "Content-Type: application/json" \ + -d '{"name": "Sandbox User", "email": "sandbox@test.example"}' \ + "https://users-api--my-feature.preview.signadot.com/users" +``` + +![Create a test user](./images/img-005.png) + +Verify the main branch remains unaffected: + +```bash +neonctl connection-string main --project-id --database-name neondb +psql "" -c "SELECT * FROM users WHERE email = 'sandbox@test.example';" +``` + +![Verify the main branch](./images/img-006.png) + +The query returns zero rows. The sandbox user exists only in the branch. + +### Cleanup + +Delete the sandbox: + +```bash +signadot sandbox delete my-feature +``` + +The Resource Plugin's delete workflow automatically removes the Neon branch: + +```bash +neonctl branches list --project-id +``` + +![Verify cleanup](./images/img-007.png) + +Only the `main` branch remains. + +## Conclusion + +Each Signadot Sandbox now gets its own forked microservice pods and its own isolated Neon database branch. The Resource Plugin handles the entire lifecycle: creating branches on sandbox creation, exposing connection strings through built-in outputs, and cleaning them up on deletion. Test data cannot leak between sandboxes, and schema migrations in one branch cannot break tests in another. + +The cost efficiency makes this practical for everyday use. Neon branches use copy-on-write storage, so you only pay for data that changes. Signadot sandboxes share baseline cluster resources. Branch creation and teardown complete in seconds. Every developer gets an isolated app and database for every pull request. \ No newline at end of file diff --git a/neon-branching-tutorial/build.sh b/neon-branching-tutorial/build.sh new file mode 100644 index 0000000..15c7ba8 --- /dev/null +++ b/neon-branching-tutorial/build.sh @@ -0,0 +1,12 @@ +#!/bin/bash +set -e + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$SCRIPT_DIR" + +echo "Building users-service image..." +docker build -t signadot/neon-demo-users:latest -f ./docker/users-service.Dockerfile ./pkg/users-service + +echo "Build complete!" +echo "Images built:" +docker images | grep signadot/neon-demo \ No newline at end of file diff --git a/neon-branching-tutorial/docker/users-service.Dockerfile b/neon-branching-tutorial/docker/users-service.Dockerfile new file mode 100644 index 0000000..11a50d1 --- /dev/null +++ b/neon-branching-tutorial/docker/users-service.Dockerfile @@ -0,0 +1,22 @@ +FROM node:20-alpine + +WORKDIR /app + +# Copy package files first for better layer caching +COPY package*.json ./ + +# Install dependencies +RUN npm install --production + +# Copy application code +COPY app.js ./ + +# Create non-root user for security +RUN addgroup -g 1001 -S nodejs && \ + adduser -S nodejs -u 1001 + +USER nodejs + +EXPOSE 3000 + +CMD ["npm", "start"] \ No newline at end of file diff --git a/neon-branching-tutorial/images/img-001.png b/neon-branching-tutorial/images/img-001.png new file mode 100644 index 0000000..7e962c2 Binary files /dev/null and b/neon-branching-tutorial/images/img-001.png differ diff --git a/neon-branching-tutorial/images/img-002.png b/neon-branching-tutorial/images/img-002.png new file mode 100644 index 0000000..6d9b0cf Binary files /dev/null and b/neon-branching-tutorial/images/img-002.png differ diff --git a/neon-branching-tutorial/images/img-003.png b/neon-branching-tutorial/images/img-003.png new file mode 100644 index 0000000..d5ad00c Binary files /dev/null and b/neon-branching-tutorial/images/img-003.png differ diff --git a/neon-branching-tutorial/images/img-004.png b/neon-branching-tutorial/images/img-004.png new file mode 100644 index 0000000..b6957df Binary files /dev/null and b/neon-branching-tutorial/images/img-004.png differ diff --git a/neon-branching-tutorial/images/img-005.png b/neon-branching-tutorial/images/img-005.png new file mode 100644 index 0000000..874a096 Binary files /dev/null and b/neon-branching-tutorial/images/img-005.png differ diff --git a/neon-branching-tutorial/images/img-006.png b/neon-branching-tutorial/images/img-006.png new file mode 100644 index 0000000..b13dbd4 Binary files /dev/null and b/neon-branching-tutorial/images/img-006.png differ diff --git a/neon-branching-tutorial/images/img-007.png b/neon-branching-tutorial/images/img-007.png new file mode 100644 index 0000000..e886320 Binary files /dev/null and b/neon-branching-tutorial/images/img-007.png differ diff --git a/neon-branching-tutorial/k8s/deployment.yaml b/neon-branching-tutorial/k8s/deployment.yaml new file mode 100644 index 0000000..a80a392 --- /dev/null +++ b/neon-branching-tutorial/k8s/deployment.yaml @@ -0,0 +1,50 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: users-service + namespace: default + labels: + app: users-service +spec: + replicas: 1 + selector: + matchLabels: + app: users-service + template: + metadata: + labels: + app: users-service + annotations: + sidecar.signadot.com/inject: "true" + spec: + containers: + - name: users-service + image: signadot/neon-demo-users:latest + imagePullPolicy: Never + ports: + - containerPort: 3000 + env: + - name: DATABASE_URL + valueFrom: + secretKeyRef: + name: users-db-credentials + key: DATABASE_URL + resources: + requests: + memory: "128Mi" + cpu: "100m" + limits: + memory: "256Mi" + cpu: "500m" + livenessProbe: + httpGet: + path: /health + port: 3000 + initialDelaySeconds: 10 + periodSeconds: 10 + readinessProbe: + httpGet: + path: /health + port: 3000 + initialDelaySeconds: 5 + periodSeconds: 5 \ No newline at end of file diff --git a/neon-branching-tutorial/k8s/service.yaml b/neon-branching-tutorial/k8s/service.yaml new file mode 100644 index 0000000..ca0c21b --- /dev/null +++ b/neon-branching-tutorial/k8s/service.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: Service +metadata: + name: users-service + namespace: default +spec: + selector: + app: users-service + ports: + - port: 3000 + targetPort: 3000 \ No newline at end of file diff --git a/neon-branching-tutorial/neon-branch-plugin.yaml b/neon-branching-tutorial/neon-branch-plugin.yaml new file mode 100644 index 0000000..540977d --- /dev/null +++ b/neon-branching-tutorial/neon-branch-plugin.yaml @@ -0,0 +1,110 @@ +name: neon-branch +spec: + description: Creates and deletes Neon database branches for sandbox isolation + + runner: + image: node:20-alpine + namespace: signadot + podTemplateOverlay: | + spec: + containers: + - name: main + env: + - name: NEON_API_KEY + valueFrom: + secretKeyRef: + name: neon-api-credentials + key: NEON_API_KEY + + create: + - name: createbranch + inputs: + - name: project-id + valueFromSandbox: true + as: + env: NEON_PROJECT_ID + - name: parent-branch + valueFromSandbox: true + as: + env: PARENT_BRANCH + - name: database-name + valueFromSandbox: true + as: + env: DATABASE_NAME + script: | + #!/bin/sh + set -e + + # Install neonctl + npm install -g neonctl + + # Generate a unique branch name using the sandbox name + # Remove any hyphens from sandbox name for Neon branch naming + SAFE_NAME=$(echo "${SIGNADOT_SANDBOX_NAME}" | tr -d '-') + BRANCH_NAME="sandbox${SAFE_NAME}" + + echo "Creating Neon branch: ${BRANCH_NAME}" + echo "Project ID: ${NEON_PROJECT_ID}" + echo "Parent branch: ${PARENT_BRANCH}" + + # Create the branch + neonctl branches create \ + --project-id "${NEON_PROJECT_ID}" \ + --name "${BRANCH_NAME}" \ + --parent "${PARENT_BRANCH}" \ + --output json > /tmp/branch-output.json + + cat /tmp/branch-output.json + + # Extract the branch ID + BRANCH_ID=$(cat /tmp/branch-output.json | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4) + echo "Created branch ID: ${BRANCH_ID}" + + # Get the connection string for the new branch + CONNECTION_STRING=$(neonctl connection-string "${BRANCH_NAME}" \ + --project-id "${NEON_PROJECT_ID}" \ + --database-name "${DATABASE_NAME}") + + echo "Connection string retrieved successfully" + + # Write outputs to files for Signadot to capture + mkdir -p /outputs + echo -n "${BRANCH_NAME}" > /outputs/branch-name + echo -n "${BRANCH_ID}" > /outputs/branch-id + echo -n "${CONNECTION_STRING}" > /outputs/connection-string + + echo "Branch creation complete" + + outputs: + - name: branch-name + valueFromPath: /outputs/branch-name + - name: branch-id + valueFromPath: /outputs/branch-id + - name: connection-string + valueFromPath: /outputs/connection-string + + delete: + - name: deletebranch + inputs: + - name: project-id + valueFromSandbox: true + as: + env: NEON_PROJECT_ID + - name: branch-name + valueFromStep: + name: createbranch + output: branch-name + as: + env: BRANCH_NAME + script: | + #!/bin/sh + set -e + + # Install neonctl + npm install -g neonctl + + echo "Deleting Neon branch: ${BRANCH_NAME}" + neonctl branches delete "${BRANCH_NAME}" \ + --project-id "${NEON_PROJECT_ID}" + + echo "Cleanup complete" \ No newline at end of file diff --git a/neon-branching-tutorial/pkg/users-service/app.js b/neon-branching-tutorial/pkg/users-service/app.js new file mode 100644 index 0000000..891abd1 --- /dev/null +++ b/neon-branching-tutorial/pkg/users-service/app.js @@ -0,0 +1,78 @@ +const express = require('express'); +const { Pool } = require('pg'); + +const app = express(); +app.use(express.json()); + +// Read connection string from environment variable +const pool = new Pool({ + connectionString: process.env.DATABASE_URL, + ssl: { rejectUnauthorized: false }, +}); + +// Health check endpoint +app.get('/health', async (req, res) => { + try { + await pool.query('SELECT 1'); + res.json({ status: 'healthy' }); + } catch (err) { + res.status(500).json({ status: 'unhealthy', error: err.message }); + } +}); + +// Get all users +app.get('/users', async (req, res) => { + try { + const result = await pool.query('SELECT * FROM users ORDER BY id'); + res.json(result.rows); + } catch (err) { + res.status(500).json({ error: err.message }); + } +}); + +// Get user by ID +app.get('/users/:id', async (req, res) => { + try { + const { id } = req.params; + const result = await pool.query('SELECT * FROM users WHERE id = $1', [id]); + if (result.rows.length === 0) { + return res.status(404).json({ error: 'User not found' }); + } + res.json(result.rows[0]); + } catch (err) { + res.status(500).json({ error: err.message }); + } +}); + +// Create a user +app.post('/users', async (req, res) => { + try { + const { name, email } = req.body; + const result = await pool.query( + 'INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *', + [name, email] + ); + res.status(201).json(result.rows[0]); + } catch (err) { + res.status(500).json({ error: err.message }); + } +}); + +// Delete a user +app.delete('/users/:id', async (req, res) => { + try { + const { id } = req.params; + const result = await pool.query('DELETE FROM users WHERE id = $1 RETURNING *', [id]); + if (result.rows.length === 0) { + return res.status(404).json({ error: 'User not found' }); + } + res.json({ message: 'User deleted', user: result.rows[0] }); + } catch (err) { + res.status(500).json({ error: err.message }); + } +}); + +const PORT = process.env.PORT || 3000; +app.listen(PORT, () => { + console.log(`Users service running on port ${PORT}`); +}); \ No newline at end of file diff --git a/neon-branching-tutorial/pkg/users-service/package.json b/neon-branching-tutorial/pkg/users-service/package.json new file mode 100644 index 0000000..0a82e21 --- /dev/null +++ b/neon-branching-tutorial/pkg/users-service/package.json @@ -0,0 +1,16 @@ +{ + "name": "users-service", + "version": "1.0.0", + "description": "Users microservice for Signadot + Neon branching demo", + "main": "app.js", + "scripts": { + "start": "node app.js" + }, + "dependencies": { + "express": "^4.18.2", + "pg": "^8.11.3" + }, + "engines": { + "node": ">=18.0.0" + } +} \ No newline at end of file diff --git a/neon-branching-tutorial/schema.sql b/neon-branching-tutorial/schema.sql new file mode 100644 index 0000000..6c01195 --- /dev/null +++ b/neon-branching-tutorial/schema.sql @@ -0,0 +1,16 @@ +-- Schema setup for Neon branching tutorial +-- Run this against your Neon database main branch + +CREATE TABLE IF NOT EXISTS users ( + id SERIAL PRIMARY KEY, + name VARCHAR(100) NOT NULL, + email VARCHAR(255) UNIQUE NOT NULL, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +); + +-- Seed data +INSERT INTO users (name, email) VALUES + ('Alice Johnson', 'alice@example.com'), + ('Bob Smith', 'bob@example.com'), + ('Carol Williams', 'carol@example.com') +ON CONFLICT (email) DO NOTHING; \ No newline at end of file diff --git a/neon-branching-tutorial/users-sandbox.yaml b/neon-branching-tutorial/users-sandbox.yaml new file mode 100644 index 0000000..8d378e5 --- /dev/null +++ b/neon-branching-tutorial/users-sandbox.yaml @@ -0,0 +1,34 @@ +name: "@{sandbox-name}" +spec: + description: "Users service sandbox with isolated Neon database branch" + cluster: "@{cluster}" + + labels: + team: platform + service: users-service + + resources: + - name: usersDb + plugin: neon-branch + params: + project-id: "@{neon-project-id}" + parent-branch: "main" + database-name: "neondb" + + forks: + - forkOf: + kind: Deployment + namespace: default + name: users-service + customizations: + env: + - name: DATABASE_URL + valueFrom: + resource: + name: usersDb + outputKey: createbranch.connection-string + + defaultRouteGroup: + endpoints: + - name: users-api + target: http://users-service.default.svc:3000 \ No newline at end of file diff --git a/xata-branching-tutorial/.gitignore b/xata-branching-tutorial/.gitignore new file mode 100644 index 0000000..0b70d7d --- /dev/null +++ b/xata-branching-tutorial/.gitignore @@ -0,0 +1,44 @@ +# Dependencies +node_modules/ +package-lock.json + +# Xata credentials and config +.xata/ + +# Environment files +.env +.env.local +.env.*.local +*.env + +# IDE and editor +.idea/ +.vscode/ +*.swp +*.swo +*~ +.DS_Store + +# Logs +*.log +npm-debug.log* +yarn-debug.log* +yarn-error.log* + +# Build outputs +dist/ +build/ +*.tgz + +# Test coverage +coverage/ +.nyc_output/ + +# Temporary files +tmp/ +temp/ +*.tmp + +# Kubernetes secrets (if accidentally created locally) +*-secret.yaml +secrets/ diff --git a/xata-branching-tutorial/Makefile b/xata-branching-tutorial/Makefile new file mode 100644 index 0000000..29014d7 --- /dev/null +++ b/xata-branching-tutorial/Makefile @@ -0,0 +1,140 @@ +.PHONY: help setup build deploy plugin-secret sandbox-create sandbox-delete test clean db-setup db-connect + +help: + @echo "Xata + Signadot Database Branching Tutorial" + @echo "" + @echo "Setup:" + @echo " make setup - Check prerequisites" + @echo " make build - Build Docker image" + @echo " make deploy - Deploy to Kubernetes" + @echo " make db-setup - Create schema and seed data" + @echo "" + @echo "Signadot:" + @echo " make plugin-secret - Create secret and install Resource Plugin" + @echo " make sandbox-create - Create test sandbox" + @echo " make sandbox-delete - Delete test sandbox" + @echo "" + @echo "Utility:" + @echo " make test - Test baseline deployment" + @echo " make db-connect - Connect to database with psql" + @echo " make clean - Remove all resources" + +setup: + @echo "Checking prerequisites..." + @command -v kubectl >/dev/null 2>&1 || { echo "ERROR: kubectl not found"; exit 1; } + @command -v xata >/dev/null 2>&1 || { echo "ERROR: xata CLI not found"; exit 1; } + @command -v signadot >/dev/null 2>&1 || { echo "ERROR: signadot CLI not found"; exit 1; } + @command -v docker >/dev/null 2>&1 || { echo "ERROR: docker not found"; exit 1; } + @command -v psql >/dev/null 2>&1 || { echo "ERROR: psql not found"; exit 1; } + @command -v jq >/dev/null 2>&1 || { echo "ERROR: jq not found"; exit 1; } + @echo "All prerequisites found." + @echo "" + @xata auth status >/dev/null 2>&1 || { echo "ERROR: Not authenticated with Xata. Run 'xata auth login'"; exit 1; } + @echo "Xata: authenticated" + @signadot cluster list >/dev/null 2>&1 || { echo "ERROR: Not authenticated with Signadot. Run 'signadot auth login'"; exit 1; } + @echo "Signadot: authenticated" + @echo "" + @echo "Setup complete!" + +build: + @echo "Building Docker image..." + docker build -t users-service:latest -f docker/users-service.Dockerfile . + @echo "Build complete." + +deploy: + @echo "Deploying to Kubernetes..." + @test -f .xata/project.json || { echo "ERROR: .xata/project.json not found. Run 'xata init' first."; exit 1; } + kubectl apply -f k8s/namespace.yaml + kubectl delete secret db-creds -n demo --ignore-not-found + kubectl create secret generic db-creds \ + --namespace=demo \ + --from-literal=DATABASE_URL="$$(xata branch url)" + kubectl apply -f k8s/deployment.yaml + kubectl apply -f k8s/service.yaml + @echo "Waiting for deployment..." + kubectl rollout status deployment/users-service -n demo --timeout=120s + @echo "Deployment complete." + +db-setup: + @echo "Setting up database..." + @test -f .xata/project.json || { echo "ERROR: .xata/project.json not found. Run 'xata init' first."; exit 1; } + psql "$$(xata branch url)" -f db/schema.sql + psql "$$(xata branch url)" -f db/seed.sql + @echo "Database setup complete." + +db-connect: + @psql "$$(xata branch url)" + +plugin-secret: + @echo "Creating Xata credentials secret..." + @test -f .xata/project.json || { echo "ERROR: .xata/project.json not found. Run 'xata init' first."; exit 1; } + @test -f .xata/branch.json || { echo "ERROR: .xata/branch.json not found. Run 'xata init' first."; exit 1; } + @if [ -z "$${XATA_API_KEY}" ]; then \ + echo "XATA_API_KEY not set."; \ + echo "Generate one at https://app.xata.io/settings/api-keys"; \ + echo ""; \ + read -p "Enter your Xata API key: " XATA_API_KEY; \ + export XATA_API_KEY; \ + fi && \ + echo "" && \ + echo "Reading configuration..." && \ + XATA_ORG_ID="$$(jq -r '.organizationId' .xata/project.json)" && \ + XATA_PROJECT_ID="$$(jq -r '.projectId' .xata/project.json)" && \ + XATA_DATABASE_NAME="$$(jq -r '.databaseName' .xata/branch.json)" && \ + echo "XATA_ORG_ID=$$XATA_ORG_ID" && \ + echo "XATA_PROJECT_ID=$$XATA_PROJECT_ID" && \ + echo "XATA_DATABASE_NAME=$$XATA_DATABASE_NAME" && \ + echo "XATA_API_KEY=(set)" && \ + echo "" && \ + kubectl create namespace signadot --dry-run=client -o yaml | kubectl apply -f - && \ + kubectl delete secret xata-credentials -n signadot --ignore-not-found && \ + kubectl create secret generic xata-credentials \ + --namespace=signadot \ + --from-literal=XATA_API_KEY="$$XATA_API_KEY" \ + --from-literal=XATA_ORG_ID="$$XATA_ORG_ID" \ + --from-literal=XATA_PROJECT_ID="$$XATA_PROJECT_ID" \ + --from-literal=XATA_DATABASE_NAME="$$XATA_DATABASE_NAME" + @echo "" + @echo "Secret created." + @echo "" + @echo "Installing Resource Plugin..." + signadot resourceplugin delete xata-branch 2>/dev/null || true + signadot resourceplugin apply -f signadot/xata-branch-plugin.yaml + @echo "Plugin installed." + +sandbox-create: + @echo "Creating test sandbox..." + @CLUSTER="$$(signadot cluster list -o json | jq -r '.[0].name')" && \ + echo "Cluster: $$CLUSTER" && \ + signadot sandbox apply -f signadot/users-sandbox.yaml \ + --set sandbox-name=xata-test \ + --set cluster="$$CLUSTER" + +sandbox-delete: + @echo "Deleting test sandbox..." + signadot sandbox delete xata-test + +test: + @echo "Testing baseline deployment..." + @kubectl port-forward svc/users-service 3000:3000 -n demo & + @sleep 3 + @echo "" + @echo "=== Health ===" + @curl -s http://localhost:3000/health | jq . + @echo "" + @echo "=== Users ===" + @curl -s http://localhost:3000/users | jq . + @echo "" + @echo "=== Info ===" + @curl -s http://localhost:3000/info | jq . + @echo "" + @pkill -f "port-forward.*3000" || true + +clean: + @echo "Cleaning up..." + -signadot sandbox delete xata-test 2>/dev/null + -signadot resourceplugin delete xata-branch 2>/dev/null + -kubectl delete secret xata-credentials -n signadot --ignore-not-found + -kubectl delete -f k8s/ --ignore-not-found + -kubectl delete secret db-creds -n demo --ignore-not-found + @echo "Cleanup complete." \ No newline at end of file diff --git a/xata-branching-tutorial/README.md b/xata-branching-tutorial/README.md new file mode 100644 index 0000000..ef726e4 --- /dev/null +++ b/xata-branching-tutorial/README.md @@ -0,0 +1,454 @@ +# True "Branch-Based Environments": Combining Signadot Sandboxes with Xata Database Branching + +Ephemeral environments solve the "it works on my machine" problem for application code. But what about the database? When five developers test against the same staging database, test data collides, schema migrations conflict, and debugging becomes a nightmare. The environment might be ephemeral, but the shared database state is not. + +Xata solves this with copy-on-write database branching. Each branch inherits the full schema and data from its parent but operates in complete isolation. Writes to a branch never affect the parent. Combined with Signadot's sandbox isolation for application workloads, you get true full-stack isolation: every sandbox gets its own forked pods *and* its own database branch. + +This tutorial demonstrates how to automate this integration. When a Signadot Sandbox spins up, a Resource Plugin creates an isolated Xata branch. When the sandbox terminates, the branch is deleted. No manual intervention. No leftover resources. + +## How It Works + +The integration relies on two components: + +**Xata Database Branching**: Xata implements branching at the storage layer using copy-on-write semantics. Creating a branch takes seconds regardless of database size because no data is physically copied until a write occurs. Only diverging blocks are duplicated. A 100GB database branches just as fast as a 1GB one. + +**Signadot Resource Plugins**: Resource Plugins extend Signadot's sandbox lifecycle with custom provisioning logic. When a sandbox starts, the plugin executes a `create` workflow. When the sandbox terminates, it executes a `delete` workflow. Outputs from the `create` workflow (such as database connection strings) can be injected directly into sandbox pods as environment variables. + +**The Integration Flow**: + +```mermaid +flowchart TB + subgraph SIGNADOT["Signadot"] + SC[Sandbox Controller] + RP[Resource Plugin] + end + + subgraph K8S["Kubernetes Cluster"] + BASE[Baseline Pod] + FORK[Forked Pod] + end + + subgraph XATA["Xata"] + MAIN[(main branch)] + SB[(sandbox branch)] + end + + SC --> RP + RP --> SB + SC --> FORK + BASE --> MAIN + FORK --> SB + MAIN -.->|copy-on-write| SB + + style SIGNADOT fill:#fff3e0,stroke:#e65100,stroke-width:2px + style K8S fill:#e3f2fd,stroke:#1565c0,stroke-width:2px + style XATA fill:#fce4ec,stroke:#c2185b,stroke-width:2px + style SC fill:#e65100,color:#ffffff + style RP fill:#f57c00,color:#ffffff + style BASE fill:#1565c0,color:#ffffff + style FORK fill:#1976d2,color:#ffffff + style MAIN fill:#c2185b,color:#ffffff + style SB fill:#d81b60,color:#ffffff + +``` + +## Prerequisites + +You need: + +- A Kubernetes cluster (minikube works fine for this tutorial) +- `kubectl` configured to access your cluster +- A [Xata account](https://xata.io/get-access) with a database created +- A [Signadot account](https://app.signadot.com/) with the operator installed +- The `xata` CLI installed and authenticated +- The `signadot` CLI installed and authenticated +- `psql` (PostgreSQL client) for database operations + +Xata requires initial database creation through the dashboard. Once that exists, all subsequent branch operations work via CLI. + +## Project Structure + +Clone the example repository: + +```bash +git clone https://github.com/signadot/xata-branching-tutorial.git +cd xata-branching-tutorial +``` + +The repository contains everything needed for this integration: + +``` +xata-branching-tutorial/ +├── db/ +│ ├── schema.sql # Table definitions +│ └── seed.sql # Sample data +├── pkg/users-service/ +│ ├── app.js # Express.js API +│ └── package.json +├── docker/ +│ └── users-service.Dockerfile +├── k8s/ +│ ├── namespace.yaml +│ ├── deployment.yaml +│ └── service.yaml +├── signadot/ +│ ├── xata-branch-plugin.yaml # Resource Plugin +│ └── users-sandbox.yaml # Sandbox specification +└── Makefile +``` + +## Step 1: Configure Xata + +### Check Prerequisites + +First, verify that all required tools are installed: + +```bash +make setup +``` + +The command checks for `kubectl`, `xata`, `signadot`, `docker`, `psql`, and `jq`. It also verifies that you're authenticated with both Xata and Signadot. + +### Initialize Your Project + +Run `xata init`: + +```bash +xata init +``` + +The CLI prompts you for a database name. If the database doesn't exist, the CLI offers to create it for you. For the purpose of this tutorial, let’s use `signadotDb`: + +Expected output: + +![Xata init](./images/img-001.png) + +### Verify Your Starting Point + +Before we create any sandboxes, confirm that only the `main` branch exists: + +```bash +xata branch list +``` + +Expected output: + +![xata branch list](./images/img-002.png) + +Only `main` exists at this point. The sandbox will create an additional branch later. + +### Create Schema and Seed Data + +The Makefile automates schema creation and data seeding: + +```bash +make db-setup +``` + +The command runs `db/schema.sql` and `db/seed.sql` against your `main` branch. The schema creates a `users` table: + +```sql +CREATE TABLE users ( + id SERIAL PRIMARY KEY, + name TEXT NOT NULL, + email TEXT NOT NULL UNIQUE, + created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() +); +``` + +Verify the data: + +```bash +make db-connect +``` + +Then run: + +```sql +SELECT * FROM users; +``` + +Expected output: + +![make db-connect](./images/img-003.png) + +These five users exist in the `main` branch. Any branch created from `main` will inherit this data through copy-on-write. + +## Step 2: Deploy the Baseline Environment + +The baseline represents your shared staging environment, the state that sandboxes fork from. + +### Build the Container Image + +If using minikube, point your shell to minikube's Docker daemon: + +```bash +eval $(minikube docker-env) +``` + +Build the image: + +```bash +make build +``` + +### Deploy to Kubernetes + +Deploy the namespace, secret, deployment, and service: + +```bash +make deploy +``` + +The command creates a `db-creds` secret with your Xata connection string, applies the Kubernetes manifests, and waits for the deployment to become ready. + +The deployment includes the DevMesh sidecar annotation for Signadot routing. + +### Verify the Baseline + +Test the deployment: + +```bash +make test +``` + +The command port-forwards to the service and queries the `/health`, `/users`, and `/info` endpoints. + +You should see all five seed users and confirmation that the service connects to the `main` branch: + +![make test](./images/img-004.png) + +## Step 3: Install the Signadot Resource Plugin + +The Resource Plugin handles the integration between Signadot and Xata. It creates a database branch when a sandbox starts and deletes it when the sandbox terminates. + +### Create Credentials and Install Plugin + +The Makefile reads credentials from your `.xata` directory and creates the Kubernetes secret automatically: + +```bash +make plugin-secret +``` + +The command: + +1. Reads `XATA_ORG_ID` and `XATA_PROJECT_ID` from `.xata/project.json` +2. Reads `XATA_DATABASE_NAME` from `.xata/branch.json` +3. Prompts for `XATA_API_KEY` if not already set +4. Creates the `xata-credentials` secret in the `signadot` namespace +5. Installs the Resource Plugin + +Note that this secret is only for API authentication. The database connection string flows through the plugin's output mechanism and gets injected directly into sandbox pods; no manual secret wiring required. + +Verify the plugin installation: + +```bash +signadot resourceplugin get xata-branch +``` + +## Step 4: Create a Sandbox + +The sandbox specification in `signadot/users-sandbox.yaml` ties everything together. + +### The Sandbox Specification + +Three sections matter: + +**Resources**: Requests a Xata branch from the `xata-branch` plugin: + +```yaml +resources: + - name: usersDb + plugin: xata-branch + params: + parent-branch: "main" +``` + +**Forks**: Creates a copy of the `users-service` deployment with a modified `DATABASE_URL` environment variable. The value comes from the resource plugin's output: + +```yaml +forks: + - forkOf: + kind: Deployment + namespace: demo + name: users-service + customizations: + env: + - name: DATABASE_URL + valueFrom: + resource: + name: usersDb + outputKey: provision.connection-string +``` + +The `outputKey` format is `{step-name}.{output-name}`. Since the create workflow's step is named `provision` and the output is named `connection-string`, the key is `provision.connection-string`. + +**Endpoints**: Creates a preview URL for external access: + +```yaml +endpoints: + - name: users-api + host: users-service.demo.svc + port: 3000 + protocol: http +``` + +### Create the Sandbox + +Use the Makefile target: + +```bash +make sandbox-create +``` + +The command automatically detects your cluster and creates a sandbox named `xata-test`. Signadot executes the following sequence: + +1. Invokes the `xata-branch` plugin's create workflow +2. The plugin creates branch `sb-xata-test` in Xata +3. Signadot captures the connection string output +4. Signadot creates a forked `users-service` pod with the branch-specific `DATABASE_URL` +5. Signadot configures routing to the preview endpoint + +Wait for the sandbox to become ready. The CLI shows progress and outputs the preview URL. + +## Step 5: Verify Data Isolation + +The sandbox now has its own Xata branch. Changes made through the sandbox endpoint affect only that branch. + +### Check the Branches in Xata + +List branches via CLI: + +```bash +xata branch list +``` + +Output now shows two branches: + +![xata branch list](./images/img-005.png) + +The `sb-xata-test` branch was created by the Resource Plugin. + +### Query the Sandbox + +Preview URLs require authentication with a Signadot API key. Generate one from [app.signadot.com](https://app.signadot.com/) under Settings > API Keys, then export it: + +```bash +export SIGNADOT_API_KEY="your-api-key-here" +``` + +Query the sandbox endpoints: + +```bash +curl -H "signadot-api-key: ${SIGNADOT_API_KEY}" \ + "https://users-api--xata-test.preview.signadot.com/users" | jq . +``` + +The sandbox serves the same five seed users because it inherited them from `main`. + +Check which branch the sandbox connects to: + +```bash +curl -H "signadot-api-key: ${SIGNADOT_API_KEY}" \ + "https://users-api--xata-test.preview.signadot.com/info" | jq . +``` + +Output: + +![Check which branch the sandbox connects](./images/img-006.png) + +The sandbox connects to the `sb-xata-test` branch. + +### Create Test Data in the Sandbox + +Add a user that only exists in the sandbox branch: + +```bash +curl -X POST \ + -H "Content-Type: application/json" \ + -H "signadot-api-key: ${SIGNADOT_API_KEY}" \ + -d '{"name": "Sandbox User", "email": "sandbox@test.com"}' \ + "https://users-api--xata-test.preview.signadot.com/users" | jq . +``` + +Query the sandbox again: + +```bash +curl -H "signadot-api-key: ${SIGNADOT_API_KEY}" \ + "https://users-api--xata-test.preview.signadot.com/users" | jq . +``` + +You now see six users: the five from seed data plus "Sandbox User". + +### Confirm Baseline Is Unchanged + +Query the baseline deployment: + +```bash +make test +``` + +The baseline shows only five users. "Sandbox User" does not exist in `main`. The branches are fully isolated. + +You can also verify directly against Xata: + +```bash +# Query main branch +make db-connect +# Run: SELECT * FROM users; +# Shows 5 users + +# Query sandbox branch (switch first) +xata branch checkout sb-xata-test +make db-connect +# Run: SELECT * FROM users; +# Shows 6 users + +# Switch back to main +xata branch checkout main +``` + +## Step 6: Cleanup + +Delete the sandbox: + +```bash +make sandbox-delete +``` + +Signadot executes the delete workflow, which runs `xata branch delete` for `sb-xata-test`. The branch and all its data are removed. + +Verify the branch is gone: + +```bash +xata branch list +``` + +Output: + +![xata branch list](./images/img-007.png) + +Output shows only `main`. The `sb-xata-test` branch no longer exists. + +To remove all resources (sandbox, plugin, secrets, and Kubernetes deployments): + +```bash +make clean +``` + +## Cost Considerations + +Xata's copy-on-write branching means you only pay for storage that diverges from the parent. A branch that modifies 1% of the data uses roughly 1% additional storage, not a full copy. Compute scales to zero when branches are inactive. + +Signadot sandboxes share baseline cluster resources. Forked pods reuse existing infrastructure. You don't provision separate Kubernetes clusters for each environment. + +Branch creation and deletion both complete in few minutes. There's no waiting for database restores or provisioning delays. + +## Conclusion + +Database isolation was the missing piece in ephemeral environments. Combining Xata's copy-on-write branching with Signadot's sandbox orchestration fills that gap. Each developer gets an isolated application layer *and* an isolated database for every pull request. + +The Resource Plugin handles the entire lifecycle automatically. Create a sandbox, get a branch. Delete the sandbox, clean up the branch. No scripts to maintain. No orphaned resources. + +For more information, see the official [Signadot Documentation](https://www.signadot.com/docs). diff --git a/xata-branching-tutorial/db/schema.sql b/xata-branching-tutorial/db/schema.sql new file mode 100644 index 0000000..9441e88 --- /dev/null +++ b/xata-branching-tutorial/db/schema.sql @@ -0,0 +1,11 @@ +-- Users table schema +CREATE TABLE IF NOT EXISTS users ( + id SERIAL PRIMARY KEY, + name TEXT NOT NULL, + email TEXT NOT NULL UNIQUE, + created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() +); + +-- Index for email lookups +CREATE INDEX IF NOT EXISTS idx_users_email ON users(email); diff --git a/xata-branching-tutorial/db/seed.sql b/xata-branching-tutorial/db/seed.sql new file mode 100644 index 0000000..e9d9467 --- /dev/null +++ b/xata-branching-tutorial/db/seed.sql @@ -0,0 +1,8 @@ +-- Seed data for testing +INSERT INTO users (name, email) VALUES + ('Alice Johnson', 'alice@example.com'), + ('Bob Smith', 'bob@example.com'), + ('Carol Williams', 'carol@example.com'), + ('David Brown', 'david@example.com'), + ('Eve Davis', 'eve@example.com') +ON CONFLICT (email) DO NOTHING; diff --git a/xata-branching-tutorial/docker/users-service.Dockerfile b/xata-branching-tutorial/docker/users-service.Dockerfile new file mode 100644 index 0000000..a1feee9 --- /dev/null +++ b/xata-branching-tutorial/docker/users-service.Dockerfile @@ -0,0 +1,13 @@ +FROM node:18-alpine + +WORKDIR /app + +COPY pkg/users-service/package*.json ./ + +RUN npm install --omit=dev + +COPY pkg/users-service/app.js ./ + +EXPOSE 3000 + +CMD ["node", "app.js"] diff --git a/xata-branching-tutorial/images/img-001.png b/xata-branching-tutorial/images/img-001.png new file mode 100644 index 0000000..083ba15 Binary files /dev/null and b/xata-branching-tutorial/images/img-001.png differ diff --git a/xata-branching-tutorial/images/img-002.png b/xata-branching-tutorial/images/img-002.png new file mode 100644 index 0000000..fe0e921 Binary files /dev/null and b/xata-branching-tutorial/images/img-002.png differ diff --git a/xata-branching-tutorial/images/img-003.png b/xata-branching-tutorial/images/img-003.png new file mode 100644 index 0000000..97cd4b2 Binary files /dev/null and b/xata-branching-tutorial/images/img-003.png differ diff --git a/xata-branching-tutorial/images/img-004.png b/xata-branching-tutorial/images/img-004.png new file mode 100644 index 0000000..715561e Binary files /dev/null and b/xata-branching-tutorial/images/img-004.png differ diff --git a/xata-branching-tutorial/images/img-005.png b/xata-branching-tutorial/images/img-005.png new file mode 100644 index 0000000..13d9ead Binary files /dev/null and b/xata-branching-tutorial/images/img-005.png differ diff --git a/xata-branching-tutorial/images/img-006.png b/xata-branching-tutorial/images/img-006.png new file mode 100644 index 0000000..b8cf199 Binary files /dev/null and b/xata-branching-tutorial/images/img-006.png differ diff --git a/xata-branching-tutorial/images/img-007.png b/xata-branching-tutorial/images/img-007.png new file mode 100644 index 0000000..584dc3e Binary files /dev/null and b/xata-branching-tutorial/images/img-007.png differ diff --git a/xata-branching-tutorial/k8s/deployment.yaml b/xata-branching-tutorial/k8s/deployment.yaml new file mode 100644 index 0000000..1382b4d --- /dev/null +++ b/xata-branching-tutorial/k8s/deployment.yaml @@ -0,0 +1,45 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: users-service + namespace: demo + labels: + app: users-service +spec: + replicas: 1 + selector: + matchLabels: + app: users-service + template: + metadata: + labels: + app: users-service + annotations: + sidecar.signadot.com/inject: "true" + spec: + containers: + - name: users-service + image: users-service:latest + imagePullPolicy: Never + ports: + - containerPort: 3000 + env: + - name: DATABASE_URL + valueFrom: + secretKeyRef: + name: db-creds + key: DATABASE_URL + - name: XATA_BRANCH + value: "main" + livenessProbe: + httpGet: + path: /health + port: 3000 + initialDelaySeconds: 10 + periodSeconds: 10 + readinessProbe: + httpGet: + path: /health + port: 3000 + initialDelaySeconds: 5 + periodSeconds: 5 \ No newline at end of file diff --git a/xata-branching-tutorial/k8s/namespace.yaml b/xata-branching-tutorial/k8s/namespace.yaml new file mode 100644 index 0000000..9d80e24 --- /dev/null +++ b/xata-branching-tutorial/k8s/namespace.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: demo + labels: + app: users-service diff --git a/xata-branching-tutorial/k8s/service.yaml b/xata-branching-tutorial/k8s/service.yaml new file mode 100644 index 0000000..1660830 --- /dev/null +++ b/xata-branching-tutorial/k8s/service.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: Service +metadata: + name: users-service + namespace: demo + labels: + app: users-service +spec: + selector: + app: users-service + ports: + - port: 3000 + targetPort: 3000 + protocol: TCP + type: ClusterIP diff --git a/xata-branching-tutorial/pkg/users-service/app.js b/xata-branching-tutorial/pkg/users-service/app.js new file mode 100644 index 0000000..a3138eb --- /dev/null +++ b/xata-branching-tutorial/pkg/users-service/app.js @@ -0,0 +1,111 @@ +const express = require('express'); +const { Pool } = require('pg'); + +const app = express(); +app.use(express.json()); + +const pool = new Pool({ + connectionString: process.env.DATABASE_URL, + ssl: { rejectUnauthorized: false } +}); + +function parseXataConnectionString(connectionString) { + try { + const url = new URL(connectionString); + const pathParts = url.pathname.slice(1).split(':'); + const database = pathParts[0] || 'unknown'; + const branch = pathParts[1] ? pathParts[1].split('?')[0] : 'main'; + return { database, branch }; + } catch (error) { + return { database: 'unknown', branch: 'unknown' }; + } +} + +app.get('/health', async (req, res) => { + try { + await pool.query('SELECT 1'); + res.json({ status: 'healthy', database: 'connected' }); + } catch (error) { + res.status(500).json({ status: 'unhealthy', error: error.message }); + } +}); + +app.get('/users', async (req, res) => { + try { + const result = await pool.query('SELECT * FROM users ORDER BY id'); + res.json(result.rows); + } catch (error) { + res.status(500).json({ error: error.message }); + } +}); + +app.get('/users/:id', async (req, res) => { + try { + const { id } = req.params; + const result = await pool.query('SELECT * FROM users WHERE id = $1', [id]); + if (result.rows.length === 0) { + return res.status(404).json({ error: 'User not found' }); + } + res.json(result.rows[0]); + } catch (error) { + res.status(500).json({ error: error.message }); + } +}); + +app.post('/users', async (req, res) => { + try { + const { name, email } = req.body; + const result = await pool.query( + 'INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *', + [name, email] + ); + res.status(201).json(result.rows[0]); + } catch (error) { + res.status(500).json({ error: error.message }); + } +}); + +app.put('/users/:id', async (req, res) => { + try { + const { id } = req.params; + const { name, email } = req.body; + const result = await pool.query( + 'UPDATE users SET name = $1, email = $2, updated_at = NOW() WHERE id = $3 RETURNING *', + [name, email, id] + ); + if (result.rows.length === 0) { + return res.status(404).json({ error: 'User not found' }); + } + res.json(result.rows[0]); + } catch (error) { + res.status(500).json({ error: error.message }); + } +}); + +app.delete('/users/:id', async (req, res) => { + try { + const { id } = req.params; + const result = await pool.query('DELETE FROM users WHERE id = $1 RETURNING *', [id]); + if (result.rows.length === 0) { + return res.status(404).json({ error: 'User not found' }); + } + res.json({ message: 'User deleted', user: result.rows[0] }); + } catch (error) { + res.status(500).json({ error: error.message }); + } +}); + +app.get('/info', (req, res) => { + const { database } = parseXataConnectionString(process.env.DATABASE_URL || ''); + res.json({ + service: 'users-service', + database, + branch: process.env.XATA_BRANCH || 'main', + timestamp: new Date().toISOString() + }); +}); + +const PORT = process.env.PORT || 3000; +app.listen(PORT, () => { + console.log(`Server running on port ${PORT}`); +}); \ No newline at end of file diff --git a/xata-branching-tutorial/pkg/users-service/package.json b/xata-branching-tutorial/pkg/users-service/package.json new file mode 100644 index 0000000..f150472 --- /dev/null +++ b/xata-branching-tutorial/pkg/users-service/package.json @@ -0,0 +1,13 @@ +{ + "name": "users-service", + "version": "1.0.0", + "description": "Simple users API for Xata + Signadot tutorial", + "main": "app.js", + "scripts": { + "start": "node app.js" + }, + "dependencies": { + "express": "^4.18.2", + "pg": "^8.11.3" + } +} diff --git a/xata-branching-tutorial/signadot/users-sandbox.yaml b/xata-branching-tutorial/signadot/users-sandbox.yaml new file mode 100644 index 0000000..3d3d16d --- /dev/null +++ b/xata-branching-tutorial/signadot/users-sandbox.yaml @@ -0,0 +1,37 @@ +name: "@{sandbox-name}" +spec: + cluster: "@{cluster}" + description: Users service sandbox with isolated Xata database branch + + labels: + team: platform + service: users-service + + resources: + - name: usersDb + plugin: xata-branch + params: + parent-branch: "main" + + forks: + - forkOf: + kind: Deployment + namespace: demo + name: users-service + customizations: + env: + - name: DATABASE_URL + valueFrom: + resource: + name: usersDb + outputKey: provision.connection-string + - name: XATA_BRANCH + valueFrom: + resource: + name: usersDb + outputKey: provision.branch-name + + defaultRouteGroup: + endpoints: + - name: users-api + target: http://users-service.demo.svc:3000 \ No newline at end of file diff --git a/xata-branching-tutorial/signadot/xata-branch-plugin.yaml b/xata-branching-tutorial/signadot/xata-branch-plugin.yaml new file mode 100644 index 0000000..307b969 --- /dev/null +++ b/xata-branching-tutorial/signadot/xata-branch-plugin.yaml @@ -0,0 +1,134 @@ +name: xata-branch +spec: + description: Creates and deletes Xata database branches for sandbox isolation + + runner: + image: debian:bookworm-slim + namespace: signadot + podTemplateOverlay: | + spec: + containers: + - name: runner + env: + - name: XATA_API_KEY + valueFrom: + secretKeyRef: + name: xata-credentials + key: XATA_API_KEY + - name: XATA_ORG_ID + valueFrom: + secretKeyRef: + name: xata-credentials + key: XATA_ORG_ID + - name: XATA_PROJECT_ID + valueFrom: + secretKeyRef: + name: xata-credentials + key: XATA_PROJECT_ID + - name: XATA_DATABASE_NAME + valueFrom: + secretKeyRef: + name: xata-credentials + key: XATA_DATABASE_NAME + + create: + - name: provision + inputs: + - name: parent-branch + valueFromSandbox: true + as: + env: PARENT_BRANCH + + script: | + #!/bin/bash + set -e + + BRANCH_NAME="sb-${SIGNADOT_SANDBOX_NAME}" + PARENT="${PARENT_BRANCH:-main}" + + echo "=== Xata Branch CREATE ===" + echo "Branch: ${BRANCH_NAME}" + echo "Parent: ${PARENT}" + + # Install dependencies + apt-get update > /dev/null 2>&1 + apt-get install -y curl jq libdigest-sha-perl > /dev/null 2>&1 + + # Install Xata CLI using official installer + echo "Installing Xata CLI..." + curl -fsSL https://xata.io/install.sh | bash > /dev/null 2>&1 + export PATH="$HOME/.config/xata/bin:$PATH" + echo "CLI installed: $(xata --version)" + + # Get parent branch ID + echo "Looking up parent branch..." + PARENT_ID=$(xata branch list --organization "${XATA_ORG_ID}" --project "${XATA_PROJECT_ID}" --json | \ + jq -r ".[] | select(.name == \"${PARENT}\") | .id") + echo "Parent ID: ${PARENT_ID}" + + # Create branch + echo "Creating branch..." + xata branch create \ + --organization "${XATA_ORG_ID}" \ + --project "${XATA_PROJECT_ID}" \ + --name "${BRANCH_NAME}" \ + --parent-branch "${PARENT_ID}" + + # Wait for branch to be ready + echo "Waiting for branch to be ready..." + xata branch wait-ready \ + --organization "${XATA_ORG_ID}" \ + --project "${XATA_PROJECT_ID}" \ + "${BRANCH_NAME}" + + # Get connection string + echo "Getting connection string..." + CONNECTION_STRING=$(xata branch url \ + --organization "${XATA_ORG_ID}" \ + --project "${XATA_PROJECT_ID}" \ + --database "${XATA_DATABASE_NAME}" \ + "${BRANCH_NAME}") + + echo "Connection: ${CONNECTION_STRING}" + + # Write outputs + mkdir -p /outputs + echo -n "${BRANCH_NAME}" > /outputs/branch-name + echo -n "${CONNECTION_STRING}" > /outputs/connection-string + + echo "=== Branch creation complete ===" + + outputs: + - name: branch-name + valueFromPath: /outputs/branch-name + - name: connection-string + valueFromPath: /outputs/connection-string + + delete: + - name: cleanup + script: | + #!/bin/bash + set -e + + BRANCH_NAME="sb-${SIGNADOT_SANDBOX_NAME}" + + echo "=== Xata Branch DELETE ===" + echo "Branch: ${BRANCH_NAME}" + + # Install dependencies + apt-get update > /dev/null 2>&1 + apt-get install -y curl libdigest-sha-perl > /dev/null 2>&1 + + # Install Xata CLI + echo "Installing Xata CLI..." + curl -fsSL https://xata.io/install.sh | bash > /dev/null 2>&1 + export PATH="$HOME/.config/xata/bin:$PATH" + + # Delete branch + echo "Deleting branch..." + xata branch delete \ + --organization "${XATA_ORG_ID}" \ + --project "${XATA_PROJECT_ID}" \ + "${BRANCH_NAME}" --yes || echo "Branch may not exist" + + echo "=== Branch deletion complete ===" \ No newline at end of file