Skip to content

Htunn/simple-service-bus

BranchesTags

Repository files navigation

Simple Service Bus

License Go Version CI

A modern, decentralized Enterprise Service Bus (ESB) written in Go. Simple Service Bus provides a lightweight, cloud-native messaging and integration platform with support for multiple protocols, pluggable message brokers, and comprehensive enterprise features.

🚀 Features

Core Capabilities

  • Multi-Protocol Support: HTTP/REST, gRPC, AMQP, MQTT, Kafka, WebSocket, GraphQL
  • Pluggable Message Brokers: Built-in in-memory broker + adapters for Kafka, NATS, RabbitMQ, Redis Streams
  • Flexible Message Routing: Content-based, header-based, and topic-based routing with rule engine
  • Message Transformation: JSON, XML, Protobuf, Avro, and template-based transformations
  • Service Orchestration: Pipeline composition and Saga pattern support

Enterprise Features

  • Security: JWT, API keys, OAuth2, mTLS authentication
  • Resilience: Circuit breaker, retry with exponential backoff, dead letter queues
  • Observability: Prometheus metrics, OpenTelemetry tracing, structured logging
  • Rate Limiting: Token bucket and sliding window algorithms
  • Service Discovery: Static config, mDNS, Kubernetes integration
  • High Availability: Stateless design, horizontal scaling

Developer Experience

  • Web Admin UI: React-based dashboard for monitoring and management
  • CLI Tool: ssbctl for command-line operations
  • REST API: Comprehensive management API with OpenAPI spec
  • Hot Reload: Live configuration updates without restart

Deployment Options

  • Docker: Multi-stage builds, <50MB images
  • Kubernetes: Helm charts with HPA, PDB, NetworkPolicy
  • Bare Metal: Systemd service files for traditional deployments

📋 Architecture

┌─────────────────────────────────────────────────────────┐
│                    Admin UI (React/Vite)                 │
├─────────────────────────────────────────────────────────┤
│                     Management API                       │
├──────────┬──────────┬───────────┬───────────────────────┤
│  Router  │Transform │  Orchestr │    Observability       │
│  Engine  │  Engine  │  Engine   │ (Metrics/Trace/Log)    │
├──────────┴──────────┴───────────┴───────────────────────┤
│                   Middleware Pipeline                     │
│  (Auth, RateLimit, CircuitBreaker, Retry, DLQ, Logger)  │
├──────────┬──────────┬───────────┬──────────┬────────────┤
│  HTTP/   │  gRPC    │   AMQP    │  MQTT    │  Kafka     │
│  REST    │          │           │          │            │
│  GraphQL │  WebSock │           │          │            │
├──────────┴──────────┴───────────┴──────────┴────────────┤
│               Built-in Broker (In-Memory)                │
│         + Pluggable External Broker Adapters             │
├─────────────────────────────────────────────────────────┤
│            Config / Service Discovery / Store             │
└─────────────────────────────────────────────────────────┘

Message Flow

sequenceDiagram
    participant P as Producer
    participant T as Transport<br/>(HTTP/gRPC/MQTT)
    participant MW as Middleware Chain
    participant R as Router Engine
    participant TF as Transformer
    participant B as Broker
    participant S as Subscriber

    P->>+T: Send message
    T->>+MW: handleIncomingMessage()
    MW->>MW: Auth → RateLimit → Logger → Metrics
    MW->>+R: Route(envelope)
    R->>R: Evaluate conditions (header/content/topic)
    R-->>-MW: Matched routes + destinations
    MW->>+TF: Transform(msg) [if content type matches]
    TF-->>-MW: Transformed message
    MW->>+B: Publish(topic, msg) for each destination
    B-->>-MW: Ack
    MW-->>-T: Response
    T-->>-P: 200 OK
    B-)S: Deliver message (async)
Loading

Publish / Subscribe

sequenceDiagram
    participant Pub as Publisher
    participant Bus as Service Bus
    participant Broker as Broker<br/>(Memory/Kafka/NATS)
    participant Sub1 as Subscriber A
    participant Sub2 as Subscriber B

    Sub1->>Bus: Subscribe("orders.new")
    Bus->>Broker: Register subscription
    Sub2->>Bus: Subscribe("orders.new")
    Bus->>Broker: Register subscription

    Pub->>+Bus: Publish("orders.new", msg)
    Bus->>Bus: Apply middleware chain
    Bus->>+Broker: Publish(topic, msg)
    Broker-->>-Bus: Ack
    Bus-->>-Pub: OK

    Broker-)Sub1: Deliver message
    Broker-)Sub2: Deliver message
    Sub1-->>Broker: Ack
    Sub2-->>Broker: Ack
Loading

Middleware Pipeline

sequenceDiagram
    participant Req as Incoming Message
    participant Auth as Auth MW
    participant RL as RateLimit MW
    participant CB as CircuitBreaker MW
    participant Log as Logger MW
    participant Met as Metrics MW
    participant Retry as Retry MW
    participant H as Handler

    Req->>+Auth: Process
    Auth->>Auth: Validate JWT / API key
    Auth->>+RL: Next
    RL->>RL: Check token bucket
    RL->>+CB: Next
    CB->>CB: Check circuit state
    CB->>+Log: Next
    Log->>Log: Log request start
    Log->>+Met: Next
    Met->>Met: Start timer
    Met->>+Retry: Next
    Retry->>+H: Execute handler
    H-->>-Retry: Result
    Note over Retry,H: Retry on failure<br/>with exponential backoff
    Retry-->>-Met: Result
    Met->>Met: Record latency + counter
    Met-->>-Log: Result
    Log->>Log: Log result
    Log-->>-CB: Result
    CB-->>-RL: Result
    RL-->>-Auth: Result
    Auth-->>-Req: Final response
Loading

🔧 Quick Start

Installation

# Using Go
go install github.com/htunn/simple-service-bus/cmd/ssb@latest
go install github.com/htunn/simple-service-bus/cmd/ssbctl@latest

# Using Docker
docker pull simple-service-bus/ssb:latest

# Using Helm
helm repo add ssb https://simple-service-bus.github.io/helm-charts
helm install my-ssb ssb/simple-service-bus

Basic Usage

# Start the service bus
ssb --config config.yaml

# Or with Docker
docker run -p 8080:8080 -p 9090:9090 simple-service-bus/ssb:latest

# Publish a message via CLI
ssbctl publish --topic orders.new --data '{"order_id": "123", "amount": 99.99}'

# Subscribe to a topic
ssbctl subscribe --topic orders.new

# View routes
ssbctl routes list

# Check health
curl http://localhost:8080/healthz

Programmatic Usage

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/htunn/simple-service-bus/pkg/message"
    "github.com/htunn/simple-service-bus/pkg/router"
)

func main() {
    ctx := context.Background()

    // Create a message
    msg := message.NewMessage([]byte(`{"orderId": "12345", "amount": 99.99}`))
    msg.ContentType = "application/json"
    msg.SetHeader("X-Source", "order-service")
    msg.SetHeader("Priority", "high")
    msg.SetMetadata("customerId", "cust-001")

    // Wrap in envelope for routing
    env := message.NewEnvelope(msg).
        WithSource("order-service").
        WithDestination("payment-service").
        WithTopic("orders.new").
        WithPriority(8).
        WithTTL(5 * time.Minute)

    // Routing example
    headerCondition := &router.HeaderCondition{
        Key:      "Priority",
        Value:    "high",
        Operator: "equals",
    }

    route := &router.Route{
        ID:           "high-priority-orders",
        Name:         "High Priority Orders",
        Condition:    headerCondition,
        Destinations: []string{"queue://express", "queue://notifications"},
        Priority:     9,
        Enabled:      true,
    }

    // Check if message matches route
    matches, err := route.Condition.Evaluate(ctx, msg)
    if err != nil {
        log.Fatal(err)
    }

    if matches {
        fmt.Printf("Message routed to: %v\n", route.Destinations)
    }
}

For more examples, see the examples/ directory.

Configuration Example

# config.yaml
server:
  http_addr: ":8080"
  grpc_addr: ":9090"
  admin_ui: true

broker:
  type: "memory"  # or kafka, nats, rabbitmq, redis

transports:
  - type: http
    enabled: true
  - type: grpc
    enabled: true

routes:
  - id: orders-route
    topics: ["orders.new"]
    destination: "order-service"
    transform: "enrich-order"

middleware:
  - type: auth
    jwt_secret: "${JWT_SECRET}"
  - type: ratelimit
    requests_per_second: 100
  - type: metrics
    enabled: true

📚 Documentation

🎯 Use Cases

  • Microservices Communication: Decouple services with async messaging
  • Event-Driven Architecture: Build event sourcing and CQRS patterns
  • API Gateway: Route and transform HTTP/gRPC requests
  • Data Integration: Connect disparate systems with protocol mediation
  • Message Orchestration: Coordinate distributed workflows with Saga patterns

Saga Pattern (Distributed Transactions)

sequenceDiagram
    participant C as Client
    participant O as Saga Orchestrator
    participant Ord as Order Service
    participant Pay as Payment Service
    participant Inv as Inventory Service

    C->>+O: Start order saga
    O->>+Ord: Step 1: Create order
    Ord-->>-O: Order created

    O->>+Pay: Step 2: Process payment
    Pay-->>-O: Payment processed

    O->>+Inv: Step 3: Reserve inventory
    Inv-->>O: ❌ Out of stock!

    Note over O: Step 3 failed — begin compensation

    O->>+Pay: Compensate: Refund payment
    Pay-->>-O: Payment refunded
    O->>+Ord: Compensate: Cancel order
    Ord-->>-O: Order cancelled
    deactivate Inv

    O-->>-C: Saga failed (rolled back)
Loading

🔌 Protocol Support

Protocol Transport Broker Adapter Status
HTTP/REST ✅ - Stable
gRPC ✅ - Stable
WebSocket ✅ - Stable
GraphQL ✅ - Beta
AMQP ✅ ✅ RabbitMQ Stable
MQTT ✅ - Stable
Kafka ✅ ✅ Stable
NATS - ✅ JetStream Stable
Redis - ✅ Streams Beta

🛠️ Development

Prerequisites

  • Go 1.22 or higher
  • Docker (for integration tests)
  • Make

Building from Source

# Clone the repository
git clone https://github.com/htunn/simple-service-bus.git
cd simple-service-bus

# Install dependencies
go mod download

# Build binaries
make build

# Run tests
make test

# Run linter
make lint

# Run integration tests (requires Docker)
make test-integration

# Build Docker image
make docker-build

Project Structure

├── cmd/               # Binary entry points
├── pkg/               # Core libraries
├── web/               # Admin UI
├── deploy/            # Deployment manifests
├── examples/          # Usage examples
└── docs/              # Documentation

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for details on:

  • Code of conduct
  • Development workflow
  • Submitting pull requests
  • Coding standards

📝 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🌟 Acknowledgments

Inspired by modern messaging frameworks:

  • Watermill - Event-driven applications in Go
  • NATS - Cloud-native messaging system
  • Dapr - Distributed application runtime
  • go-micro - Microservices framework

Built with ❤️ by the Simple Service Bus Contributor

About

A modern, decentralized Enterprise Service Bus (ESB) written in Go. Simple Service Bus provides a lightweight, cloud-native messaging and integration platform with support for multiple protocols, pluggable message brokers, and comprehensive enterprise features.

Resources

Readme

License

Apache-2.0 license

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages