Real-time Fraud Detection System with Apache Flink, StateFun, Kafka, and Rust

This project is a local simulation of a real-time fraud detection system using Apache Flink with StateFun, Kafka for ingestion and alerts, and a remote function implemented in Rust via an HTTP server.

🧠 Overview

The architecture follows a modern event-driven design:

1. Kafka (transactions) receives transaction events.
2. StateFun (Flink) consumes these events and routes them to:
   • A Rust function server via HTTP for fraud analysis.
3. The function:
   • Decodes a CloudEvent.
   • Determines if the transaction is fraudulent.
   • Sends alert messages to the Kafka alerts topic if needed.
4. Kafka stores these alerts, which can be consumed by other services (dashboards, alerting systems, etc).

🔧 Technologies Used

• Apache Kafka: Messaging system with two topics (transactions, alerts)
• Apache Flink + StateFun: Stateful function orchestration
• Rust + Axum: Remote fraud detection function
• CloudEvents: Event format
• Serde: Serialization in Rust
• Docker Compose: Local orchestration

📁 Structure

.
├── Dockerfile.flink          # Flink JobManager/TaskManager image
├── Dockerfile                # Rust HTTP server
├── docker-compose.yml        # Defines services
├── conf/
│   └── flink-conf.yaml       # Flink config
├── module.yaml               # StateFun module
└── src/                      # Rust source code

⚙️ Requirements

• Docker & Docker Compose
• Rust toolchain (for development)
• Git

🚀 Getting Started

# Clone repository
git clone https://github.com/YOUR_USERNAME/real-time-fraud-detection.git
cd real-time-fraud-detection

# Start system
docker compose up --build

Wait until all containers are up (especially jobmanager and statefun-server).

📦 Kafka Topics

• Create topics are done by kafka-init service:
  • transactions: for incoming transactions
  • alerts: for outgoing alerts

🔄 module.yaml Explained

# Remote HTTP Function
kind: io.statefun.endpoints.v2/http
spec:
  functions: rust/*
  urlPathTemplate: http://statefun-server:8000/function
  contentType: application/json

# Kafka Ingress
kind: io.statefun.kafka.v1/ingress
spec:
  id: rust/transactions
  address: kafka-broker:19092
  consumerGroupId: rust-group
  topics:
    - topic: transactions
      valueType: io.statefun.types/json 
      targets:
        - rust/fraud-detector

# Kafka Egress
egresses:
  - id: rust/alerts-egress
    type: io.statefun.kafka.v1/egress
    targets:
      - kafka-topic: alerts
        bootstrap-servers: kafka-broker:19092

🧪 Testing It

To send a transaction:

docker exec -it statefun-server-1 curl -X POST   -H "content-type: application/json"   -d '{"transaction_id":"tx001","user_id":"user_1","amount":3000,"duration_ms":2000}'   http://localhost:8000/function

To consume alerts from Kafka:

docker exec -it kafka-broker kafka-console-consumer \
  --bootstrap-server kafka-broker:19092 \
  --topic alerts \
  --from-beginning

📜 License

MIT or Apache-2.0