FlowWise Codebase

🚀 New Rust backend: A newer implementation is available at naisslab/FlowWise-Rust.

• ⚙️ Native broker: Uses a built-in broker instead of Apache Pulsar for simpler deployment and tighter routing control.
• 🧠 GPU batch inference: Supports batched GPU inference to improve accelerator use and high-throughput serving.

This repository contains the code and artifacts for the paper "FlowWise: Stateful Fast-Slow Model Serving for Streaming Traffic Intelligence" at SoCC'26.

FlowWise is a high-performance system designed for real-time network traffic analysis. It integrates efficient packet capture, a diverse suite of machine learning models, and a low-latency serving framework to provide immediate and accurate insights into network behavior. The system is flexible and can be deployed in various network environments, provided they meet the necessary throughput and latency thresholds.

---

🛠️ System Architecture & Implementation

The architecture of FlowWise is built on several key technologies, creating a robust and efficient pipeline for network analysis.

1. Packet Capture & Processing

High-speed packet capture is the foundation of our system. We use nPrint in conjunction with the PF_RING C APIs to capture network packets efficiently and expressively.

• Promiscuous Mode: We enable PF_RING_PROMISC to capture all packets on the attached network segment, not just those addressed to the host machine.
• Feature Extraction: Following the nPrint design, the system captures the first 1024 bits from IPv4, TCP, and UDP headers. We have also extended this to allow for dynamic feature subscriptions.

2. Machine Learning Models

FlowWise employs a variety of ML models to handle different analysis tasks, ranging from simple to complex. All models are converted to the ONNX format (using opset v12 and ml-opset v2) for standardized deployment.

Some selected models in the papers are as follows, but the framework can support any model (from TensorFlow, PyTorch, Keras, scikit-learn, Chainer, MXNet, and MATLAB) that can be converted to ONNX:

• Classical Models:
  • Decision Trees: Configured with at least 15 samples per leaf.
  • Random Forest
• Gradient Boosting:
  • LightGBM: Tuned with a learning rate of 0.03, 128 leaves, a feature fraction of 0.9, and a minimum of 3 data points per leaf.
  • XGBoost: Configured with 100 estimators.
• Deep Learning:
  • CNNs: A custom Convolutional Neural Network architecture designed for sequential packet data, featuring convolution, pooling, fully connected layers, and dropout for regularization.

For each application, we train five different models across packet depths ranging from 1 to 20, resulting in a total of 100 models with varying complexities.

3. Low-Latency Model Serving

Model serving is managed by EdgeServe, a low-latency streaming system designed for decentralized prediction. This enables efficient message routing and allows all nodes to produce and consume data and predictions.

• Dynamic Model Selection: For each new flow, the fastest model is used to process the first packet.
• Adaptive Analysis: If this initial prediction is deemed unsatisfactory by our flow selection algorithm, slower and more complex models are invoked to analyze additional features and subsequent packets.
• Queueing: A FIFO (First-In, First-Out) queue manages the asynchronous arrival of data, ensuring the system remains responsive and efficient.
• Parallelism: For ONNX Runtime deployment, both inter- and intra-operation parallelism are set to 1.

---

💻 Core Technologies

• Packet Processing: nPrint & PF_RING
• Model Serving: EdgeServe
• ML Frameworks: Scikit-learn, PyTorch, LightGBM, XGBoost
• Deployment: ONNX Runtime

---

📂 Repository Structure

The repository is organized as follows, with FlowWise containing the core logic, alongside directories for experiments, scripts, and the testbed.

├── 📁 Experiments/
│   └── # Experiment configurations, setup, and results.
│
├── 📁 FlowWise/
│   ├── 📁 consumers/
│   │   └── # Code related to EdgeServe data consumers.
│   ├── 📁 flowwise_nprint/
│   │   └── # Core implementation integrating FlowWise logic with nPrint.
│   └── 📁 profiling/
│       └── # Scripts for performance analysis and profiling.
│
├── 📁 Scripts/
│   └── # General utility and helper scripts.
│
└── 📁 Testbed/
└── # Environment setup for testing and system validation.