Skip to content

Commit 69eacf8

Browse files
[DOCS] Restructuring docs for Smart Tolling
Signed-off-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com>
1 parent cdd1c24 commit 69eacf8

File tree

6 files changed

+257
-216
lines changed

6 files changed

+257
-216
lines changed

metro-ai-suite/metro-vision-ai-app-recipe/smart-tolling/README.md

Lines changed: 35 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -6,36 +6,52 @@ Enabling such use cases across multiple viewpoints helps in understanding the ob
66

77
![Smart Tolling System Architecture](./docs/user-guide/_assets/smart_tolling_architecture.png)
88

9-
## Use Cases
9+
## Key Features
1010

11-
- **Vehicle Axle Detection**: Determines vehicle class based on axle and wheel count
12-
- Usage: Used for toll classification and revenue calculation.
13-
- **License Plate Detection**: Identifies vehicles uniquely using license plates
14-
- Usage: Used for reading vehicle license plate text from both front and rear views.
15-
- **Visualization & Analytics**:Provides real-time and historical insights for toll operators.
11+
**Multi vision**:
1612

17-
## Key Benefits
13+
Scene-based analytics allow insights beyond single sensor views.
1814

19-
- **Accuracy**: Multi view accuracy.
20-
- **Revenue Protection**: Advanced "Lift Axle" detection using Computer Vision allows for accurate tariffing.
21-
- **Audit Compliance**: Every transaction includes an "Image Evidence" for simplified auditing.
15+
- **Vehicle axle detection**:
16+
17+
Vehicle class is determined based on axle and wheel count.\
18+
Intended for toll classification, as well as revenue calculation and protection.\
19+
Multi view and the advanced "lift axle" detection using computer vision
20+
ensures accurate tariffing.
21+
22+
- **License plate detection**:
23+
24+
The application identifies vehicles uniquely by their license plates, which are
25+
read from both front and rear views.\
26+
The image evidence is included in every transaction for simplified auditing.
27+
28+
**Visualization & analytics**:
29+
30+
Provides real-time and historical insights for toll operators.
31+
32+
**Modularity**:
33+
34+
Architecture based on modular microservices enables composability and reconfiguration.
35+
36+
**High-throughput processing**:
37+
38+
Optimized video pipelines for Intel edge devices.
2239

2340
## How it Works
2441

2542
The system uses the **Metro Edge Architecture** based on three key principles:
2643

27-
1. **Perception**: Deep Learning Streamer (DLStreamer) processes 3/4 camera feeds.
28-
2. **Control**: SceneScape Controller aggregates metadata.
29-
3. **Analytics**: Node-RED transforms events into traffic insights (Traffic Volume, Flow Efficiency, Tariffing).
30-
31-
For more details, refer to [How it Works](./docs/user-guide/how-it-works.md).
44+
- **Perception**: Deep Learning Streamer (DLStreamer) [processes 3/4 camera feeds](./docs/user-guide/perception-layer.md).
45+
- **Control**: SceneScape Controller [aggregates metadata](./docs/user-guide/how-it-works.md#analytics-pipeline-downstream).
46+
- **Analytics**: Node-RED [transforms events into traffic insights](./docs/user-guide/how-it-works.md#node-red-transformation)
47+
(Traffic Volume, Flow Efficiency, Tariffing).
3248

3349
## Learn More
3450

35-
- [System Requirements](./docs/user-guide/get-started/system-requirements.md): Hardware, OS and Software Prerequisites.
36-
- [Get Started](./docs/user-guide/get-started.md): Installation, Configuration and Launch steps.
37-
- [Technical Reference](./docs/user-guide/how-it-works/technical-reference.md): Engineering Specs, Zero-Copy Pipeline and API usage.
38-
- [Troubleshooting](./docs/user-guide/troubleshooting.md): Solutions to common issues.
51+
- [System Requirements](./docs/user-guide/get-started/system-requirements.md)
52+
- [Get Started](docs/user-guide/get-started.md)
53+
- [How It Works](./docs/user-guide/how-it-works.md)
54+
- [Troubleshooting](./docs/user-guide/troubleshooting.md)
3955

4056
## License
4157

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# API Reference
2+
3+
Integrators can subscribe to `scenescape/cmd/camera/#` for real-time events.
4+
See the sample JSON Payload Schema (v2) below:
5+
6+
```json
7+
{
8+
"id": "toll_side_1",
9+
"timestamp": "2026-02-03T12:00:00.000Z",
10+
"objects": {
11+
"vehicle": [
12+
{
13+
"id": 1,
14+
"vehicle_type": "truck",
15+
"confidence": 0.98,
16+
"bounding_box_px": { "x": 100, "y": 200, "width": 500, "height": 300 },
17+
18+
"axle_count": 3,
19+
"wheel_count": 6,
20+
"wheels_touching_ground": 4, // 1 Lift Axle detected!
21+
22+
"vehicle_color": "white",
23+
"vehicle_image_b64": "..." // High-res crop for audit
24+
}
25+
]
26+
}
27+
}
28+
```
Lines changed: 101 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,59 +1,125 @@
11
# How It Works
22

3-
This section provides a high-level view of how the application integrates with a typical system architecture.
3+
This section provides a high-level view of how the application integrates with a
4+
typical system architecture.
45

56
![High-Level System Diagram](./_assets/smart_tolling_architecture.png)
67

78
## Diagram Description
89

9-
- **Inputs**:
10+
### Inputs
1011

11-
Video recordings are used to simulate a live feed from cameras deployed at a toll.
12-
The application can be configured to work with live cameras.
12+
Video recordings are used to simulate a live feed from cameras deployed at a toll.
13+
The application can be configured to work with live cameras.
1314

14-
- **Video Files** - Tolling cameras that capture videos simultaneously from front,
15-
rear and side profiles.
16-
- **Scene Database** - Pre-configured intersection scene with satellite view of
17-
tolling area, calibrated cameras and regions of interest.
15+
- **Video Files** - Tolling cameras that capture videos simultaneously from front,
16+
rear and side profiles.
17+
- **Scene Database** - Pre-configured intersection scene with satellite view of
18+
tolling area, calibrated cameras and regions of interest.
1819

19-
- **Processing**:
20+
### Core (Processing)
2021

21-
- **Video Analytics** - Deep Learning Streamer Pipeline Server
22-
(DL Streamer Pipeline Server) utilizes a pre-trained object detection model
23-
to generate object detection metadata and and a local NTP server for
24-
synchronized timestamps. This metadata is published to the MQTT broker.
25-
- **Sensor Fusion** - Scene Controller Microservice fuses the metadata from
26-
video analytics utilizing scene data obtained through the Scene Management API.
27-
It uses the fused tracks and the configured analytics (regions of interest)
28-
to generate events that are published to the MQTT broker.
29-
- **Aggregate Scene Analytics** - Region of interests analytics are read from
30-
the MQTT broker and stored in an InfluxDB bucket that enables time series analysis through Flux queries.
31-
- **Outputs**:
32-
- Fused object tracks are available on the MQTT broker and visualized through the Scene Management UI.
33-
- Aggregated toll analytics are visualized through a Grafana dashboard.
22+
- [**Video Analytics**](./perception-layer.md) - Deep Learning Streamer Pipeline Server
23+
(DL Streamer Pipeline Server) utilizes a pre-trained object detection model
24+
to [generate object detection metadata](#zero-copy-video-pipeline) and and a local
25+
NTP server for synchronized timestamps. This metadata is published to the MQTT broker.
26+
- **Sensor Fusion** - Scene Controller Microservice fuses the metadata from
27+
video analytics utilizing scene data obtained through the Scene Management API.
28+
It uses the fused tracks and the configured analytics (regions of interest)
29+
to generate events that are published to the MQTT broker.
30+
- [**Aggregate Scene Analytics**](#node-red-transformation) - Region of interests
31+
analytics are read from the MQTT broker and
32+
[stored in an InfluxDB bucket](#storage-influxdb) that enables time series
33+
analysis through Flux queries.
3434

35-
## Key Features
35+
### Live Feed Output
3636

37-
- **Feature 1**: Architecture based on modular microservices enables composability and reconfiguration.
38-
- **Feature 2**: Optimized video pipelines for Intel edge devices.
39-
- **Feature 3**: Scene-based analytics allow insights beyond single sensor views.
37+
- Fused object tracks are available on the MQTT broker and visualized through
38+
the Scene Management UI.
39+
- [Aggregated toll analytics](#analytics-pipeline-downstream) are visualized
40+
through a Grafana dashboard.
41+
42+
### Workflow
43+
44+
1. Video loops or RTSP is fed into DL Streamer.
45+
2. Trained AI models detect vehicles and license plates.
46+
3. Metadata is published to MQTT.
47+
4. SceneScape maps detections to scene regions to get exact location of objects on the scene.
48+
5. Exit events are generated when vehicles leave the region.
49+
6. Node-RED processes only finalized exit events by subscribing to SceneScape topics.
50+
7. Data is written to InfluxDB for system to access for consistent information.
51+
8. Grafana visualizes real time and historical data enabling access to metrics
52+
and vehicle details.
53+
54+
## Optimizations
55+
56+
The system achieves high-throughput processing on Edge hardware through specific
57+
optimizations defined in `config.json`. The [`docker-compose.yml`](./_assets/docker-compose.yml)
58+
file mentions all the services and the pipelines are configured in `config.json` file.
59+
60+
### Zero-Copy Video Pipeline
61+
62+
Unlike standard OpenCV pipelines that copy frames to CPU RAM, this solution utilizes **VASurface Sharing plugin**.
63+
64+
- **Mechanism:** Decoded video frames remain in GPU memory (`video/x-raw(memory:VAMemory)`).
65+
- **Benefit:** Zero-copy inference eliminates PCIe bandwidth bottlenecks, reducing end-to-end latency by ~40%.
66+
- **Config Evidence:** `pre-process-backend=va-surface-sharing` used in all `gvadetect` elements.
67+
68+
### Dynamic ROI Inference (Hierarchical Execution)
69+
70+
To maximize efficiency, heavy neural networks (like Axle Counting) do not run on the full 4K frame.
71+
72+
- **Logic:** The "Vehicle Type" model runs first to find the bounding box.
73+
- **Optimization:** The Axle model is configured with `inference-region=roi-list`,
74+
forcing it to execute *only* within the coordinates of the detected vehicle.
75+
- **Impact:** Reduces pixel processing load by >80% for sparse traffic scenes.
76+
77+
### Hybrid Workload Distribution
78+
79+
The pipeline intelligently maps models to available accelerators to prevent resource contention:
80+
81+
- **GPU (Flex Series):** Handles heavy convolution tasks (Vehicle Detection, LPR, Axle Counting).
82+
- **CPU (Xeon):** Handles lighter classification tasks (Vehicle Color) and post-processing adapters (`gvapython`).
83+
84+
## Analytics Pipeline (Downstream)
85+
86+
Raw metadata is valuable, but actionable insights come from the Analytics Pipeline.
87+
88+
## Node-RED Transformation
89+
90+
- **Input:** The **MQTT IN Node** subscribes to `scenescape/event/region/+/+/objects`.
91+
- **Logic:** The **Function node** aggregates counts per region and calculates **Dwell Time** (congestion).
92+
- **Output:** The **InfluxDB OUT Node** writes normalized data points to InfluxDB.
93+
94+
![Node-RED Flow](./_assets/smart_tolling_nodered.png)
95+
96+
### Storage (InfluxDB)
97+
98+
InfluxDB acts as a single source of truth. All critical and shared data is
99+
stored in one location, ensuring every user and system accesses the same,
100+
accurate and consistent information.
101+
102+
![InfluxDB Dashboard 1](./_assets/smart_tolling_influx_db.png)
103+
104+
### Visualization (Grafana)
105+
106+
The system ships with a pre-configured dashboard (`anthem-intersection.json` schema)
107+
focusing on Traffic Volume, Flow Efficiency and Safety Alerts.
108+
109+
![Grafana Dashboard 1](./_assets/garfana_Dashboard1.png)
40110

41111
## Learn More
42112

43-
- [System Requirements](./get-started/system-requirements.md):
44-
Check the hardware and software requirements for deploying the application.
45-
- [Get Started](./get-started.md):
46-
Follow step-by-step instructions to set up the application.
47-
- [Technical Reference](./how-it-works/technical-reference.md): Learn more about engineering specification and
48-
how to use Zero-Copy Pipeline and API.
49-
- [Support and Troubleshooting](./troubleshooting.md):
50-
Find solutions to common issues and troubleshooting steps.
113+
- [System Requirements](./get-started/system-requirements.md)
114+
- [Get Started](./get-started.md)
115+
- [API Reference](./api-reference.md)
116+
- [Support and Troubleshooting](./troubleshooting.md)
51117

52118
<!--hide_directive
53119
:::{toctree}
54120
:hidden:
55121
56-
./how-it-works/technical-reference
122+
./perception-layer
57123
58124
:::
59125
hide_directive-->

0 commit comments

Comments
 (0)