Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
# Benchtool

This tutorial will show you how to use Benchtool to measure the
frequency of messages published in selected ROS 2 topics.

## When to use it over `ros2 topic hz`?

Use Benchtool when:

- You want to measure the publishing frequency of multiple topics at
once.
- You want to save the results to a file for later analysis (csv).
- You want temporally accurate data.
- You want to measure topics that produce a high volume of data (above
1-2 GB/s `ros2 topic hz` stats produce highly unreliable data).
- You want to measure the publishing frequency of specific
transformations (TFs).

Use `ros2 topic hz` when:

- You want to measure arbitrary topics quickly. (Due to technical
limitations this tool is not capable of recording arbitrary topics,
and support for them needs to be included in the source code. On the
bright size this is easily extendable and requires 3 lines of code
to add a new topic type.)

## Prerequisites

Complete the [get started guide](../../../gsg_robot/index.rst)
before continuing.

## Installation

<!--hide_directive::::{tab-set}
:::{tab-item}hide_directive--> **Jazzy**
<!--hide_directive:sync: tab1hide_directive-->

```bash
sudo apt install ros-jazzy-benchtool
# (optionally) install also a bag for purpose of demonstration
sudo apt install ros-jazzy-bagfile-2d-lidar
```

<!--hide_directive:::
:::{tab-item}hide_directive--> **Humble**
<!--hide_directive:sync: tab2hide_directive-->

```bash
sudo apt install ros-humble-benchtool
# (optionally) install also a bag for purpose of demonstration
sudo apt install ros-humble-bagfile-2d-lidar
```

<!--hide_directive:::
::::hide_directive-->

## Configuration

First, you need to set up `benchtool.toml` file, below is an example of
a configuration file. You can configure it to suit your needs.

```toml
[main]
video_topics = ["/camera/color/image_raw", "/camera/aligned_depth_to_color/image_raw"]
tf_child_frames = ["camera"]
```

This list provides the names of the topics and the corresponding ROS
message types that are used in the `Benchtool`.

- Video Topics
- Message Type: `sensor_msgs::msg::Image`
- Name: `video_topics`
- Map Topics
- Message Type: `nav_msgs::msg::OccupancyGrid`
- Name: `map_topics`
- TF Child Frames
- Message Type: `tf2_msgs::msg::TFMessage`
- Name: `tf_child_frames`
- Markers Topics
- Message Type: `visualization_msgs::msg::MarkerArray`
- Name: `markers_topics`
- Path Topics
- Message Type: `nav_msgs::msg::Path`
- Name: `path_topics`
- Point Cloud2 Topics
- Message Type: `sensor_msgs::msg::PointCloud2`
- Name: `point_cloud2_topics`

This tool also supports subscriptions to arbitrary topic types, to do
this you need to create array of tables called `generics` in
config toml file. Bellow is a file that has the same behavior as above
but uses `[[generics]]` syntax.
`[[generics]]` section can appear any number of times,
<https://toml.io/en/v1.0.0#array-of-tables>

```toml
[main]
tf_child_frames = ["camera"]

[[generics]]
type = "sensor_msgs/msg/Image"
topics = ["/camera/color/image_raw","/camera/aligned_depth_to_color/image_raw"]
```

## Usage

To start the Benchtool, run the following command:

```bash
ros2 run benchtool benchtool --ros-args -p toml:=./benchtool.toml -p o_csv:=./results.csv
```

This will start the Benchtool with the configuration file
`benchtool.toml` and save the results to
`results.csv`.

## Advanced usage - Prometheus Server + Grafana Dashboard

This tool by default will also create a Prometheus Server. In this
section you will learn how to use the Benchtool with Prometheus Server
and Grafana Dashboard to visualize the data.

First, you need to install Docker Engine through
<https://docs.docker.com/engine/install/>.

Then you need to prepare config file for Prometheus Server. Save it in
`/home/$USER/prometheus.yml`.

``` yaml
global:
scrape_interval: 15s
scrape_timeout: 10s
scrape_protocols:
- OpenMetricsText1.0.0
- OpenMetricsText0.0.1
- PrometheusText0.0.4
evaluation_interval: 15s
alerting:
alertmanagers:
- follow_redirects: true
enable_http2: true
scheme: http
timeout: 10s
api_version: v2
static_configs:
- targets: []
scrape_configs:
- job_name: prometheus
honor_timestamps: true
track_timestamps_staleness: false
scrape_interval: 15s
scrape_timeout: 10s
scrape_protocols:
- OpenMetricsText1.0.0
- OpenMetricsText0.0.1
- PrometheusText0.0.4
metrics_path: /metrics
scheme: http
enable_compression: true
follow_redirects: true
enable_http2: true
static_configs:
- targets:
- localhost:9090
- localhost:8080 # This is the port where Benchtool will be running
```

Now let's start the Benchtool with Prometheus Server and Grafana
Dashboard.

```bash
docker run -d --name prometheus -p 9090:9090 --network=host -v /home/$USER/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus
docker run -d -p 3000:3000 --network=host grafana/grafana-enterprise
ros2 run benchtool benchtool --ros-args -p toml:=./benchtool.toml -p o_csv:=./results.csv
```

Now you can access Grafana Dashboard at <http://localhost:3000> ,
Prometheus Server at <http://localhost:9090> and Benchtool at
<http://localhost:8080/metrics>

After you open Grafana Dashboard you need to login with default
credentials `(admin:admin)` and add Prometheus Server `(localhost:9090)` as
a data source. Then you can create a new dashboard and add a new graph.
In the query editor you can select the data source and the metric you
want to visualize. Right now Benchtool supports the following metrics:

- `frequency` with labels `topic` and `type`
(e.g. `frequency{topic="/camera/color/image_raw",
type="sensor_msgs/msg/Image"}`)
- `total_messages` with labels `topic` and `type`
(e.g. `total_messages{topic="/camera/color/image_raw",
type="sensor_msgs/msg/Image"}`)

Image of the Grafana Dashboard:

![image](../../../images/grafana_example.png)

This file was deleted.

Loading
Loading