You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-**Explanation**: This command runs GPU monitoring while executing the benchmark specified by the Docker image `synthetic_regression`. The system will collect GPU metrics and generate a completion plot at the end. Live monitoring of GPU metrics is enabled by default.
-**Explanation**: Similar to the first example, this command runs the `synthetic_regression` Docker image benchmark and collects GPU metrics. Additionally, the collected data is exported to Meerkat for long-term storage and further analysis. This is useful when you need to monitor metrics over time and visualize them later using external tools such as the Grafana Dashboard.
-**Explanation**: This is a comprehensive example that runs the `stemdl_classificatio` benchmark in a Docker container and collects GPU metrics at a 10-second interval. The `--carbon_region` flag specifies the carbon intensity region as "South England" to track the carbon emissions impact. Live plotting of GPU metrics is enabled (`--live_plot`), and data will be exported to Meerkat DB via VictoriaMetrics (`--export_to_meerkat`). The `--monitor_logs` flag enables logging of both GPU metrics and the Docker container logs, allowing for deeper analysis of benchmark performance.
26
+
-**Explanation**: This is a comprehensive example that runs the `stemdl_classificatio` benchmark in a Docker container and collects GPU metrics at a 10-second interval. The `--carbon-region` flag specifies the carbon intensity region as "South England" to track the carbon emissions impact. Live plotting of GPU metrics is enabled (`--live-plot`), and data will be exported to Meerkat DB via VictoriaMetrics (`--export-to-meerkat`). The `--monitor-logs` flag enables logging of both GPU metrics and the Docker container logs, allowing for deeper analysis of benchmark performance.
27
27
28
28
### Example 4: Run and Monitor Benchmark in the Background without the Need for a Container:
-**Explanation**: In this example, a benchmark command (`./run_1.sh`) is executed in the background using `tmux` instead of a Docker container. GPU metrics are collected at 1-second intervals, and live plotting of these metrics is enabled. This is useful when you have a script or binary that doesn't require containerization and want to monitor the system's GPU usage in real-time. Running benchmarks in `tmux` allows the process to continue in the background, making it ideal for long-running benchmarks that don't need constant attention.
This command was run on a VM with 2 V100 GPUs forthe resultsin [Collecting Results Section](collecting_results.md#gpu-metrics-timeseries-plot-png).
71
71
72
72
73
73
## (Grafana) Timeseries Plot Live
74
74
75
-
If the `--export_to_meerkat` tag is used the results can be viewed live from the Grafana Dashboard. Data from multiple VMs can be collected all at once allowing for a live comparison of performance as well.
75
+
If the `--export-to-meerkat` tag is used the results can be viewed live from the Grafana Dashboard. Data from multiple VMs can be collected all at once allowing for a live comparison of performance as well.
76
76
77
77
See example results [Collecting Results Section](collecting_results.md#gpu-metric-grafana-plots).
0 commit comments