few nts

Author: cijothomas

tools/pipeline_perf_test/backend/readme.md

@@ -1,9 +1,13 @@
-# Ingestion Service / Backend
+# Backend Service
 
-A destination to which telemetry is exported to ultimately.
+The backend service acts as a destination for exported telemetry data. It
+currently supports OTLP/gRPC on port `5317`, counts the logs it receives, and
+exposes these count at the `:5000/metrics` endpoint.
 
-- Options include:
-  - Null Sink (i.e., drop everything)
-  - A fake service that adds configurable latency and tracks requests received.
-  - A real backend for validating full pipeline integrity (vendor forks may
-    leverage this)
+## Planned Enhancements
+
+- A Null Sink to discard all incoming data.
+- A mock service that introduces configurable latency and tracks incoming
+  requests.
+- A fully functional backend to validate end-to-end pipeline integrity (allowing
+  vendor-specific forks to extend functionality).

tools/pipeline_perf_test/load_generator/readme.md

@@ -1,8 +1,11 @@
 # Load Generator
 
-Scripts or other mechanisms to generate load against
-system under test.
+A simple Python script that continuously sends OTLP logs for a specified
+duration. At the end of the run, it outputs the total count of logs sent to
+stdout, which can be parsed to determine the number of logs sent.
 
-- Initially, a basic script firing OTLP requests, later extended to:
-  - Use language-specific OpenTelemetry SDKs
-  - Integrate load gen tools like Locust or custom telemetry generators
+## Future Enhancements
+
+- Utilize language-specific OpenTelemetry SDKs.
+- Integrate load generation tools like Locust or custom telemetry generators.
+- Extend the script to support configurable options such as log size and length.

tools/pipeline_perf_test/orchestrator/orchestrator.py

@@ -172,8 +172,14 @@ def get_backend_received_count() -> int:
         print(f"Failed to query backend service: {e}")
         return -1
 
-# example usage
-# python3 orchestrator/orchestrator.py --collector-config system_under_test/otel-collector/collector-config.yaml --duration 30
+# Pre-requisites:
+# 1. Create and activate a virtual environment:
+#    python3 -m venv venv
+#    source venv/bin/activate  # On Windows use: venv\Scripts\activate
+# 2. Install dependencies:
+#    pip install -r requirements.txt
+# 3. Run the orchestrator:
+#    python3 orchestrator/orchestrator.py --collector-config system_under_test/otel-collector/collector-config.yaml --duration 30
 def main():
     parser = argparse.ArgumentParser(description="Orchestrate OTel pipeline perf test")
     parser.add_argument("--duration", type=int, default=10, help="Duration to perform perf test in seconds")