diff --git a/doca-demo/README.md b/doca-demo/README.md new file mode 100644 index 0000000..244fd14 --- /dev/null +++ b/doca-demo/README.md @@ -0,0 +1,256 @@ +# OpenTelemetry C++ Starter for DOCA + +## Prerequisites + +Ensure you have the following installed: + +- **Git** +- **C++ Compiler** +- **Make** +- **CMake** +- **Docker** (for running the OTLP collector and Jeager) + +### Create Project Directory + +Create a folder named otel-cpp-starter. + +move into the newly created folder. This will serve as your working directory. + +```bash +mkdir otel-cpp-starter +cd otel-cpp-starter +``` + +Next, install and build OpenTelemetry C++ locally using CMake, following these steps: + +In your terminal, navigate back to the otel-cpp-starter directory. Then, clone the OpenTelemetry C++ GitHub repository to your local machine. + +```bash +git clone https://github.com/open-telemetry/opentelemetry-cpp.git +``` + +Change your working directory to the OpenTelemetry C++ SDK directory. + +```bash +cd opentelemetry-cpp +``` + +Create a build directory and navigate into it. + +```bash +mkdir build +cd build +``` + +In the build directory run CMake, to configure and generate the build system without enabling tests: + +```bash +cmake -DBUILD_TESTING=OFF .. +Or, if the cmake --build fails, you can also try: + +cmake -DBUILD_TESTING=OFF -DWITH_ABSEIL=ON .. +Execute the build process: + +cmake --build . +Install OpenTelemetry C++ in otel-cpp-starter/otel-cpp: + +cmake --install . --prefix ../../otel-cpp +``` + +## Traces + +### Initialize tracing + +```bash +auto provider = opentelemetry::trace::Provider::GetTracerProvider(); +auto tracer = provider->GetTracer("foo_library", "1.0.0"); +``` + +The TracerProvider acquired in the first step is a singleton object that is usually provided by the OpenTelemetry C++ SDK. It is used to provide specific implementations for API interfaces. In case no SDK is used, the API provides a default no-op implementation of a TracerProvider. + +The Tracer acquired in the second step is needed to create and start Spans. + +#### Start a span + +```bash +auto span = tracer->StartSpan("HandleRequest"); +``` + +This creates a span, sets its name to "HandleRequest", and sets its start time to the current time. Refer to the API documentation for other operations that are available to enrich spans with additional data. + +#### Mark a span as active + +```bash +auto scope = tracer->WithActiveSpan(span); +``` + +This marks a span as active and returns a Scope object. The scope object controls how long a span is active. The span remains active for the lifetime of the scope object. + +The concept of an active span is important, as any span that is created without explicitly specifying a parent is parented to the currently active span. A span without a parent is called root span. + +## Exporters + +### Available exporters + +The registry contains a list of exporters for C++. + +Among exporters, OpenTelemetry Protocol (OTLP) exporters are designed with the OpenTelemetry data model in mind, emitting OTel data without any loss of information. Furthermore, many tools that operate on telemetry data support OTLP (such as Prometheus, Jaeger, and most vendors), providing you with a high degree of flexibility when you need it. To learn more about OTLP, see OTLP Specification. + +This page covers the main OpenTelemetry C++ exporters and how to set them up. + +## OTLP Exporter + +### Collector Setup + +In an empty directory, create a file called collector-config.yaml with the following content: + +```yaml +receivers: + otlp: + protocols: + grpc: + endpoint: 0.0.0.0:4317 + http: + endpoint: 0.0.0.0:4318 +exporters: + debug: + verbosity: detailed +service: + pipelines: + traces: + receivers: [otlp] + exporters: [debug] + metrics: + receivers: [otlp] + exporters: [debug] + logs: + receivers: [otlp] + exporters: [debug] +``` + +Now run the collector in a docker container: + +```bash +docker run -p 4317:4317 -p 4318:4318 --rm -v $(pwd)/collector-config.yaml:/etc/otelcol/config.yaml otel/opentelemetry-collector +``` + +This collector is now able to accept telemetry via OTLP. Later you may want to configure the collector to send your telemetry to your observability backend. + +#### Dependencies + +If you want to send telemetry data to an OTLP endpoint (like the OpenTelemetry Collector, Jaeger or Prometheus), you can choose between two different protocols to transport your data: + +HTTP/protobuf +gRPC +Make sure that you have set the right cmake build variables while building OpenTelemetry C++ from source: + +```bash +-DWITH_OTLP_GRPC=ON: To enable building OTLP gRPC exporter. +-DWITH_OTLP_HTTP=ON: To enable building OTLP HTTP exporter. +``` + +In this tutorial, we use HTTP endpoint. + +## Usage + +```bash +#include "opentelemetry/exporters/otlp/otlp_http_exporter_factory.h" +#include "opentelemetry/exporters/otlp/otlp_http_exporter_options.h" +#include "opentelemetry/sdk/trace/processor.h" +#include "opentelemetry/sdk/trace/batch_span_processor_factory.h" +#include "opentelemetry/sdk/trace/batch_span_processor_options.h" +#include "opentelemetry/sdk/trace/tracer_provider_factory.h" +#include "opentelemetry/trace/provider.h" +#include "opentelemetry/sdk/trace/tracer_provider.h" + +#include "opentelemetry/exporters/otlp/otlp_http_metric_exporter_factory.h" +#include "opentelemetry/exporters/otlp/otlp_http_metric_exporter_options.h" +#include "opentelemetry/metrics/provider.h" +#include "opentelemetry/sdk/metrics/aggregation/default_aggregation.h" +#include "opentelemetry/sdk/metrics/export/periodic_exporting_metric_reader.h" +#include "opentelemetry/sdk/metrics/export/periodic_exporting_metric_reader_factory.h" +#include "opentelemetry/sdk/metrics/meter_context_factory.h" +#include "opentelemetry/sdk/metrics/meter_provider.h" +#include "opentelemetry/sdk/metrics/meter_provider_factory.h" + +#include "opentelemetry/exporters/otlp/otlp_http_log_record_exporter_factory.h" +#include "opentelemetry/exporters/otlp/otlp_http_log_record_exporter_options.h" +#include "opentelemetry/logs/provider.h" +#include "opentelemetry/sdk/logs/logger_provider_factory.h" +#include "opentelemetry/sdk/logs/processor.h" +#include "opentelemetry/sdk/logs/simple_log_record_processor_factory.h" + +namespace trace_api = opentelemetry::trace; +namespace trace_sdk = opentelemetry::sdk::trace; + +namespace metric_sdk = opentelemetry::sdk::metrics; +namespace metrics_api = opentelemetry::metrics; + +namespace otlp = opentelemetry::exporter::otlp; + +namespace logs_api = opentelemetry::logs; +namespace logs_sdk = opentelemetry::sdk::logs; + + + +void InitTracer() + { + // Create an OpenTelemetry Protocol (OTLP) exporter. + auto exporter = otlp::OtlpHttpExporterFactory::Create(opts); + auto processor = trace_sdk::SimpleSpanProcessorFactory::Create(std::move(exporter)); + + resource::ResourceAttributes attributes = { + {resource::SemanticConventions::kServiceName, "DOCA NVIDIA DEMO"}, // The application name. + {resource::SemanticConventions::kHostName, "$"} + }; + auto resource = opentelemetry::sdk::resource::Resource::Create(attributes); + + std::shared_ptr provider = + trace_sdk::TracerProviderFactory::Create(std::move(processor), std::move(resource)); + + // Set the trace provider. + trace::Provider::SetTracerProvider(provider); + } +void CleanupTracer() +{ + std::shared_ptr none; + trace::Provider::SetTracerProvider(none); +} +``` + +and then we define our function traces and we pass the service name and operation name on to it. + +```bash +void traces(std::string serviceName, std::string operationName) +{ + auto span = getTracer(serviceName)->StartSpan(operationName); + + span->SetAttribute("service.name", serviceName); + + span->End(); +} +``` + +we call it by using the traces and can pass the service and operation names respectively + +```bash + traces("doca", "doca prepare"); +``` + +The traces are called and it send the traces to otlp endpoint. + +After that, by using otlp the traces is sent to the jeager and it can be accessed on http... + +The Build steps are as follows, + +```bash +meson build +cd build +./doca_telemetry_export +``` + +make sure to run the docker containers for otel and jeager before executing above commands. + +the will display on browser on [Jeager](https://localhost:16686) + +![demo](images/demo.png) diff --git a/doca-demo/images/demo.png b/doca-demo/images/demo.png new file mode 100644 index 0000000..ba6f593 Binary files /dev/null and b/doca-demo/images/demo.png differ diff --git a/doca-demo/main.cc b/doca-demo/main.cc new file mode 100644 index 0000000..0ce69ce --- /dev/null +++ b/doca-demo/main.cc @@ -0,0 +1,329 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "opentelemetry/exporters/otlp/otlp_http_exporter_factory.h" +#include "opentelemetry/exporters/otlp/otlp_http_exporter_options.h" +#include "opentelemetry/context/propagation/global_propagator.h" +#include "opentelemetry/context/propagation/text_map_propagator.h" +#include "opentelemetry/exporters/ostream/span_exporter_factory.h" +#include "opentelemetry/nostd/shared_ptr.h" +#include "opentelemetry/sdk/trace/simple_processor_factory.h" +#include "opentelemetry/sdk/trace/tracer_context.h" +#include "opentelemetry/sdk/trace/tracer_context_factory.h" +#include "opentelemetry/sdk/trace/tracer_provider_factory.h" +#include "opentelemetry/trace/propagation/http_trace_context.h" +#include "opentelemetry/trace/provider.h" +#include "opentelemetry/ext/http/client/http_client_factory.h" +#include "opentelemetry/sdk/resource/semantic_conventions.h" +#include "opentelemetry/sdk/common/global_log_handler.h" + +#define NB_EXAMPLE_STRINGS 5 /* Amount of example strings */ +#define MAX_EXAMPLE_STRING_SIZE 256 /* Indicates the max length of string */ +#define SINGLE_FIELD_VALUE 1 /* Indicates the field contains one value */ +namespace trace = opentelemetry::trace; +namespace nostd = opentelemetry::nostd; +namespace trace_sdk = opentelemetry::sdk::trace; +namespace otlp = opentelemetry::exporter::otlp; +namespace internal_log = opentelemetry::sdk::common::internal_log; +namespace resource = opentelemetry::sdk::resource; + +opentelemetry::nostd::shared_ptr getTracer( + std::string serviceName) +{ + auto provider = opentelemetry::trace::Provider::GetTracerProvider(); + return provider->GetTracer(serviceName); +} + opentelemetry::exporter::otlp::OtlpHttpExporterOptions opts; + + +void traces(std::string serviceName, std::string operationName) +{ + auto span = getTracer(serviceName)->StartSpan(operationName); + + span->SetAttribute("service.name", serviceName); + + span->End(); +} + +void InitTracer() + { + // Create an OpenTelemetry Protocol (OTLP) exporter. + auto exporter = otlp::OtlpHttpExporterFactory::Create(opts); + auto processor = trace_sdk::SimpleSpanProcessorFactory::Create(std::move(exporter)); + + resource::ResourceAttributes attributes = { + {resource::SemanticConventions::kServiceName, "DOCA NVIDIA DEMO"}, // The application name. + {resource::SemanticConventions::kHostName, "$"} + }; + auto resource = opentelemetry::sdk::resource::Resource::Create(attributes); + + std::shared_ptr provider = + trace_sdk::TracerProviderFactory::Create(std::move(processor), std::move(resource)); + + // Set the trace provider. + trace::Provider::SetTracerProvider(provider); + } + + void CleanupTracer() + { + std::shared_ptr none; + trace::Provider::SetTracerProvider(none); + } + + nostd::shared_ptr get_tracer() + { + auto provider = trace::Provider::GetTracerProvider(); + return provider->GetTracer("library name to trace", OPENTELEMETRY_SDK_VERSION); + } +DOCA_LOG_REGISTER(TELEMETRY); + +static std::vector example_strings = { + "example_str_1", "example_str_2", "example_str_3", "example_str_4", "example_str_5" +}; + +/* Event struct from which report will be serialized */ +struct TestEventType { + doca_telemetry_timestamp_t timestamp; + int32_t event_number; + int32_t iter_number; + uint64_t string_number; + char example_string[MAX_EXAMPLE_STRING_SIZE]; +} __attribute__((packed)); + +/* + * This function fills up event buffer with the example string of specified number. + * It also saves number of iteration, number of string and overall number of events. + */ +static doca_error_t prepare_example_event(int32_t iter_number, uint64_t string_number, TestEventType &event) { + static int collected_example_events_count = 0; + doca_telemetry_timestamp_t timestamp; + doca_error_t result = DOCA_SUCCESS; + traces("doca", "doca prepare"); +result = doca_telemetry_get_timestamp(×tamp); + + if (result != DOCA_SUCCESS) + return result; + + event.timestamp = timestamp; + event.event_number = collected_example_events_count++; + event.iter_number = iter_number; + event.string_number = string_number; + + if (example_strings[string_number].length() >= MAX_EXAMPLE_STRING_SIZE) + return DOCA_ERROR_INVALID_VALUE; + + strncpy(event.example_string, example_strings[string_number].c_str(), MAX_EXAMPLE_STRING_SIZE); + return DOCA_SUCCESS; +} + +/* + * Registers the example fields to the doca_telemetry_type. + */ +static doca_error_t telemetry_register_fields(doca_telemetry_type *type) { + doca_error_t result; + doca_telemetry_field *field; + const int nb_fields = 5; + + struct { + const char *name; + const char *desc; + const char *type_name; + uint16_t len; + } fields_info[] = { + {"timestamp", "Event timestamp", DOCA_TELEMETRY_FIELD_TYPE_TIMESTAMP, SINGLE_FIELD_VALUE}, + {"event_number", "Event number", DOCA_TELEMETRY_FIELD_TYPE_INT32, SINGLE_FIELD_VALUE}, + {"iter_num", "Iteration number", DOCA_TELEMETRY_FIELD_TYPE_INT32, SINGLE_FIELD_VALUE}, + {"string_number", "String number", DOCA_TELEMETRY_FIELD_TYPE_UINT64, SINGLE_FIELD_VALUE}, + {"example_string", "String example", DOCA_TELEMETRY_FIELD_TYPE_CHAR, MAX_EXAMPLE_STRING_SIZE}, + }; + + for (int i = 0; i < nb_fields; i++) { + result = doca_telemetry_field_create(&field); + if (result != DOCA_SUCCESS) + return result; + traces("register fields", fields_info[i].name); + + doca_telemetry_field_set_name(field, fields_info[i].name); + doca_telemetry_field_set_description(field, fields_info[i].desc); + doca_telemetry_field_set_type_name(field, fields_info[i].type_name); + doca_telemetry_field_set_array_len(field, fields_info[i].len); + + result = doca_telemetry_type_add_field(type, field); + if (result != DOCA_SUCCESS) + return result; + } + + return result; +} + +doca_error_t telemetry_export() { + bool file_write_enable = true; + bool ipc_enabled = true; + int repetition = 10; + doca_error_t result; + int32_t iteration = 0; + uint64_t string_number = 0; + doca_telemetry_schema *doca_schema = nullptr; + doca_telemetry_source *doca_source = nullptr; + TestEventType test_event; + doca_telemetry_type_index_t example_index; + doca_telemetry_type *example_type; + + result = doca_telemetry_schema_init("example_doca_schema_name", &doca_schema); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot init doca schema"); + return result; + } + + doca_telemetry_schema_set_buf_size(doca_schema, sizeof(test_event) * 5); + + result = doca_telemetry_type_create(&example_type); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot create type"); + goto err_schema; + } + + result = telemetry_register_fields(example_type); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot register fields"); + doca_telemetry_type_destroy(example_type); + goto err_schema; + } + + if (file_write_enable) + doca_telemetry_schema_set_file_write_enabled(doca_schema); + + if (ipc_enabled) + doca_telemetry_schema_set_ipc_enabled(doca_schema); + + result = doca_telemetry_schema_add_type(doca_schema, "example_event", example_type, &example_index); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot add type to doca_schema!"); + goto err_schema; + } + + result = doca_telemetry_schema_start(doca_schema); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot start doca_schema!"); + goto err_schema; + } + + result = doca_telemetry_source_create(doca_schema, &doca_source); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot create doca_source!"); + goto err_schema; + } + + doca_telemetry_source_set_id(doca_source, "source_1"); + doca_telemetry_source_set_tag(doca_source, "source_1_tag"); + + result = doca_telemetry_source_start(doca_source); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot start doca_source!"); + goto err_source; + } + + for (iteration = 0; iteration < repetition; iteration++) { + for (string_number = 0; string_number < NB_EXAMPLE_STRINGS; string_number++) { + result = prepare_example_event(iteration, string_number, test_event); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Failed to create event"); + goto err_source; + } + + result = doca_telemetry_source_report(doca_source, example_index, &test_event, 1); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot report to doca_source!"); + goto err_source; + + } + + if (iteration % 2 == 0) { + /* + * Optionally force DOCA Telemetry source buffer to flush. + * Handy for bursty events or specific event types. + */ + result = doca_telemetry_source_flush(doca_source); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Cannot flush doca_source!"); + goto err_source; + } + } + } + + } + + doca_telemetry_source_destroy(doca_source); + doca_telemetry_schema_destroy(doca_schema); + return DOCA_SUCCESS; + +err_source: + doca_telemetry_source_destroy(doca_source); +err_schema: + doca_telemetry_schema_destroy(doca_schema); + return result; +} + +int main(int argc, char *argv[]) { + InitTracer(); + + doca_error_t result; + struct doca_log_backend *sdk_log; + int exit_status = EXIT_FAILURE; + + /* Register a logger backend */ + result = doca_log_backend_create_standard(); + if (result != DOCA_SUCCESS) + goto sample_exit; + + /* Register a logger backend for internal SDK errors and warnings */ + result = doca_log_backend_create_with_file_sdk(stderr, &sdk_log); + if (result != DOCA_SUCCESS) + goto sample_exit; + result = doca_log_backend_set_sdk_level(sdk_log, DOCA_LOG_LEVEL_WARNING); + if (result != DOCA_SUCCESS) + goto sample_exit; + + DOCA_LOG_INFO("Starting the sample"); + + /* Parse cmdline/json arguments */ + result = doca_argp_init("doca_telemetry_export", NULL); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Failed to init ARGP resources: %s", doca_error_get_descr(result)); + goto sample_exit; + } + result = doca_argp_start(argc, argv); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("Failed to parse sample input: %s", doca_error_get_descr(result)); + goto argp_cleanup; + } + result = telemetry_export(); + if (result != DOCA_SUCCESS) { + DOCA_LOG_ERR("telemetry_export() encountered an error: %s", doca_error_get_descr(result)); + goto argp_cleanup; + } + + exit_status = EXIT_SUCCESS; + +argp_cleanup: + doca_argp_destroy(); +sample_exit: + if (exit_status == EXIT_SUCCESS) + DOCA_LOG_INFO("Sample finished successfully"); + else + DOCA_LOG_INFO("Sample finished with errors"); + CleanupTracer(); + + return exit_status; + +} diff --git a/doca-demo/meson.build b/doca-demo/meson.build new file mode 100644 index 0000000..28bce76 --- /dev/null +++ b/doca-demo/meson.build @@ -0,0 +1,68 @@ +# +# Copyright (c) 2022-2024 NVIDIA CORPORATION & AFFILIATES, ALL RIGHTS RESERVED. +# +# This software product is a proprietary product of NVIDIA CORPORATION & +# AFFILIATES (the "Company") and all right, title, and interest in and to the +# software product, including all associated intellectual property rights, are +# and shall remain exclusively with the Company. +# +# This software product is governed by the End User License Agreement +# provided with the software product +# + +project('DOCA_SAMPLE', 'C', 'CPP', + # Get version number from file. + version: run_command(find_program('cat'), + files('/opt/mellanox/doca/applications/VERSION'), check: true).stdout().strip(), + license: 'Proprietary', + default_options: ['buildtype=debug'], + meson_version: '>= 0.61.2' +) + +SAMPLE_NAME = 'telemetry_export' + +# Comment this line to restore warnings of experimental DOCA features +add_project_arguments('-D DOCA_ALLOW_EXPERIMENTAL_API', language: ['c', 'cpp']) + +sample_dependencies = [] +# Required for all DOCA programs +sample_dependencies += dependency('doca-common') +# The DOCA library of the sample itself +sample_dependencies += dependency('doca-telemetry') +# Utility DOCA library for executables +sample_dependencies += dependency('doca-argp') +sample_dependencies += dependency('libcurl') + sample_dependencies +=dependency('opentelemetry-cpp', modules: [ + 'opentelemetry-cpp::proto', + 'opentelemetry-cpp::api', + 'opentelemetry-cpp::sdk', + 'opentelemetry-cpp::common', + 'opentelemetry-cpp::trace', + 'opentelemetry-cpp::metrics', + 'opentelemetry-cpp::logs', + 'opentelemetry-cpp::version', + 'opentelemetry-cpp::resources', + 'opentelemetry-cpp::ext', + 'opentelemetry-cpp::http_client_curl', + 'opentelemetry-cpp::otlp_recordable', + 'opentelemetry-cpp::otlp_http_client', + 'opentelemetry-cpp::otlp_http_exporter', + 'opentelemetry-cpp::otlp_http_log_record_exporter', + 'opentelemetry-cpp::otlp_http_metric_exporter', + 'opentelemetry-cpp::ostream_span_exporter', + 'opentelemetry-cpp::ostream_metrics_exporter', + 'opentelemetry-cpp::ostream_log_record_exporter', + 'opentelemetry-cpp::in_memory_span_exporter', + 'opentelemetry-cpp::in_memory_metric_exporter' + ]) +sample_srcs = [ + # Main function for the executable + 'main.cc', +] + +sample_inc_dirs = [] + +executable('doca_' + SAMPLE_NAME, sample_srcs, + dependencies : sample_dependencies, + include_directories: sample_inc_dirs, + install: false)