|
| 1 | +--- |
| 2 | +title: "How to export telemetry data to HyperDX" |
| 3 | +description: "Complete guide to exporting telemetry data with OpenTelemetry and HyperDX. Learn how to configure collectors, visualize traces, logs, metrics, and debug distributed applications effectively." |
| 4 | +url: /examples/hyperdx_tutorial/ |
| 5 | +--- |
| 6 | + |
| 7 | +[HyperDX](https://hyperdx.io) is an open source observability platform that |
| 8 | +unifies logs, traces, metrics, exceptions, and session replays into a single |
| 9 | +interface. It helps developers debug applications faster by providing a complete |
| 10 | +view of your system's behavior and performance. |
| 11 | + |
| 12 | +[OpenTelemetry](https://opentelemetry.io/) (often abbreviated as OTel) provides |
| 13 | +a standardized way to collect and export telemetry data. Deno includes built-in |
| 14 | +OpenTelemetry support, allowing you to instrument your applications without |
| 15 | +additional dependencies. This integration works seamlessly with platforms like |
| 16 | +HyperDX to collect and visualize telemetry data. |
| 17 | + |
| 18 | +In this tutorial, we'll build a simple application and export its telemetry data |
| 19 | +to HyperDX: |
| 20 | + |
| 21 | +- [Setting up your application](#setup-your-app) |
| 22 | +- [Configuring the OpenTelemetry collector](#setup-the-collector) |
| 23 | +- [Viewing telemetry data](#viewing-telemetry-data) |
| 24 | +- [Next steps](#whats-next) |
| 25 | + |
| 26 | +You can find the complete source code for this tutorial |
| 27 | +[on GitHub](https://github.com/denoland/examples/tree/main/with-hyperdx). |
| 28 | + |
| 29 | +## Set up the app |
| 30 | + |
| 31 | +For this tutorial, we'll use a simple chat application to demonstrate how to |
| 32 | +export telemetry data. You can find the |
| 33 | +[code for the app on GitHub](https://github.com/denoland/examples/tree/main/with-hyperdx). |
| 34 | + |
| 35 | +Either take a copy of that repository or create a |
| 36 | +[main.ts](https://github.com/denoland/examples/blob/main/with-hyperdx/main.ts) |
| 37 | +file and a |
| 38 | +[.env](https://github.com/denoland/examples/blob/main/with-hyperdx/.env.example) |
| 39 | +file. |
| 40 | + |
| 41 | +In order to run the app you will need an OpenAI API key. You can get one by |
| 42 | +signing up for an account at [OpenAI](https://platform.openai.com/signup) and |
| 43 | +creating a new secret key. You can find your API key in the |
| 44 | +[API keys section](https://platform.openai.com/account/api-keys) of your OpenAI |
| 45 | +account. Once you have an API key, set up an `OPENAI_API-KEY` environment |
| 46 | +variable in your `.env` file: |
| 47 | + |
| 48 | +```env title=".env" |
| 49 | +OPENAI_API_KEY=your_openai_api_key |
| 50 | +``` |
| 51 | + |
| 52 | +## Set up the collector |
| 53 | + |
| 54 | +First, create a free HyperDX account to get your API key. Then, we'll set up two |
| 55 | +files to configure the OpenTelemetry collector: |
| 56 | + |
| 57 | +1. Create a `Dockerfile`: |
| 58 | + |
| 59 | +```dockerfile title="Dockerfile" |
| 60 | +FROM otel/opentelemetry-collector:latest |
| 61 | + |
| 62 | +COPY otel-collector.yml /otel-config.yml |
| 63 | + |
| 64 | +CMD ["--config", "/otel-config.yml"] |
| 65 | +``` |
| 66 | + |
| 67 | +This Dockerfile: |
| 68 | + |
| 69 | +- Uses the official OpenTelemetry Collector as the base image |
| 70 | +- Copies your configuration into the container |
| 71 | +- Sets up the collector to use your config when it starts |
| 72 | + |
| 73 | +2. Create a file called `otel-collector.yml`: |
| 74 | + |
| 75 | +```yml title="otel-collector.yml" |
| 76 | +receivers: |
| 77 | + otlp: |
| 78 | + protocols: |
| 79 | + grpc: |
| 80 | + endpoint: 0.0.0.0:4317 |
| 81 | + http: |
| 82 | + endpoint: 0.0.0.0:4318 |
| 83 | + |
| 84 | +exporters: |
| 85 | + otlphttp/hdx: |
| 86 | + endpoint: "https://in-otel.hyperdx.io" |
| 87 | + headers: |
| 88 | + authorization: $_HYPERDX_API_KEY |
| 89 | + compression: gzip |
| 90 | + |
| 91 | +processors: |
| 92 | + batch: |
| 93 | + |
| 94 | +service: |
| 95 | + pipelines: |
| 96 | + traces: |
| 97 | + receivers: [otlp] |
| 98 | + processors: [batch] |
| 99 | + exporters: [otlphttp/hdx] |
| 100 | + metrics: |
| 101 | + receivers: [otlp] |
| 102 | + processors: [batch] |
| 103 | + exporters: [otlphttp/hdx] |
| 104 | + logs: |
| 105 | + receivers: [otlp] |
| 106 | + processors: [batch] |
| 107 | + exporters: [otlphttp/hdx] |
| 108 | +``` |
| 109 | +
|
| 110 | +This configuration file sets up the OpenTelemetry collector to receive telemetry |
| 111 | +data from your application and export it to HyperDX. It includes: |
| 112 | +
|
| 113 | +- The receivers section accepts data via gRPC (4317) and HTTP (4318) |
| 114 | +- The Exporters section sends data to HyperDX with compression and |
| 115 | + authentication |
| 116 | +- The processors section batches telemetry data for efficient transmission |
| 117 | +- The pipelines section defines separate flows for logs, traces, and metrics |
| 118 | +
|
| 119 | +Build and run the docker instance to start collecting your telemetry data with |
| 120 | +the following command: |
| 121 | +
|
| 122 | +```sh |
| 123 | +docker build -t otel-collector . && docker run -p 4317:4317 -p 4318:4318 otel-collector |
| 124 | +``` |
| 125 | + |
| 126 | +## Generating telemetry data |
| 127 | + |
| 128 | +Now that we have the app and the docker container set up, we can start |
| 129 | +generating telemetry data. Run your application with these environment variables |
| 130 | +to send data to the collector: |
| 131 | + |
| 132 | +```sh |
| 133 | +OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 \ |
| 134 | +OTEL_SERVICE_NAME=chat-app \ |
| 135 | +OTEL_DENO=true \ |
| 136 | +deno run --unstable-otel --allow-net --allow-env --env-file --allow-read main.ts |
| 137 | +``` |
| 138 | + |
| 139 | +This command: |
| 140 | + |
| 141 | +- Points the OpenTelemetry exporter to your local collector (`localhost:4318`) |
| 142 | +- Names your service "chat-app" in HyperDX |
| 143 | +- Enables Deno's OpenTelemetry integration |
| 144 | +- Runs your application with the necessary permissions |
| 145 | + |
| 146 | +To generate some telemetry data, make a few requests to your running application |
| 147 | +in your browser at [`http://localhost:8000`](http://localhost:8000). |
| 148 | + |
| 149 | +Each request will: |
| 150 | + |
| 151 | +1. Generate traces as it flows through your application |
| 152 | +2. Send logs from your application's console output |
| 153 | +3. Create metrics about the request performance |
| 154 | +4. Forward all this data through the collector to HyperDX |
| 155 | + |
| 156 | +## Viewing telemetry data |
| 157 | + |
| 158 | +In your HyperDX dashboard, you'll see different views of your telemetry data: |
| 159 | + |
| 160 | +### Logs View |
| 161 | + |
| 162 | + |
| 163 | + |
| 164 | +Click any log to see details: |
| 165 | + |
| 166 | + |
| 167 | +### Request Traces |
| 168 | + |
| 169 | +See all logs within a single request: |
| 170 | + |
| 171 | + |
| 172 | +### Metrics Dashboard |
| 173 | + |
| 174 | +Monitor system performance: |
| 175 | + |
| 176 | + |
| 177 | +🦕 Now that you have telemetry export working, you could: |
| 178 | + |
| 179 | +1. Add custom spans and attributes to better understand your application |
| 180 | +2. Set up alerts based on latency or error conditions |
| 181 | +3. Deploy your application and collector to production using platforms like: |
| 182 | + - [Fly.io](https://docs.deno.com/examples/deploying_deno_with_docker/) |
| 183 | + - [Digital Ocean](https://docs.deno.com/examples/digital_ocean_tutorial/) |
| 184 | + - [AWS Lightsail](https://docs.deno.com/examples/aws_lightsail_tutorial/) |
| 185 | + |
| 186 | +🦕 For more details on OpenTelemetry configuration with HyperDX, see their |
| 187 | +[documentation](https://www.hyperdx.io/docs/install/opentelemetry). |
0 commit comments