|
| 1 | +# Instrumenting a Java Application on Azure App Service with OpenTelemetry |
| 2 | + |
| 3 | +This example demonstrates how to instrument a Java Application on Azure |
| 4 | +App Service using OpenTelemetry, |
| 5 | +and export traces, metrics, and logs to a collector running in Azure, which will then |
| 6 | +export that data to Splunk. |
| 7 | + |
| 8 | +We'll show how the splunk-otel-javaagent.jar file can be uploaded to Azure and added |
| 9 | +to the application startup command. |
| 10 | + |
| 11 | +The example application is based on the [default Spring Boot web application template](https://github.com/rd-1-2022/rest-service), |
| 12 | +and the deployment steps are based on [Quickstart: Create a Java app on Azure App Service](https://learn.microsoft.com/en-us/azure/app-service/quickstart-java?tabs=springboot&pivots=java-javase). |
| 13 | + |
| 14 | +## Prerequisites |
| 15 | + |
| 16 | +The following tools are required to build and deploy the Java application to Azure: |
| 17 | + |
| 18 | +* Java 17+ JDK |
| 19 | +* [Maven](https://maven.apache.org/install.html) |
| 20 | +* An Azure account with permissions to create and execute Azure web applications |
| 21 | +* [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) |
| 22 | + |
| 23 | +## Splunk Distribution of the OpenTelemetry Collector |
| 24 | + |
| 25 | +For this example, we deployed the Splunk Distribution of the OpenTelemetry Collector onto a virtual machine |
| 26 | +in Azure using Gateway mode, and ensured it's accessible to our Azure function. |
| 27 | + |
| 28 | +We configured it with the `SPLUNK_HEC_TOKEN` and `SPLUNK_HEC_URL` environment variables, so that it |
| 29 | +exports logs to our Splunk Cloud instance. |
| 30 | + |
| 31 | +Please refer to [Install the Collector using packages and deployment tools](https://docs.splunk.com/observability/en/gdi/opentelemetry/install-the-collector.html#collector-package-install) |
| 32 | +for collector installation instructions. |
| 33 | + |
| 34 | +## Authenticate with Azure |
| 35 | + |
| 36 | +Open a command line terminal and use the following command to authenticate with Azure: |
| 37 | + |
| 38 | +``` bash |
| 39 | +az login |
| 40 | +``` |
| 41 | + |
| 42 | +## Configure the Maven Plugin |
| 43 | + |
| 44 | +We'll use the Maven plugin to deploy our Java application to Azure. In order to do this, we'll |
| 45 | +need to configure it first with the following command: |
| 46 | + |
| 47 | +``` bash |
| 48 | +cd my-webapp |
| 49 | +mvn com.microsoft.azure:azure-webapp-maven-plugin:2.13.0:config |
| 50 | +``` |
| 51 | + |
| 52 | +When prompted, provide the following inputs: |
| 53 | + |
| 54 | +* Create new configuration: `Y` |
| 55 | +* Define value for OS: `2` (for Linux) |
| 56 | +* Define value for javaVersion: `1` (for Java 17) |
| 57 | +* Define value for pricingTier: `3` (for P1v2) |
| 58 | +* Confirm: `Y` |
| 59 | + |
| 60 | +## Build and Deploy the Java application to Azure |
| 61 | + |
| 62 | +Let's build the application with Maven: |
| 63 | + |
| 64 | +``` bash |
| 65 | +mvn clean package |
| 66 | +``` |
| 67 | + |
| 68 | +And then deploy it to Azure: |
| 69 | + |
| 70 | +``` bash |
| 71 | +mvn azure-webapp:deploy |
| 72 | +``` |
| 73 | + |
| 74 | +## Test the Application |
| 75 | + |
| 76 | +Point your browser to `http://<appName>.azurewebsites.net/greeting` to test the application. |
| 77 | + |
| 78 | +> Note: you'll need to substitute your application name into the above URL, which was provided as |
| 79 | +> an output from the earlier Maven command used to deploy the application. |
| 80 | +
|
| 81 | +You should see output such as the following: |
| 82 | + |
| 83 | +```` |
| 84 | +{"id":1,"content":"Hello, World!"} |
| 85 | +```` |
| 86 | + |
| 87 | +## Download the latest splunk-otel-javaagent.jar file |
| 88 | + |
| 89 | +Let's download the latest splunk-otel-javaagent.jar file, which we'll use |
| 90 | +to instrument our Java application: |
| 91 | + |
| 92 | +```` |
| 93 | +curl -L https://github.com/signalfx/splunk-otel-java/releases/latest/download/splunk-otel-javaagent.jar \ |
| 94 | +-o splunk-otel-javaagent.jar |
| 95 | +```` |
| 96 | + |
| 97 | +## Upload splunk-otel-javaagent.jar |
| 98 | + |
| 99 | +To instrument our Java application running in Azure with OpenTelemetry, we'll |
| 100 | +need to upload the `splunk-otel-javaagent.jar` file to Azure with the following |
| 101 | +command: |
| 102 | + |
| 103 | +``` bash |
| 104 | +az webapp deploy --src-path splunk-otel-javaagent.jar \ |
| 105 | + --type static \ |
| 106 | + --target-path /home/site/wwwroot/splunk-otel-javaagent.jar \ |
| 107 | + --resource-group <resource group name> \ |
| 108 | + --name <app name> |
| 109 | +``` |
| 110 | + |
| 111 | +> Note: you'll need to substitute the resource group name and app name for your application |
| 112 | +> before running the above command. |
| 113 | +
|
| 114 | +## Set Environment Variables |
| 115 | + |
| 116 | +Next, we'll set the following environment variables, which tells `splunk-otel-javaagent.jar` |
| 117 | +what data to collect and where to send it: |
| 118 | + |
| 119 | +* `JAVA_OPTS`: `-javaagent:/home/site/wwwroot/splunk-otel-javaagent.jar` |
| 120 | +* `OTEL_SERVICE_NAME`: `azure-java-web-app` |
| 121 | +* `OTEL_EXPORTER_OTLP_ENDPOINT`: `http(s)://<collector URL>:4318` |
| 122 | +* `OTEL_RESOURCE_ATTRIBUTES`: `deployment.environment=test` |
| 123 | +* `SPLUNK_PROFILER_ENABLED`: `true` |
| 124 | +* `SPLUNK_PROFILER_MEMORY_ENABLED`: `true` |
| 125 | + |
| 126 | +> Note: remember to substitute the URL for your collector |
| 127 | +
|
| 128 | +The environment variables should look like the following in Azure: |
| 129 | + |
| 130 | + |
| 131 | + |
| 132 | +The approach used to upload `splunk-otel-javaagent.jar` to Azure and activate it by setting the `JAVA_OPTS` environment |
| 133 | +variable is based on the steps in [Manually upgrade the monitoring extension/agent](https://learn.microsoft.com/en-us/azure/azure-monitor/app/codeless-app-service?tabs=java#manually-upgrade-the-monitoring-extensionagent). |
| 134 | + |
| 135 | +> Important: Setting `JAVA_OPTS` requires Application Insights to be disabled in the Azure portal for the application being |
| 136 | +> monitored. If this is not possible, then follow the steps in |
| 137 | +> [Manually upgrade the monitoring extension/agent](https://learn.microsoft.com/en-us/azure/azure-monitor/app/codeless-app-service?tabs=java#manually-upgrade-the-monitoring-extensionagent) |
| 138 | +> to include `-javaagent:/home/site/wwwroot/splunk-otel-javaagent.jar` in the Startup Command instead. |
| 139 | +
|
| 140 | +Note that we've enabled both the CPU and memory profiler, so we can utilize the |
| 141 | +AlwaysOn Profiling capabilities. |
| 142 | + |
| 143 | +## Test the Application |
| 144 | + |
| 145 | +Access the application by navigating your web browser to the following URL: |
| 146 | + |
| 147 | +```` |
| 148 | +http://<appName>.azurewebsites.net/greeting |
| 149 | +```` |
| 150 | + |
| 151 | +As before, you should see output such as the following: |
| 152 | + |
| 153 | +```` |
| 154 | +{"id":1,"content":"Hello, World!"} |
| 155 | +```` |
| 156 | + |
| 157 | +### View Traces in Splunk Observability Cloud |
| 158 | + |
| 159 | +After a minute or so, you should start to see traces for the Java application |
| 160 | +appearing in Splunk Observability Cloud: |
| 161 | + |
| 162 | + |
| 163 | + |
| 164 | +### View AlwaysOn Profiling Data in Splunk Observability Cloud |
| 165 | + |
| 166 | +You should also see profiling data appear: |
| 167 | + |
| 168 | + |
| 169 | + |
| 170 | +### View Metrics in Splunk Observability Cloud |
| 171 | + |
| 172 | +Metrics are collected by splunk-otel-javaagent.jar automatically. For example, |
| 173 | +the `jvm.memory.used` metric shows us the amount of memory used in the JVM |
| 174 | +by type of memory: |
| 175 | + |
| 176 | + |
| 177 | + |
| 178 | +### View Logs with Trace Context |
| 179 | + |
| 180 | +The Splunk Distribution of OpenTelemetry Java automatically adds trace context |
| 181 | +to logs. However, it doesn't add this context to the actual log file (unless |
| 182 | +you explicitly configure the logging framework to do so). Instead, the trace |
| 183 | +context is added behind the scenes to the log events exported to the |
| 184 | +OpenTelemetry Collector. |
| 185 | + |
| 186 | +For example, if we add the debug exporter to the logs pipeline of the collector, |
| 187 | +we can see that the trace_id and span_id have been added to the following log event |
| 188 | +for our application: |
| 189 | + |
| 190 | +```` |
| 191 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: ScopeLogs #0 |
| 192 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: ScopeLogs SchemaURL: |
| 193 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: InstrumentationScope org.springframework.web.servlet.DispatcherServlet |
| 194 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: LogRecord #0 |
| 195 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: ObservedTimestamp: 2025-02-10 16:15:59.063905628 +0000 UTC |
| 196 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Timestamp: 2025-02-10 16:15:59.063822027 +0000 UTC |
| 197 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: SeverityText: INFO |
| 198 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: SeverityNumber: Info(9) |
| 199 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Body: Str(Initializing Servlet 'dispatcherServlet') |
| 200 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Trace ID: 551fa951d47e609c046e23dbef393c03 |
| 201 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Span ID: c62ad3bd39da28ec |
| 202 | +Feb 10 16:15:59 splunk-otel-collector otelcol[776]: Flags: 1 |
| 203 | +```` |
| 204 | + |
| 205 | +The OpenTelemetry Collector can be configured to export log data to |
| 206 | +Splunk platform using the Splunk HEC exporter. The logs can then be made |
| 207 | +available to Splunk Observability Cloud using Log Observer Connect. This will |
| 208 | +provide full correlation between spans generated by Java instrumentation |
| 209 | +with metrics and logs. |
0 commit comments