Reformat OT instrumentation guide

Author: lindenmckenzie

guides/OT_INSTRUMENTATION.md

@@ -1,15 +1,16 @@
-Instrumenting ONS services for Open Telemetry
-===============
+# Instrumenting ONS services for Open Telemetry
+
+## Instrumenting Java services for OT
 
-# Instrumenting Java services for OT
 These lines to be added in the Dockerfile(s), and to any run.sh scripts that are used locally:
 
 `-javaagent:target/dependency/aws-opentelemetry-agent-1.31.0.jar \`
 
 `-Dotel.propagators=tracecontext,baggage \`
 
 For instance:
-```
+
+```sh
 ENTRYPOINT java $JAVA_OPTS \
 -Drestolino.realm=$REALM \
 -Drestolino.files=$RESTOLINO_STATIC \
@@ -21,9 +22,9 @@ ENTRYPOINT java $JAVA_OPTS \
 com.github.davidcarboni.restolino.Main
 ```
 
-
 The following entry needs to be added to the pom:
-```
+
+```xml
 <!-- OpenTelemetry-->
 <dependency>
     <groupId>software.amazon.opentelemetry</groupId>
@@ -32,7 +33,6 @@ The following entry needs to be added to the pom:
 </dependency>
 ```
 
-
 The following environment variables need to be set on the instance:
 
 `OTEL_SERVICE_NAME=<service name>`
@@ -43,9 +43,9 @@ The URL specified has an identifier of http. In fact the protocol in use is GRPC
 on GRPC). However, the java URL parsing lib doesn't recognise 'grpc://' as a valid protocol, so the configuration
 requires it to be specified as above.
 
-
 Spans can be created around individual calls within the code as follows:
-```
+
+```java
 import io.opentelemetry.api.GlobalOpenTelemetry;
 import io.opentelemetry.api.trace.Span;
 import io.opentelemetry.context.Scope;
@@ -67,41 +67,43 @@ try (Scope scope = span.makeCurrent()) {
 }
 ```
 
-
 (NB - using the tracer pulled from the global scope this way works, but it's preferable to instantiate the tracer in the init code for the service then inject it into your method. See here for full guidance: [Open Telemetry Docs](https://opentelemetry.io/docs/instrumentation/java/manual/#:~:text=To%20create%20Spans%2C%20you%20only,set%20by%20the%20OpenTelemetry%20SDK.&text=It's%20required%20to%20call%20end,you%20want%20it%20to%20end)).
 
-
-## Logging Implementation
+### Logging Implementation
 
 The existing logging library has been modified to extract the traceId from the traceparent header (if it exists) and add it to the TraceId section of the log message. This will enable log entries to be correlated with trace ids which will allow engineers to zero in on problems quickly and accurately.
 
-
 The original logging library was modified in order to manage the change centrally and avoid the need for code changes across multiple applications.
 
+## Instrumenting Go services for OT
 
-# Instrumenting Go services for OT
 The following environment variables need to be created:
-```
+
+```go
 OTServiceName              string        `envconfig:"OTEL_SERVICE_NAME"`
 OTExporterOTLPEndpoint     string        `envconfig:"OTEL_EXPORTER_OTLP_ENDPOINT"`
 OTBatchTimeout             time.Duration `envconfig:"OTEL_BATCH_TIMEOUT"`
 ```
+
 These can then be set in the config:
-```
+
+```go
 cfg = &Config{
     OTExporterOTLPEndpoint:     "localhost:4317",
     OTServiceName:              "service-name",
     OTBatchTimeout:              5 * time.Second,
 }
 ```
-Note that the exporter endpoint is `<hostname>:<port>`, unlike the java configuration there is no protocol identifier
+
+Note that the exporter endpoint is `<hostname>:<port>`, unlike the Java configuration there is no protocol identifier
 
 Import the shared init library for go dp-otel-go
 
 `import "github.com/ONSdigital/dp-otel-go"`
 
 From the init code of the library initialise the otel services:
-```
+
+```go
 //Set up OpenTelemetry
 cfg, err := config.Get()
 
@@ -122,12 +124,14 @@ defer func() {
     err = errors.Join(err, otelShutdown(context.Background()))
 }()
 ```
+
 NB: if this isn't done any calls to the otel service will fail silently. If you find that traces are not coming through, ensure this code is getting called.
 
+### Instrumenting http handlers
 
-## Instrumenting http handlers
 There are a wide range of different facilities for instrumenting http calls. The simplest (taken here from dp-search-api) simply creates a new opentelemetry handler to pass to the server and attaches otelmux middlewarer to the router:
-```
+
+```go
 import "go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"
 import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
 ...
@@ -139,10 +143,9 @@ router.Use(otelmux.Middleware(cfg.OTServiceName))
 server := serviceList.GetHTTPServer(cfg.BindAddr, otelHandler)
 ```
 
-
 Where gorillamux or Chi is not being used for the router, it may be necessary to instrument individual routes as follows:
 
-```
+```go
 func routes(router *mux.Router, hc *healthcheck.HealthCheck) *RendererAPI {
     api := RendererAPI{router: router}
 
@@ -160,28 +163,31 @@ func routes(router *mux.Router, hc *healthcheck.HealthCheck) *RendererAPI {
 ```
 
 Where the server is configured with an api field:
-```
+
+```go
 return &Service{
-	api:                 searchAPI,
+  api:                 searchAPI,
 }
 ```
-Handlers can be wrapped as below: 
-```
+
+Handlers can be wrapped as below:
+
+```go
 func (a *SearchAPI) RegisterGetSearch(...) *SearchAPI {
-	a.Router.Handle(
-		"/search",
-		otelhttp.NewHandler(
-			SearchHandlerFunc(
+  a.Router.Handle(
+    "/search",
+    otelhttp.NewHandler(
+      SearchHandlerFunc(
                 ...
-			), "/search"),
-	).Methods(http.MethodGet)
-	return a
+      ), "/search"),
+  ).Methods(http.MethodGet)
+  return a
 }
 ```
 
-
 The following shows an alternative way to instrument:
-```
+
+```go
 func CreateRendererAPI(ctx context.Context, bindAddr string, allowedOrigins string, errorChan chan error, hc *healthcheck.HealthCheck) {
     router := mux.NewRouter()
     routes(router, hc)
@@ -201,35 +207,37 @@ func CreateRendererAPI(ctx context.Context, bindAddr string, allowedOrigins stri
 }
 ```
 
-
 A purely middleware approach can also be taken where Alice is already in place chaining middleware. Both otelmux and otelhttp are used here to capture all requests with sufficient detail. Here you can see an example instrumentation:
-```
+
+```go
 func New(cfg Config) http.Handler {
     router := mux.NewRouter()
-	router.Use(otelmux.Middleware(cfg.OTServiceName))
-	middleware := []alice.Constructor{
-		otelhttp.NewMiddleware(cfg.OTServiceName),
+  router.Use(otelmux.Middleware(cfg.OTServiceName))
+  middleware := []alice.Constructor{
+    otelhttp.NewMiddleware(cfg.OTServiceName),
         ...
-	}
-	newAlice := alice.New(middleware...).Then(router)
+  }
+  newAlice := alice.New(middleware...).Then(router)
 }
 ```
 
-## Instrumenting http calls
+### Instrumenting http calls
+
 Outgoing service calls need to be instrumented to include the traceparent header when the handler itself is not instrumented. This can be done as follows:
-```
+
+```go
 import ("go.opentelemetry.io/otel"
 "go.opentelemetry.io/otel/propagation")
 ...
 
 otel.GetTextMapPropagator().Inject(req.Context(), propagation.HeaderCarrier(req.Header))
 ```
 
-
-## Manually adding spans:
+### Manually adding spans
 
 Similarly to the Java approach, you can create a span manually as follows:
-```
+
+```go
 import ("go.opentelemetry.io/otel")
 ...
 ...
@@ -239,18 +247,19 @@ ctx, span := tracer.Start(r.Context(), "table render span")
 defer span.End()
 ```
 
-## Mongo Instrumentation:
+### Mongo Instrumentation
+
 The `dp-mongodb` package has been instrumented centrally as of version 3.7.0. This means that there is no need for additional instrumentation in services that import this package version or above. Make sure this version or above is imported!
 
+### Kafka Instrumentation
 
-## Kafka Instrumentation:
 The `dp-kafka` package has also be instrumented centerally as of version 4, this requires the context to be passed in order to work:
-```
+
+```go
 kafkaProducer.Channels().Output <- kafka.BytesMessage{Value: bytes, Context: ctx}
 ```
 
-
-## Logging Implementation
+### Logging Implementation - Go
 
 Go http request middleware was created to extract the traceId from the traceparent header and insert into the expected place in the request context (as controlled by the RequestIdKey in the github.com/ONSdigital/dp-net/v2/request package)