Merge pull request #168 from MovieStoreGuy/msg/add-rulers

Include ninja entry for extension

Author: rcastley

content/en/conf/opentelemetry-collector/1-installation.md

@@ -14,13 +14,14 @@ Obtain the `.deb` package for your platform from the [OpenTelemetry Collector Co
 wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.75.0/otelcol-contrib_0.75.0_linux_amd64.deb
 ```
 
-{{%expand "🥷 Ninja: Build your own collector using _Open Telemetry Collector Builder (ocb)_"%}}
+***
+{{%expand "🥷 Ninja: Build your own collector using Open Telemetry Collector Builder (ocb)"%}}
 For this part we will require the following installed on your system:
 
 - Golang (latest version)
 - ocb installed
   - Most recent from the [project releases](https://github.com/open-telemetry/opentelemetry-collector/releases)
-  - Install using go by `go install go.opentelemetry.io/collector/cmd/builder@latest`
+  - Install using go by `go install go.opentelemetry.io/collector/cmd/builder@v0.75.0`
 - (Optional) Docker
 
 ## Why build your own collector?
@@ -80,7 +81,7 @@ exporters:
 - gomod: go.opentelemetry.io/collector/exporter/otlpexporter v0.75.0
 - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/splunkhecexporter v0.75.0
 - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/signalfxexporter v0.75.0
-- gomod:  github.com/open-telemetry/opentelemetry-collector-contrib/exporter/sapmexporter v0.75.0
+- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/sapmexporter v0.75.0
 
 processors:
 - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.75.0
@@ -119,6 +120,7 @@ Which leave you with the following directory structure:
 
 1. https://opentelemetry.io/docs/collector/custom-collector/
 {{% /expand%}}
+***
 
 ## 2. Installing the OpenTelemetry Collector Contrib distribution
 
@@ -204,9 +206,9 @@ cat /etc/otelcol-contrib/config.yaml
 extensions:
   health_check:
   pprof:
-    endpoint: 0.0.0.0:1777
+    endpoint: 127.0.0.1:1777
   zpages:
-    endpoint: 0.0.0.0:55679
+    endpoint: 127.0.0.1:55679
 
 receivers:
   otlp:
@@ -223,7 +225,7 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.1:8888']
 
   jaeger:
     protocols:

content/en/conf/opentelemetry-collector/2-extensions.md

@@ -20,7 +20,7 @@ sudo vi /etc/otelcol-contrib/config.yaml
 ```yaml {hl_lines=[3]}
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
+    endpoint: "127.0.0.0:13133"
 ```
 
 {{% /tab %}}
@@ -29,11 +29,11 @@ extensions:
 ```yaml {hl_lines=[3]}
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
+    endpoint: 127.0.0.0:13133
   pprof:
-    endpoint: 0.0.0.0:1777
+    endpoint: 127.0.0.0:1777
   zpages:
-    endpoint: 0.0.0.0:55679
+    endpoint: 127.0.0.0:55679
 
 receivers:
   otlp:
@@ -50,7 +50,7 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.0:8888']
 
   jaeger:
     protocols:
@@ -94,6 +94,61 @@ Restart the collector:
 sudo systemctl restart otelcol-contrib
 ```
 
+***
+
+{{% expand "🥷 Ninja: Improve data durability with storage extension" %}}
+For this, we will need to validate our distrobution has the `file_storage` extension installed,
+this can be down by running the command `otelcol-contrib components` and it should so something like:
+
+```yaml
+# ... truncated for clarity
+extensions:
+    - file_storage
+```
+
+This extension provides exporters the ability to queue data to disk in the event that exporter is unable
+to send data to the configured endpoint.
+
+In order to configure the extension, you will need to update to include the following information:
+
+```yaml
+extensions:
+  file_storage:
+    directory: /tmp/otel-data
+    timeout: 10s
+    compaction:
+       directory: /tmp/otel-data
+       on_start: true
+       on_rebound: true
+       rebound_needed_threshold_mib: 5
+       rebound_trigger_threshold_mib: 3
+
+# ... truncated for clarity
+
+service:
+  extension:
+  # Additiona extensions here 
+  - file_storage
+```
+
+## Why queue data to disk?
+
+This allows the collector to queue data (and even restart) to ensure data is sent 
+the upstream provider.
+
+## Considerations for queuing data to disk?
+
+There is a potential that this could impact data throughput performance due disk performance.
+
+#### References
+
+1. https://community.splunk.com/t5/Community-Blog/Data-Persistence-in-the-OpenTelemetry-Collector/ba-p/624583
+1. https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/storage/filestorage
+
+{{% /expand %}}
+
+***
+
 ## Health Check
 
 This extension enables an HTTP url that can be probed to check the status of the OpenTelemetry Collector. This extension can be used as a liveness and/or readiness probe on Kubernetes. To learn more about the curl command, check out the [curl man page.](https://curl.se/docs/manpage.html)
@@ -150,4 +205,5 @@ Example URL: [http://localhost:55679/debug/extensionz](http://localhost:55679/de
 
 ![ExtensionZ](../images/extensionz.png)
 
-Now that we have reviewed extensions, lets dive into the data pipeline portion of the workshop. The data pipeline in the OpenTelemetry Collector is made up of receivers, processors, and exporters. We will first start with receivers.
+Now that we have reviewed extensions, lets dive into the data pipeline portion of the workshop. The data pipeline in the OpenTelemetry Collector is made up of receivers, processors, and exporters. We will first start with receivers.
+

content/en/conf/opentelemetry-collector/3-receivers.md

@@ -48,11 +48,11 @@ receivers:
 ```yaml {hl_lines=["9-30"]}
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
+    endpoint: 127.0.0.1:13133
   pprof:
-    endpoint: 0.0.0.0:1777
+    endpoint: 127.0.0.1:1777
   zpages:
-    endpoint: 0.0.0.0:55679
+    endpoint: 127.0.0.1:55679
 
 receivers:
   hostmetrics:
@@ -90,7 +90,7 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.1:8888']
 
   jaeger:
     protocols:
@@ -142,7 +142,7 @@ prometheus/internal:
     - job_name: 'otel-collector'
       scrape_interval: 10s
       static_configs:
-      - targets: ['0.0.0.0:8888']
+      - targets: ['127.0.0.1:8888']
 ```
 
 {{% /tab %}}
@@ -151,11 +151,11 @@ prometheus/internal:
 ```yaml {hl_lines=[39]}
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
+    endpoint: 127.0.0.1:13133
   pprof:
-    endpoint: 0.0.0.0:1777
+    endpoint: 127.0.0.1:1777
   zpages:
-    endpoint: 0.0.0.0:55679
+    endpoint: 127.0.0.1:55679
 
 receivers:
   hostmetrics:
@@ -193,7 +193,7 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.1:8888']
 
   jaeger:
     protocols:

content/en/conf/opentelemetry-collector/4-processors.md

@@ -34,11 +34,11 @@ processors:
 ``` yaml
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
+    endpoint: 127.0.0.1:13133
   pprof:
-    endpoint: 0.0.0.0:1777
+    endpoint: 127.0.0.1:1777
   zpages:
-    endpoint: 0.0.0.0:55679
+    endpoint: 127.0.0.1:55679
 
 receivers:
   hostmetrics:
@@ -76,7 +76,7 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.1:8888']
 
   jaeger:
     protocols:

content/en/conf/opentelemetry-collector/6-service.md

@@ -96,11 +96,11 @@ service:
 ``` yaml
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
+    endpoint: 127.0.0.1:13133
   pprof:
-    endpoint: 0.0.0.0:1777
+    endpoint: 127.0.0.1:1777
   zpages:
-    endpoint: 0.0.0.0:55679
+    endpoint: 127.0.0.1:55679
 
 receivers:
   hostmetrics:
@@ -138,7 +138,7 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.1:8888']
 
   jaeger:
     protocols:

content/en/conf/opentelemetry-collector/_index.md

@@ -97,9 +97,9 @@ Vanilla/Contrib/Vendor
 extensions:
   health_check:
   pprof:
-    endpoint: 0.0.0.0:1777
+    endpoint: 127.0.0.1:1777
   zpages:
-    endpoint: 0.0.0.0:55679
+    endpoint: 127.0.0.1:55679
 
 receivers:
   otlp:
@@ -116,7 +116,7 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.1:8888']
 
   jaeger:
     protocols:
@@ -169,10 +169,10 @@ service:
 
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
+    endpoint: 127.0.0.1:13133
   http_forwarder:
     ingress:
-      endpoint: 0.0.0.0:6060
+      endpoint: 127.0.0.1:6060
     egress:
       endpoint: "${SPLUNK_API_URL}"
       # Use instead when sending to gateway
@@ -182,7 +182,7 @@ extensions:
     collectd:
       configDir: "${SPLUNK_COLLECTD_DIR}"
   zpages:
-    #endpoint: 0.0.0.0:55679
+    #endpoint: 127.0.0.1:55679
   memory_ballast:
     # In general, the ballast should be set to 1/3 of the collector's memory, the limit
     # should be 90% of the collector's memory.
@@ -211,19 +211,19 @@ receivers:
   jaeger:
     protocols:
       grpc:
-        endpoint: 0.0.0.0:14250
+        endpoint: 127.0.0.1:14250
       thrift_binary:
-        endpoint: 0.0.0.0:6832
+        endpoint: 127.0.0.1:6832
       thrift_compact:
-        endpoint: 0.0.0.0:6831
+        endpoint: 127.0.0.1:6831
       thrift_http:
-        endpoint: 0.0.0.0:14268
+        endpoint: 127.0.0.1:14268
   otlp:
     protocols:
       grpc:
-        endpoint: 0.0.0.0:4317
+        endpoint: 127.0.0.1:4317
       http:
-        endpoint: 0.0.0.0:4318
+        endpoint: 127.0.0.1:4318
   # This section is used to collect the OpenTelemetry Collector metrics
   # Even if just a Splunk APM customer, these metrics are included
   prometheus/internal:
@@ -232,23 +232,23 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.1:8888']
         metric_relabel_configs:
           - source_labels: [ __name__ ]
             regex: '.*grpc_io.*'
             action: drop
   smartagent/signalfx-forwarder:
     type: signalfx-forwarder
-    listenAddress: 0.0.0.0:9080
+    listenAddress: 127.0.0.1:9080
   smartagent/processlist:
     type: processlist
   signalfx:
-    endpoint: 0.0.0.0:9943
+    endpoint: 127.0.0.1:9943
     # Whether to preserve incoming access token and use instead of exporter token
     # default = false
     #access_token_passthrough: true
   zipkin:
-    endpoint: 0.0.0.0:9411
+    endpoint: 127.0.0.1:9411
 
 processors:
   batch:
@@ -356,11 +356,11 @@ End goal for metrics only:
 ```yaml
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
+    endpoint: 127.0.0.1:13133
   pprof:
-    endpoint: 0.0.0.0:1777
+    endpoint: 127.0.0.1:1777
   zpages:
-    endpoint: 0.0.0.0:55679
+    endpoint: 127.0.0.1:55679
 
 receivers:
   hostmetrics:
@@ -387,7 +387,7 @@ receivers:
       - job_name: 'otel-collector'
         scrape_interval: 10s
         static_configs:
-        - targets: ['0.0.0.0:8888']
+        - targets: ['127.0.0.1:8888']
 
 processors:
   batch: