fix: use otlp exporter type instead of otlp_grpc (#194)

florentianayuwono · renovate[bot] · cbartz · web-flow · commit 4e4df51b9752 · 2026-04-30T19:32:42.000+07:00
* feat: enable log forwarding through otel collector * chore(deps): update dependency packaging to v26.1 (#180) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency sphinx-ubuntu-images to v0.2.0 (#181) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): replace astral-sh/setup-uv action with astral-sh/setup-uv v8 (#182) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * fix(deps): update module github.com/jackc/pgx/v5 to v5.9.2 (#183) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * fix(dashboard): remove dead override hiding job queue time series (#184) * fix(dashboard): remove dead override hiding job queue time series The "Job queue time" panel had a leftover hideSeriesFrom override that excluded every series except one specific named expression. Combined with the panel's current query (which already aggregates with sum by (le)), the override hid all data, leaving an empty chart. The override is obsolete now that the query collapses every label except le into a single combined histogram, so removing it restores visualisation without any other change. * ci: pin astral-sh/setup-uv to v8.1.0 The astral-sh/setup-uv repository does not publish a floating v8 major tag (only v8.0.0 and v8.1.0 specific tags exist), so referencing @v8 fails to resolve and breaks the workflow on every PR. * chore: update Copilot collections to v0.11.0 (#172) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Christopher Bartz <christopher.bartz@canonical.com> * chore(docs): update contributing guidelines (charmkeeper) (#170) * chore(docs): update contributing guidelines (charmkeeper) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(docs): build errors * chore: clarify PR checklist and remove unnecessary item * chore(docs): revert changes in CONTRIBUTING.md * fix(docs): whoops we're ignoring the changelog * fix: update pr checklist to incorporate previous items, remove duplicate items --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: run shell as bash * Update docs/how-to/enable-log-forwarding.md Co-authored-by: Erin Conley <erin.conley@canonical.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Erin Conley <erin.conley@canonical.com> * fix docs * add index * Update actions/enable-log-forwarding/enable-log-forwarding.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update actions/enable-log-forwarding/enable-log-forwarding.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * address code reviews Co-authored-by: Copilot <copilot@github.com> * add license Co-authored-by: Copilot <copilot@github.com> * fix docs Co-authored-by: Copilot <copilot@github.com> * refactor code Co-authored-by: Copilot <copilot@github.com> * Apply suggestions from code review Co-authored-by: Erin Conley <erin.conley@canonical.com> * address code reviews Co-authored-by: Copilot <copilot@github.com> * remove pyyaml Co-authored-by: Copilot <copilot@github.com> * parameterized test Co-authored-by: Copilot <copilot@github.com> * fix: use otlp exporter type instead of otlp_grpc The opentelemetry-collector snap does not register an 'otlp_grpc' exporter type. The correct type is 'otlp', which uses gRPC by default. 'otlp_grpc' caused the collector to fail to start after the config fragment was written. Also update the self-hosted smoke test to use the runner-injected OTLP endpoint (removing the hardcoded localhost) and write real log entries after forwarding is configured, so logs reach Loki. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add tls insecure to generated otlp exporter config The OTLP endpoint otlp.actions.self-hosted-runners.internal:4317 uses a self-signed certificate that does not match its hostname. The existing github.yaml config already sets tls.insecure: true for the same endpoint. Add the same setting to the generated exporter fragment to prevent the TLS handshake failure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix logging action * style: fix black formatting and missing final newline Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Christopher Bartz <christopher.bartz@canonical.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Erin Conley <erin.conley@canonical.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Weii Wang <weii.wang@canonical.com>
diff --git a/.github/workflows/enable_log_forwarding_action_tests.yaml b/.github/workflows/enable_log_forwarding_action_tests.yaml
@@ -53,16 +53,22 @@ jobs:
     runs-on: [self-hosted-linux-amd64-noble-edge]
     env:
       TEST_CONFIG_FILE: 91-enable-log-forwarding-smoke-${{ github.run_id }}-${{ github.run_attempt }}.yaml
+      TEST_LOG_FILE: /var/log/enable-log-forwarding-smoke-${{ github.run_id }}.log
     steps:
       - uses: actions/checkout@v6
 
       - name: Run enable log forwarding action
         uses: ./actions/enable-log-forwarding
         with:
-          files: |
-            /var/log/syslog
+          files: ${{ env.TEST_LOG_FILE }}
           config-file-name: ${{ env.TEST_CONFIG_FILE }}
-          otlp-endpoint: 127.0.0.1:4317
+
+      - name: Populate log file to forward
+        run: |
+          for i in {1..5}; do
+            echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) INFO enable-log-forwarding smoke test: entry $i run_id=${{ github.run_id }}" | sudo tee -a ${TEST_LOG_FILE}
+            sleep 2
+          done
 
       - name: Verify generated config file exists
         run: |
diff --git a/actions/enable-log-forwarding/enable_log_forwarding.py b/actions/enable-log-forwarding/enable_log_forwarding.py
@@ -15,7 +15,22 @@
 from typing import Sequence
 
 CONFIG_DIR = "/etc/otelcol/config.d"
-EXPORTER_NAME = "otlp_grpc"
+EXPORTER_NAME = "otlp/github_runner_optin"
+BATCH_PROCESSOR_NAME = "batch/github_runner_optin"
+SERVICE_NAME = "self-hosted-runner"
+LOKI_RESOURCE_LABELS = (
+    "service.name",
+    "github.repository",
+    "github.runner",
+    "github.workflow",
+    "github.job",
+    "github.run.id",
+    "github.run.attempt",
+)
+LOKI_ATTRIBUTE_LABELS = (
+    "log.file.name",
+    "log.file.path",
+)
 SNAP_CMD = Path("/usr/bin/snap")
 SUDO_CMD = Path("/usr/bin/sudo")
 MKDIR_CMD = Path("/usr/bin/mkdir")
@@ -174,16 +189,29 @@ def exporter_exists_in_config_dir(
 def build_resource_attributes() -> list[dict[str, str]]:
     """Build static GitHub resource attributes attached to forwarded logs."""
     attrs = [
+        ("service.name", SERVICE_NAME),
         ("github.repository", os.getenv("GITHUB_REPOSITORY", "unknown")),
         ("github.runner", os.getenv("RUNNER_NAME", "unknown")),
         ("github.workflow", os.getenv("GITHUB_WORKFLOW", "unknown")),
         ("github.job", os.getenv("GITHUB_JOB", "unknown")),
         ("github.run.id", os.getenv("GITHUB_RUN_ID", "unknown")),
         ("github.run.attempt", os.getenv("GITHUB_RUN_ATTEMPT", "unknown")),
+        ("loki.resource.labels", ", ".join(LOKI_RESOURCE_LABELS)),
     ]
     return [{"key": key, "value": value, "action": "upsert"} for key, value in attrs]
 
 
+def build_log_attribute_actions() -> list[dict[str, str]]:
+    """Build log-attribute actions for Loki label promotion hints."""
+    return [
+        {
+            "key": "loki.attribute.labels",
+            "value": ", ".join(LOKI_ATTRIBUTE_LABELS),
+            "action": "upsert",
+        }
+    ]
+
+
 def build_config(
     files: Sequence[str],
     resolved_endpoint: str,
@@ -196,18 +224,28 @@ def build_config(
             "filelog/github_runner_optin": {
                 "include": list(files),
                 "start_at": "end",
+                "include_file_name": True,
+                "include_file_path": True,
             }
         },
         "processors": {
             "resource/github_runner_optin": {
                 "attributes": build_resource_attributes(),
-            }
+            },
+            "attributes/github_runner_optin": {
+                "actions": build_log_attribute_actions(),
+            },
+            BATCH_PROCESSOR_NAME: {},
         },
         "service": {
             "pipelines": {
                 "logs/github_runner_optin": {
                     "receivers": ["filelog/github_runner_optin"],
-                    "processors": ["resource/github_runner_optin", "batch"],
+                    "processors": [
+                        "resource/github_runner_optin",
+                        "attributes/github_runner_optin",
+                        BATCH_PROCESSOR_NAME,
+                    ],
                     "exporters": [exporter_name],
                 }
             }
@@ -217,6 +255,7 @@ def build_config(
         config["exporters"] = {
             exporter_name: {
                 "endpoint": resolved_endpoint,
+                "tls": {"insecure": True},
             }
         }
 
diff --git a/actions/enable-log-forwarding/tests/test_enable_log_forwarding.py b/actions/enable-log-forwarding/tests/test_enable_log_forwarding.py
@@ -159,6 +159,7 @@ def test_build_config_exporter_block_is_conditionally_defined(
     assert ("exporters" in config) is has_exporters_block
     if has_exporters_block:
         assert config["exporters"][module.EXPORTER_NAME]["endpoint"] == "otel:4318"
+        assert config["exporters"][module.EXPORTER_NAME]["tls"] == {"insecure": True}
 
 
 @pytest.mark.parametrize(
