Feature: Autogenerate Telegraf docs from influxdata/telegraf repo.

.gitignore

@@ -11,6 +11,10 @@ node_modules
 /content/influxdb*/**/api/**/*.html
 !api-docs/**/.config.yml
 /api-docs/redoc-static.html*
+/telegraf-build
+!telegraf-build/templates
+!telegraf-build/scripts
+!telegraf-build/README.md
 .vscode/*
 .idea
 **/config.toml

telegraf-build/README.md

@@ -0,0 +1,99 @@
+
+# Telegraf release and docs build 
+
+The Telegraf release build process (using code in `influxdata/telegraf-internal`) includes automation for building `influxdata/docs-v2` documentation from `influxdata/telegraf` plugin README files.
+
+## Release build process
+
+1. Telegraf team triggers the release script, which generates a `docs` binary.
+   The binary takes a Telegraf release version tag as an argument.
+2. When executed (by the Telegraf team for an official release or on your local
+   system for testing), the binary:
+
+   a. Clones the `docs-v2` repo and the `telegraf` repo and checks out the specified release tag.
+   b. Applies `docs-v2` frontmatter templates (`docs-v2/telegraf-build/templates`) to the `telegraf` plugin README files.
+   c. Commits the changes to the local `docs-v2` repo and creates a PR for the changes.
+
+## Build Telegraf docs for local testing and preview
+
+> [!Warn]
+> 
+> Use the following steps for local testing and preview only.
+>
+> _Don't commit the generated files to version control._
+>
+> Submit edits and fixes to the `/influxdata/telegraf` repo.
+> The Telegraf release process triggers documentation builds and
+> submits them to `influxdata/docs-v2` for review.
+>  
+
+Follow steps to test the Telegraf docs build process and preview generated docs on your local system (not for an official Telegraf release):
+
+- [Build using Docker](#build-using-docker)
+- [Build manually](#build-manually)
+
+### Run Docker to build Telegraf docs for testing and preview 
+
+1. If you don't already have an SSH key pair, generate one for your GitHub-associated email address, add your private key to your SSH agent, and add then add the public key to your GitHub account.
+
+The Dockerfile leverages Docker's BuildKit and the `--ssh` flag to use your SSH keys for GitHub authentication.
+
+1. Open a terminal and navigate to the directory containing the Dockerfile (`./scripts`), then enter the following command to build the Docker image:
+
+   ```bash
+   docker build --ssh default -t telegraf-build .
+   ```
+
+2. Run the Docker container using the built image and mount a volume to `/app/repos/docs-v2/telegraf`:
+
+   ```bash
+   docker run --rm \
+   -v /Users/me/Documents/github/docs-v2/content/telegraf:/app/repos/docs-v2/content/telegraf \
+   telegraf-build v1.33.0
+   ```
+
+Replace `/Users/me/Documents/github/docs-v2/content/telegraf` with the actual path on your host machine where you want to access the generated documentation.
+
+### Manually build Telegraf docs for testing and preview
+
+To test manually run the build process on your local system 
+(without a release triggered by `influxdata/telegraf`):
+
+1. Install a recent version of Go for your system. 
+
+2. Clone the `influxdata/telegraf-internal` repo to your local system (for example, to `~/Documents/github/telegraf-internal`) 
+
+3. To generate the release binaries (`telegraf-internal/telegraf_release/bin/`),change into the `~/Documents/github/telegraf-internal` directory and run `make`. 
+
+4. To generate the documentation, run the `telegraf-internal/telegraf_release/bin/docs` binary and include the Telegraf release tag to build--for example:
+
+  ```bash
+  # Change to `telegraf-build` in your local docs-v2 repo.
+  cd ~/Documents/github/docs-v2/telegraf-build
+   # Run the `docs` binary to generate the documentation.
+  ~/Documents/github/telegraf-internal/telegraf_release/bin/docs v1.33.0
+  ```
+
+  You can skip steps for local testing:
+
+  ```bash
+   ~/Documents/github/telegraf-internal/telegraf_release/bin/docs -skip changelog,pull-request v1.33.0
+   ```
+
+  The binary looks for `.tmpl` template files in `./templates` `telegraf-internal/telegraf_release/docs/templates/`, however we expect to permanently move them to `docs-v2/telegraf-build/templates` soon.
+
+  The `docs` binary:
+    a. Clones the `docs-v2` repo and the `telegraf` repo and checks out the specified Telegraf release tag.
+    b. Commits the changes to the local `docs-v2` repo and creates a PR for the changes.
+
+5. To test the templates and preview the changes on your local machine, change to `telegraf-build/repos/docs-v2`, install dependencies, and start Hugo:
+
+   ```bash
+   cd ~/Documents/github/docs-v2/telegraf-build/repos/docs-v2
+   # Install dependencies
+   yarn install
+   # Start Hugo server
+   npx hugo serve
+   ```
+
+   Alternatively, copy the generated files your existing local `docs-v2` repo (but, _don't_ commit them to version control).

telegraf-build/scripts/Dockerfile

@@ -0,0 +1,46 @@
+# Use the official Golang 1.21 image as the base image
+FROM golang:1.21
+
+# Set the working directory inside the container
+WORKDIR /app
+
+# Install necessary dependencies
+RUN apt-get update && apt-get install -y \
+    git \
+    make \
+    wget \
+    tar \
+    bash \
+    openssh-client
+
+# Configure SSH for GitHub access
+RUN mkdir -p /root/.ssh && chmod 0700 /root/.ssh
+RUN ssh-keyscan github.com >> /root/.ssh/known_hosts
+
+# Clone the telegraf-internal repository using SSH
+# Use the --mount option to access the SSH agent
+RUN --mount=type=ssh git clone [email protected]:influxdata/telegraf-internal.git
+
+# Set the working directory to the cloned repository
+WORKDIR /app/telegraf-internal
+
+# Checkout the docs branch
+RUN git checkout docs
+
+# (Optional) Ensure Go modules are enabled
+ENV GO111MODULE=on
+
+# Fetch dependencies (if using Go modules)
+RUN go mod tidy
+
+# Remove binaries from the previous build
+RUN make clean
+
+# Build the release binaries
+RUN make release
+
+# Set the working directory to the docs directory
+WORKDIR /app/telegraf-internal/telegraf_release/docs
+
+# Run the docs binary. In your docker run command, specify the Telegraf release tag
+CMD ["/app/telegraf-internal/telegraf_release/bin/docs"]

telegraf-build/templates/aggregator_index.tmpl

@@ -0,0 +1,15 @@
+---
+title: "Telegraf Aggregator Plugins"
+description: "Telegraf aggregator plugins aggregator data across multiple metrics."
+menu:
+  telegraf_v1_ref:
+    name: Aggregator plugins
+    identifier: aggregator_plugins_reference
+    weight: 10
+tags: [aggregator-plugins]
+---
+
+Telegraf aggregator plugins aggregator data across multiple metrics using e.g.
+statistical functions like min, max or mean.
+
+{{`{{<children>}}`}}

telegraf-build/templates/aggregator_plugin.tmpl

@@ -0,0 +1,18 @@
+---
+description: "Telegraf plugin for aggregating metrics using {{.Name}}"
+menu:
+  telegraf_v1_ref:
+    parent: aggregator_plugins_reference
+    name: {{.Name}}
+    identifier: aggregator-{{.ID}}
+tags: [{{.Name}}, "aggregator-plugins", "configuration"{{range .Tags}}, "{{.}}"{{end}}]
+{{if .Introduced}}introduced: "{{.Introduced}}"{{end}}
+{{if .Deprecated}}deprecated: "{{.Deprecated}}"{{end}}
+{{if .Removal}}removal: "{{.Removal}}"{{end}}
+{{if .OS}}os_support: "{{.OS}}"{{end}}
+related:
+  - /telegraf/v1/configure_plugins/
+  - {{.URL}}, {{.Name}} Plugin Source
+---
+
+{{.Readme}}

telegraf-build/templates/input_index.tmpl

@@ -0,0 +1,14 @@
+---
+title: "Telegraf Input Plugins"
+description: "Telegraf input plugins collect metrics from the system, services, and third-party APIs."
+menu:
+  telegraf_v1_ref:
+    name: Input plugins
+    identifier: input_plugins_reference
+    weight: 10
+tags: [input-plugins]
+---
+
+Telegraf input plugins collect metrics from the system, services, and third-party APIs.
+
+{{`{{< children >}}`}}

telegraf-build/templates/input_plugin.tmpl

@@ -0,0 +1,18 @@
+---
+description: "Telegraf plugin for collecting metrics from {{.Name}}"
+menu:
+  telegraf_v1_ref:
+    parent: input_plugins_reference
+    name: {{.Name}}
+    identifier: input-{{.ID}}
+tags: [{{.Name}}, "input-plugins", "configuration"{{range .Tags}}, "{{.}}"{{end}}]
+{{if .Introduced}}introduced: "{{.Introduced}}"{{end}}
+{{if .Deprecated}}deprecated: "{{.Deprecated}}"{{end}}
+{{if .Removal}}removal: "{{.Removal}}"{{end}}
+{{if .OS}}os_support: "{{.OS}}"{{end}}
+related:
+  - /telegraf/v1/configure_plugins/
+  - {{.URL}}, {{.Name}} Plugin Source
+---
+
+{{.Readme}}

telegraf-build/templates/output_index.tmpl

@@ -0,0 +1,14 @@
+---
+title: "Telegraf Output Plugins"
+description: "Telegraf output plugins send metrics to various destinations."
+menu:
+  telegraf_v1_ref:
+    name: Output plugins
+    identifier: output_plugins_reference
+    weight: 20
+tags: [output-plugins]
+---
+
+Telegraf output plugins send metrics to various destinations.
+
+{{`{{<children>}}`}}

telegraf-build/templates/output_plugin.tmpl

@@ -0,0 +1,18 @@
+---
+description: "Telegraf plugin for sending metrics to {{.Name}}"
+menu:
+  telegraf_v1_ref:
+    parent: output_plugins_reference
+    name: {{.Name}}
+    identifier: output-{{.ID}}
+tags: [{{.Name}}, "output-plugins", "configuration"{{range .Tags}}, "{{.}}"{{end}}]
+{{if .Introduced}}introduced: "{{.Introduced}}"{{end}}
+{{if .Deprecated}}deprecated: "{{.Deprecated}}"{{end}}
+{{if .Removal}}removal: "{{.Removal}}"{{end}}
+{{if .OS}}os_support: "{{.OS}}"{{end}}
+related:
+  - /telegraf/v1/configure_plugins/
+  - {{.URL}}, {{.Name}} Plugin Source
+---
+
+{{.Readme}}

telegraf-build/templates/plugin_inputs.tmpl

@@ -0,0 +1,13 @@
+---
+description: "Telegraf plugin for collecting metrics from {{.Name}}"
+menu:
+  telegraf_v1_ref:
+    parent: input_plugins_reference
+    name: {{.Name}}
+    identifier: input-{{.ID}}
+tags: [{{.Name}}, "input-plugins", "configuration"]
+related:
+  - /telegraf/v1/configure_plugins/
+---
+
+{{.Readme}}

telegraf-build/templates/plugin_list_entry.tmpl

@@ -0,0 +1,15 @@
+  - name: {{ .Name }}
+    id: {{ .ID }}
+    description: |
+{{ .Description }}
+    introduced: {{ .Introduced }}
+{{- with .Deprecated }}
+    deprecated: {{ . }}
+{{- end }}
+{{- with .Removal }}
+    removal: {{ . }}
+{{- end }}
+{{ with .OS }}
+    os_support: {{ . }}
+{{- end }}
+    tags: [{{ .Tags | join ", " }}]

telegraf-build/templates/plugin_outputs.tmpl

@@ -0,0 +1,13 @@
+---
+description: "Telegraf plugin for sending metrics to {{.Name}}"
+menu:
+  telegraf_v1_ref:
+    parent: output_plugins_reference
+    name: {{.Name}}
+    identifier: output-{{.ID}}
+tags: [{{.Name}}, "output-plugins", "configuration"]
+related:
+  - /telegraf/v1/configure_plugins/
+---
+
+{{.Readme}}

telegraf-build/templates/processor_index.tmpl

@@ -0,0 +1,15 @@
+---
+title: "Telegraf Processor Plugins"
+description: "Telegraf processor plugins transform individual metrics."
+menu:
+  telegraf_v1_ref:
+    name: Processor plugins
+    identifier: processor_plugins_reference
+    weight: 10
+tags: [processor-plugins]
+---
+
+Telegraf processor plugins transform individual metrics by e.g. converting
+tags and fields or data-types.
+
+{{`{{<children>}}`}}

telegraf-build/templates/processor_plugin.tmpl

@@ -0,0 +1,18 @@
+---
+description: "Telegraf plugin for transforming metrics using {{.Name}}"
+menu:
+  telegraf_v1_ref:
+    parent: processor_plugins_reference
+    name: {{.Name}}
+    identifier: processor-{{.ID}}
+tags: [{{.Name}}, "processor-plugins", "configuration"{{range .Tags}}, "{{.}}"{{end}}]
+{{if .Introduced}}introduced: "{{.Introduced}}"{{end}}
+{{if .Deprecated}}deprecated: "{{.Deprecated}}"{{end}}
+{{if .Removal}}removal: "{{.Removal}}"{{end}}
+{{if .OS}}os_support: "{{.OS}}"{{end}}
+related:
+  - /telegraf/v1/configure_plugins/
+  - {{.URL}}, {{.Name}} Plugin Source
+---
+
+{{.Readme}}