[tmpnet] Enable monitoring of nodes running in kube

[maru-ava]

PR Chain: tmpnet+kube

This PR chain enables tmpnet to deploy temporary networks to Kubernetes. Early PRs refactor tmpnet to support the addition in #3615 of a new tmpnet node runtime for kube.

• [tmpnet] s/Network.ChainConfigs/Network.PrimaryChainConfigs/ #3854
• [tmpnet] Provide genesis, subnet and chain config via content flags #3857
• [tmpnet] Ensure Node has a reference to Network #3870
• [tmpnet] Switch back to using maps for subnet config #3877
• [tmpnet] Refactor runtime configuration in preparation for kube #3867
• [tmpnet] Unify start network flag usage between e2e and tmpnetctl #3871
• [tmpnet] Avoid serializing the node data directory #3881
• [tmpnet] Rename NodeProcess to ProcessRuntime #3890
• [tmpnet] Switch FlagsMap from map[string]any to map[string]string #3884
• [tmpnet] Ensure tmpnet methods always have a logger #3893
• [tmpnet] Ensure all node runtime methods accept a context #3894
• [tmpnet] Move WaitForHealthy from a function to a tmpnet.Node method #3896
• [tmpnet] Delegate writing of the flag file to the runtime #3897
• [tmpnet] Move monitoring label handling to node #3898
• [tmpnet] Enable runtime-specific restart behavior #3882
• [tmpnet] Define reusable flags for configuring kubernetes client access #3945
• [tmpnet] Separate start of prometheus and promtail collectors #3947
• [tmpnet] Ensure GetNodeURIs returns locally-accessible URIs to ensure kube compatibility #3973
• [tmpnet] Enable deployment to kube #3615
• >>>>>> [tmpnet] Enable monitoring of nodes running in kube #3794 <<<<<<

Why this should be merged

• Ensures nodes deployed to kube have labels and annotations to enable the same labeling as nodes running a processes.
• Enables deployment of prometheus and promtail to kube to ensure collection of logs and metrics from nodes deployed on the cluster.

How this works

• adds labels and annotations to node pods
• enables deployment of the collectors via tmpnetctl start-kind-cluster --start-metrics-collector --start-logs-collector
• updates the kube e2e ci job to check that logs and metrics were collected
• defines collector deployment in yaml for compatibility with argocd usage

How this was tested

CI configures a metrics check to verify that logs and metrics were collected

Need to be documented in RELEASES.md?

N/A

TODO

• merge [tmpnet] Enable deployment to kube #3615
• Verify that metrics and logs are collected from nodes running in kubernetes

[DeferCleanup (Suite)] 
/home/runner/work/avalanchego/avalanchego/tests/fixture/e2e/ginkgo_test_context.go:96

  Timeline >>
   [05-28|19:02:40.088] INFO tmpnet/check_monitoring.go:79 checking if logs exist {"url": "https://loki-poc.avax-dev.network/", "query": "sum(count_over_time({network_uuid=\"27080d29-7152-4a5c-a883-af4b6a47700b\"}[1h]))"}
   [05-28|19:02:40.507] INFO tmpnet/check_monitoring.go:45 collected count is non-zero {"type": "logs", "count": 381224}
  << Timeline
------------------------------
[DeferCleanup (Suite)] PASSED [0.291 seconds]
[DeferCleanup (Suite)] 
/home/runner/work/avalanchego/avalanchego/tests/fixture/e2e/ginkgo_test_context.go:96

  Timeline >>
   [05-28|19:02:40.509] INFO tmpnet/check_monitoring.go:186 checking if metrics exist {"url": "https://prometheus-poc.avax-dev.network/", "query": "count({network_uuid=\"27080d29-7152-4a5c-a883-af4b6a47700b\"})"}
   [05-28|19:02:40.800] INFO tmpnet/check_monitoring.go:45 collected count is non-zero {"type": "metrics", "count": 7497}
  << Timeline
------------------------------
[DeferCleanup (Suite)] PASSED [0.000 seconds]

[Copilot]

Pull Request Overview

Enables deployment and testing of Prometheus and Promtail collectors in a local Kind cluster via tmpnetctl start-kind-cluster, wiring new flags through CLI, tests, and Kubernetes manifests.

• Introduce CollectorVars to register --start-metrics-collector and --start-logs-collector flags and corresponding test checks
• Add YAML manifests for Promtail DaemonSet and Prometheus Agent StatefulSet
• Integrate DeployKubeCollectors into StartKindCluster and propagate monitoring labels in NewNodeStatefulSet

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/upgrade/upgrade_test.go	Swap individual bool flags for unified CollectorVars
tests/fixture/tmpnet/yaml/promtail-daemonset.yaml	Add Promtail DaemonSet manifest
tests/fixture/tmpnet/yaml/prometheus-agent.yaml	Add Prometheus Agent StatefulSet manifest
tests/fixture/tmpnet/tmpnetctl/main.go	Wire CollectorVars into start-kind-cluster command
tests/fixture/tmpnet/start_kind_cluster.go	Update StartKindCluster signature and invoke DeployKubeCollectors
tests/fixture/tmpnet/monitor_kube.go	Implement DeployKubeCollectors and helper functions
tests/fixture/tmpnet/kube_runtime.go	Pass monitoring labels to node runtime
tests/fixture/tmpnet/kube.go	Extend NewNodeStatefulSet to apply labels and annotations
tests/fixture/tmpnet/flags/collector.go	Define CollectorVars and flag registration
tests/fixture/tmpnet/README.md	Document new flags and YAML directory
scripts/start_kind_cluster.sh	Propagate new start flags in helper script