Nr 370484 fb metrics (#2033)#2034

Open

voorepreethi wants to merge 7 commits intomasterfrom

NR-370484-fb-metrics

Collaborator

voorepreethi commented Apr 14, 2025

This PR includes Fluentbit metrics forwarder in the default IA based on feature flag : "fluent_bit_metrics"
For more details please refer jira: https://new-relic.atlassian.net/browse/NR-370484


          Nr 370484 fb metrics (#2033)

99244a6

* feat: Updated Fb metrics by default

* feat: Updated test cases

* feat:Updated host details

* feat: Updated test cases and scrape interval

voorepreethi requested a review from a team as a code owner

April 14, 2025 05:09

rajrohanyadav mentioned this pull request

feat: Updated Fb metrics by default #2031

Closed

voorepreethi added 4 commits

April 14, 2025 14:33


          fix: Updated linting

4a3bd80


          fix: Updated lint

149d9ce


          fix: Updated liniting

5f18f41


          fix: Updated test cases

93e98a2

coveralls commented Apr 15, 2025

Pull Request Test Coverage Report for Build 14462832288

Details

95 of 110 (86.36%) changed or added relevant lines in 4 files are covered.
3 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.1%) to 57.491%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/integrations/v4/logs/loader.go	2	4	50.0%
pkg/integrations/v4/logs/cfg.go	90	103	87.38%

Files with Coverage Reduction	New Missed Lines	%
internal/agent/event_sender.go	1	78.11%
internal/integrations/v4/testhelp/testemit/testemit.go	2	90.63%

Totals
Change from base Build 14325595165:	0.1%
Covered Lines:	15476
Relevant Lines:	26919

💛 - Coveralls

maya-jha reviewed

View reviewed changes

internal/agent/cmdchannel/fflag/ffhandler.go

               		flag == FlagDmRegisterDeprecated ||
-              		flag == FlagFullInventoryDeletion
+              		flag == FlagFullInventoryDeletion ||
+              		flag == FlagFluentBitMetrics

maya-jha Apr 15, 2025

How does this feature flag work? At what stage its being used? During installation?

Collaborator Author

voorepreethi Apr 16, 2025

Right now this works on start of agent , so on every FF change we need to restart the Agent. Let me check if there is a way to not restart

pkg/integrations/v4/logs/cfg.go Show resolved Hide resolved

pkg/integrations/v4/logs/cfg.go Outdated

Comment on lines 421 to 422

		MetricsPath: "/api/v2/metrics/prometheus",
		ScrapeInterval: "60s",

maya-jha Apr 15, 2025

make them constant.

maya-jha suggested changes

View reviewed changes

maya-jha left a comment

Please provide details about how changes were tested.
Any changes in default logging.yml?
What changes will be required to get it working with super agent?
Here option to turn off metrics would be more important as there is no way to update this.
Have you looked at health check options. Can this be useful in future ?https://docs.fluentbit.io/manual/administration/monitoring#health-check-for-fluent-bit

pkg/integrations/v4/logs/cfg.go Outdated

               	var fbOSConfig FBOSConfig
               	addOSDependantConfig(&fbOSConfig)
+              	enableMetrics := isMetricsEnabled(ff)

maya-jha Apr 16, 2025

Will user have option to turn it off? Do we have PM alignment on this if we are not offering the ability to turn it off?

Collaborator Author

voorepreethi Apr 16, 2025

Yes. we has a alignment with PM
cc: @nr-rkallempudi

pkg/integrations/v4/logs/cfg.go Outdated Show resolved Hide resolved

pkg/integrations/v4/logs/cfg_template.go

Comment on lines +129 to +145

+                  Log_Level     {{ .LogLevel }}
+                  {{- end }}
+                  {{- if .Daemon }}
+                  Daemon        {{ .Daemon }}
+                  {{- end }}
+                  {{- if .ParsersFile }}
+                  Parsers_File  {{ .ParsersFile }}
+                  {{- end }}
+                  {{- if .HTTPServer }}
+                  HTTP_Server   {{ .HTTPServer }}
+                  {{- end }}
+                  {{- if .HTTPListen }}
+                  HTTP_Listen   {{ .HTTPListen }}
+                  {{- end }}
+                  {{- if .HTTPPort }}
+                  HTTP_Port     {{ .HTTPPort }}
+                  {{- end }}

maya-jha Apr 16, 2025

from where we are getting these values?

Collaborator Author

voorepreethi Apr 16, 2025

sorry?

pkg/integrations/v4/logs/cfg.go

-              	// Newrelic OUTPUT plugin will send all the collected logs to Vortex
-              	fb.Output = newNROutput(logFwdCfg)
+              	// Including Prometheus scrapper input plugin by default to pull Fluent bit metrics based on ff
+              	includePrometheusScrapperInputPlugin(&fb, enableMetrics)

maya-jha Apr 16, 2025

Are we planning to send all metrics? What are the cost implications here?

Collaborator Author

voorepreethi Apr 16, 2025

Yes. We are planning to send all the metrics. Will share the Cost analysis doc


          fix: Updated constants

6a50663

Collaborator Author

voorepreethi commented Apr 17, 2025 •

edited

Loading

@maya-jha 1. Please provide details about how changes were tested.
--> https://github.com/newrelic/infrastructure-agent/blob/master/README.md#compile-and-build-the-agent. Build the code locally and update the IA bundles with the locally build files and restarting the Agent .
2. Any changes in default logging.yml?
--> No
3. What changes will be required to get it working with super agent?
--> IA is already having Super Agent support , so No Changes required for this changes to be included.
4. Here option to turn off metrics would be more important as there is no way to update this.
--> I have raised this point , with PM and Rajeev, they asked for a similar experience like k8s.
5. Have you looked at health check options. Can this be useful in future ?https://docs.fluentbit.io/manual/administration/monitoring#health-check-for-fluent-bit
Sure , we can check on this


          feat: added hostid

a2c63ec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet