Skip to content

[TT-15243] Influx2 latency metric #900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
andrei-tyk merged 2 commits into from
Oct 9, 2025
Merged

[TT-15243] Influx2 latency metric #900

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
13c57d9
feat: add latency metrics to influx2 pump
deeparavirdc Jul 9, 2025
9af7ab7
Merge branch 'master' into influx2-latency-metric
andrei-tyk Oct 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 16 additions & 14 deletions pumps/influx2.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,24 +191,26 @@
for _, v := range data {
ar := v.(analytics.AnalyticsRecord)
mapping := map[string]interface{}{
"method": ar.Method,
"path": ar.Path,
"response_code": ar.ResponseCode,
"api_key": ar.APIKey,
"time_stamp": ar.TimeStamp,
"api_version": ar.APIVersion,
"api_name": ar.APIName,
"api_id": ar.APIID,
"org_id": ar.OrgID,
"oauth_id": ar.OauthID,
"raw_request": ar.RawRequest,
"request_time": ar.RequestTime,
"raw_response": ar.RawResponse,
"ip_address": ar.IPAddress,
"method": ar.Method,
"path": ar.Path,
"response_code": ar.ResponseCode,
"api_key": ar.APIKey,
"time_stamp": ar.TimeStamp,
"api_version": ar.APIVersion,
"api_name": ar.APIName,
"api_id": ar.APIID,
"org_id": ar.OrgID,
"oauth_id": ar.OauthID,
"raw_request": ar.RawRequest,
"request_time": ar.RequestTime,
"raw_response": ar.RawResponse,
"ip_address": ar.IPAddress,
"total_latency": ar.Latency.Total,
"upstream_latency": ar.Latency.Upstream,
}
tags := make(map[string]string)
fields := make(map[string]interface{})

Check notice on line 213 in pumps/influx2.go

View check run for this annotation

probelabs / Visor: performance

performance Issue 

A new map `mapping` is created for each analytic record inside the loop. For high-throughput scenarios where the `data` slice can contain many records, this leads to frequent memory allocations and increases pressure on the garbage collector, potentially degrading performance.
Raw output 
To optimize memory usage and reduce GC overhead, consider using a pool of maps (e.g., `sync.Pool`) to reuse map allocations. A map can be acquired from the pool at the start of the loop iteration and released back to the pool at the end after it's been used. This is a common pattern for optimizing hot loops that process batches of data.
var tag string
// Select tags from config
for _, t := range i.dbConf.Tags {
Expand Down
73 changes: 73 additions & 0 deletions pumps/influx2_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
package pumps

import (
"testing"
"time"

"github.com/TykTechnologies/tyk-pump/analytics"
"github.com/stretchr/testify/assert"
)

func TestInflux2PumpMappingIncludesLatency(t *testing.T) {
// Create a sample analytics record
ar := analytics.AnalyticsRecord{
Method: "GET",
Path: "/test/path",
ResponseCode: 200,
APIKey: "test-api-key",
TimeStamp: time.Date(2025, 7, 3, 12, 0, 0, 0, time.UTC),
APIVersion: "v1",
APIName: "Test API",
APIID: "test-api-id",
OrgID: "test-org-id",
OauthID: "test-oauth-id",
RawRequest: "raw-request-data",
RequestTime: 123,
RawResponse: "raw-response-data",
IPAddress: "1.2.3.4",
Latency: analytics.Latency{
Total: 456,
Upstream: 123,
},
}

// Simulate how your Influx2 pump builds the mapping
mapping := map[string]interface{}{
"method": ar.Method,
"path": ar.Path,
"response_code": ar.ResponseCode,
"api_key": ar.APIKey,
"time_stamp": ar.TimeStamp,
"api_version": ar.APIVersion,
"api_name": ar.APIName,
"api_id": ar.APIID,
"org_id": ar.OrgID,
"oauth_id": ar.OauthID,
"raw_request": ar.RawRequest,
"request_time": ar.RequestTime,
"raw_response": ar.RawResponse,

Check warning on line 48 in pumps/influx2_test.go

View check run for this annotation

probelabs / Visor: quality

logic Issue 

The test `TestInflux2PumpMappingIncludesLatency` does not test the production code's behavior. Instead of calling the `WriteData` method of `Influx2Pump`, it duplicates the implementation logic for creating the data map. This makes the test brittle and provides a false sense of security, as changes to the implementation in `influx2.go` will not be caught by this test.
Raw output 
Refactor the test to be a proper unit test. It should instantiate the `Influx2Pump`, mock its dependencies (like the InfluxDB client), call the `WriteData` method, and then assert that the data passed to the mock client is correct. This will ensure the test is verifying the actual behavior of the component, not just a reimplementation of its logic.
"ip_address": ar.IPAddress,

Check warning on line 49 in pumps/influx2_test.go

View check run for this annotation

probelabs / Visor: style

style Issue 

The test `TestInflux2PumpMappingIncludesLatency` duplicates the mapping logic from `pumps/influx2.go` instead of testing the production code directly. This makes the test brittle, as changes in the production mapping logic will not be caught by this test, and the test itself will need to be manually kept in sync.
Raw output 
Refactor the mapping creation in `pumps/influx2.go` into a separate, testable function. This new function can then be called from both the `WriteData` method and this test, ensuring the test validates the actual production logic and improving maintainability.
"total_latency": ar.Latency.Total,
"upstream_latency": ar.Latency.Upstream,
}

// Check the new latency fields exist and have the right values
assert.Equal(t, int64(456), mapping["total_latency"])
assert.Equal(t, int64(123), mapping["upstream_latency"])

// Check some other fields for completeness
assert.Equal(t, "GET", mapping["method"])
assert.Equal(t, "/test/path", mapping["path"])
assert.Equal(t, 200, mapping["response_code"])
assert.Equal(t, "test-api-key", mapping["api_key"])
assert.Equal(t, "v1", mapping["api_version"])
assert.Equal(t, "Test API", mapping["api_name"])
assert.Equal(t, "test-api-id", mapping["api_id"])
assert.Equal(t, "test-org-id", mapping["org_id"])
assert.Equal(t, "test-oauth-id", mapping["oauth_id"])
assert.Equal(t, "raw-request-data", mapping["raw_request"])
assert.Equal(t, int64(123), mapping["request_time"])
assert.Equal(t, "raw-response-data", mapping["raw_response"])
assert.Equal(t, "1.2.3.4", mapping["ip_address"])
assert.Equal(t, time.Date(2025, 7, 3, 12, 0, 0, 0, time.UTC), mapping["time_stamp"])
}
Loading