Skip to content

Commit 58f57ba

Browse files
leoromanovskyleoromanovsky
authored
test(ffe): add agentless serverless exposure contract (#7477)
1 parent fe915fc commit 58f57ba

28 files changed

Lines changed: 542 additions & 109 deletions

‎.github/workflows/run-end-to-end.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -323,6 +323,12 @@ jobs:
323323
- name: Run FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS scenario
324324
if: steps.build.outcome == 'success' && !cancelled() && contains(inputs.scenarios, '"FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS"')
325325
run: ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS
326+
- name: Run FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_DIRECT scenario
327+
if: steps.build.outcome == 'success' && !cancelled() && contains(inputs.scenarios, '"FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_DIRECT"')
328+
run: ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_DIRECT
329+
- name: Run FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_SERVERLESS scenario
330+
if: steps.build.outcome == 'success' && !cancelled() && contains(inputs.scenarios, '"FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_SERVERLESS"')
331+
run: ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_SERVERLESS
326332
- name: Run AGENT_SUPPORTING_SPAN_EVENTS scenario
327333
if: steps.build.outcome == 'success' && !cancelled() && contains(inputs.scenarios, '"AGENT_SUPPORTING_SPAN_EVENTS"')
328334
run: ./run.sh AGENT_SUPPORTING_SPAN_EVENTS

‎manifests/cpp_httpd.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,8 @@ manifest:
5252
tests/debugger/test_debugger_probe_snapshot.py::Test_Debugger_Line_Probe_Snaphots::test_process_tags_snapshot_svc: missing_feature (Not yet implemented)
5353
tests/ffe/test_agentless_configuration.py: missing_feature (C++ is not in scope for v1 agentless server SDK coverage; FFL-2731 tracks the system-tests contract)
5454
tests/ffe/test_dynamic_evaluation.py: missing_feature
55-
tests/ffe/test_exposures.py: missing_feature
55+
tests/ffe/test_exposure_egress.py: missing_feature (Not yet implemented)
56+
tests/ffe/test_exposures_datadog_agent.py: missing_feature
5657
tests/ffe/test_flag_eval_evp.py: missing_feature (FFL-2446)
5758
tests/ffe/test_flag_eval_metrics.py: missing_feature
5859
tests/integration_frameworks/llm/anthropic/test_anthropic_llmobs.py::TestAnthropicLlmObsMessages::test_create_error: bug (MLOB-1234)

‎manifests/cpp_nginx.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -285,7 +285,8 @@ manifest:
285285
tests/docker_ssi/test_docker_ssi_appsec.py::TestDockerSSIAppsecFeatures::test_telemetry_source_ssi: missing_feature
286286
tests/ffe/test_agentless_configuration.py: missing_feature (C++ is not in scope for v1 agentless server SDK coverage; FFL-2731 tracks the system-tests contract)
287287
tests/ffe/test_dynamic_evaluation.py: missing_feature
288-
tests/ffe/test_exposures.py: missing_feature
288+
tests/ffe/test_exposure_egress.py: missing_feature (Not yet implemented)
289+
tests/ffe/test_exposures_datadog_agent.py: missing_feature
289290
tests/ffe/test_flag_eval_evp.py: missing_feature (FFL-2446)
290291
tests/ffe/test_flag_eval_metrics.py: missing_feature
291292
tests/integration_frameworks/llm/anthropic/test_anthropic_llmobs.py::TestAnthropicLlmObsMessages::test_create_error: bug (MLOB-1234)

‎manifests/dotnet.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -748,7 +748,10 @@ manifest:
748748
tests/ffe/test_dynamic_evaluation.py: v3.36.0
749749
tests/ffe/test_dynamic_evaluation.py::Test_FFE_Flag_Parse_Error_Isolation: bug (FFL-2184)
750750
tests/ffe/test_dynamic_evaluation.py::Test_FFE_Unknown_Operator_Tolerance: bug (FFL-2184)
751-
tests/ffe/test_exposures.py: v3.36.0
751+
tests/ffe/test_exposure_egress.py: v3.36.0
752+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Direct: missing_feature (Not yet implemented)
753+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Sidecar: missing_feature (Not yet implemented)
754+
tests/ffe/test_exposures_datadog_agent.py: v3.36.0
752755
tests/ffe/test_flag_eval_evp.py: missing_feature (FFL-2446)
753756
tests/ffe/test_flag_eval_metrics.py: v3.44.0
754757
tests/integration_frameworks/llm/anthropic/test_anthropic_llmobs.py::TestAnthropicLlmObsMessages::test_create_error: bug (MLOB-1234)

‎manifests/golang.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1171,7 +1171,10 @@ manifest:
11711171
tests/ffe/test_dynamic_evaluation.py::Test_FFE_RC_Down_From_Start: v2.4.0
11721172
tests/ffe/test_dynamic_evaluation.py::Test_FFE_RC_Unavailable: v2.4.0
11731173
tests/ffe/test_dynamic_evaluation.py::Test_FFE_Unknown_Operator_Tolerance::test_unknown_operator_errors: bug (FFL-2182)
1174-
tests/ffe/test_exposures.py: v2.6.0-dev # Easy win for chi, echo, gin, net-http, net-http-orchestrion, uds-echo and version 2.5.0
1174+
tests/ffe/test_exposure_egress.py: v2.6.0-dev # Easy win for chi, echo, gin, net-http, net-http-orchestrion, uds-echo and version 2.5.0
1175+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Direct: missing_feature (Not yet implemented)
1176+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Sidecar: missing_feature (Not yet implemented)
1177+
tests/ffe/test_exposures_datadog_agent.py: v2.6.0-dev # Easy win for chi, echo, gin, net-http, net-http-orchestrion, uds-echo and version 2.5.0
11751178
tests/ffe/test_flag_eval_evp.py: v2.10.0-dev
11761179
tests/ffe/test_flag_eval_evp.py::Test_FFE_EVP_Flagevaluation_Degradation::test_ffe_evp_flagevaluation_degradation: flaky (FFL-2676)
11771180
tests/ffe/test_flag_eval_evp.py::Test_FFE_EVP_Flagevaluation_ObserveFullData_Absent_Hashed: missing_feature (FFL-2784)

‎manifests/java.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3344,11 +3344,17 @@ manifest:
33443344
weblog: [spring-boot]
33453345
- declaration: bug (FFL-2184)
33463346
excluded_weblog: [spring-boot]
3347-
tests/ffe/test_exposures.py:
3347+
tests/ffe/test_exposure_egress.py:
33483348
- weblog_declaration:
33493349
"*": irrelevant
33503350
spring-boot: v1.56.0
3351-
tests/ffe/test_exposures.py::Test_FFE_EXP_5_Missing_Targeting_Key: bug (FFL-1729)
3351+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Direct: missing_feature (Not yet implemented)
3352+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Sidecar: missing_feature (Not yet implemented)
3353+
tests/ffe/test_exposures_datadog_agent.py:
3354+
- weblog_declaration:
3355+
"*": irrelevant
3356+
spring-boot: v1.56.0
3357+
tests/ffe/test_exposures_datadog_agent.py::Test_FFE_EXP_5_Missing_Targeting_Key: bug (FFL-1729)
33523358
tests/ffe/test_flag_eval_evp.py: missing_feature (FFL-2446)
33533359
tests/ffe/test_flag_eval_metrics.py:
33543360
- weblog_declaration:

‎manifests/nodejs.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1789,11 +1789,17 @@ manifest:
17891789
"*": incomplete_test_app
17901790
express4: *ref_5_77_0
17911791
tests/ffe/test_dynamic_evaluation.py::Test_FFE_Unknown_Operator_Tolerance::test_unknown_operator_errors: bug (FFL-2182)
1792-
tests/ffe/test_exposures.py:
1792+
tests/ffe/test_exposure_egress.py:
17931793
- weblog_declaration:
17941794
"*": incomplete_test_app
17951795
express4: *ref_5_77_0
1796-
tests/ffe/test_exposures.py::Test_FFE_EXP_5_Missing_Targeting_Key: bug (FFL-1730)
1796+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Direct: missing_feature (dd-trace-js#9527)
1797+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Sidecar: missing_feature (dd-trace-js#9527)
1798+
tests/ffe/test_exposures_datadog_agent.py:
1799+
- weblog_declaration:
1800+
"*": incomplete_test_app
1801+
express4: *ref_5_77_0
1802+
tests/ffe/test_exposures_datadog_agent.py::Test_FFE_EXP_5_Missing_Targeting_Key: bug (FFL-1730)
17971803
tests/ffe/test_flag_eval_evp.py: missing_feature (FFL-2446)
17981804
tests/ffe/test_flag_eval_metrics.py:
17991805
- weblog_declaration:

‎manifests/php.yml‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1228,12 +1228,19 @@ manifest:
12281228
"*": v1.21.0-dev
12291229
laravel11x: incomplete_test_app
12301230
symfony7x: incomplete_test_app
1231-
tests/ffe/test_exposures.py:
1231+
tests/ffe/test_exposure_egress.py:
12321232
- weblog_declaration:
12331233
"*": v1.21.0-dev
12341234
laravel11x: incomplete_test_app
12351235
symfony7x: incomplete_test_app
1236-
tests/ffe/test_exposures.py::Test_FFE_Exposure_Events::test_ffe_multiple_remote_config_files: flaky (FFL-2676)
1236+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Direct: missing_feature (Not yet implemented)
1237+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Sidecar: missing_feature (Not yet implemented)
1238+
tests/ffe/test_exposures_datadog_agent.py:
1239+
- weblog_declaration:
1240+
"*": v1.21.0-dev
1241+
laravel11x: incomplete_test_app
1242+
symfony7x: incomplete_test_app
1243+
tests/ffe/test_exposures_datadog_agent.py::Test_FFE_Exposure_Events::test_ffe_multiple_remote_config_files: flaky (FFL-2676)
12371244
tests/ffe/test_flag_eval_evp.py: missing_feature (FFL-2446)
12381245
tests/ffe/test_flag_eval_evp.py::Test_FFE_EVP_Flagevaluation_Burst_Aggregation: bug (FFL-2676)
12391246
tests/ffe/test_flag_eval_evp.py::Test_FFE_EVP_Flagevaluation_Degradation: bug (FFL-2676)

‎manifests/python.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1441,7 +1441,10 @@ manifest:
14411441
tests/ffe/test_agentless_configuration.py: missing_feature (FFL-2699 tracks Python agentless configuration-source implementation; FFL-2731 tracks the system-tests contract)
14421442
tests/ffe/test_dynamic_evaluation.py::Test_FFE_RC_Down_From_Start: v4.0.0
14431443
tests/ffe/test_dynamic_evaluation.py::Test_FFE_RC_Unavailable: flaky (FFL-1622)
1444-
tests/ffe/test_exposures.py: v4.2.0-dev
1444+
tests/ffe/test_exposure_egress.py: v4.2.0-dev
1445+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Direct: missing_feature (Not yet implemented)
1446+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Sidecar: missing_feature (Not yet implemented)
1447+
tests/ffe/test_exposures_datadog_agent.py: v4.2.0-dev
14451448
tests/ffe/test_flag_eval_evp.py: missing_feature (FFL-2446)
14461449
tests/ffe/test_flag_eval_metrics.py: v4.7.0
14471450
tests/ffe/test_flag_eval_metrics.py::Test_FFE_Eval_Nested_Attributes_Ignored: irrelevant (FFL-1980)

‎manifests/ruby.yml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2022,7 +2022,13 @@ manifest:
20222022
- weblog_declaration:
20232023
"*": irrelevant
20242024
rails72: v2.23.0-dev
2025-
tests/ffe/test_exposures.py:
2025+
tests/ffe/test_exposure_egress.py:
2026+
- weblog_declaration:
2027+
"*": irrelevant
2028+
rails72: v2.23.0-dev
2029+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Direct: missing_feature (Not yet implemented)
2030+
tests/ffe/test_exposure_egress.py::Test_FFE_Exposure_Egress_Agentless_Sidecar: missing_feature (Not yet implemented)
2031+
tests/ffe/test_exposures_datadog_agent.py:
20262032
- weblog_declaration:
20272033
"*": irrelevant
20282034
rails72: v2.23.0-dev

0 commit comments

Comments
 (0)