NDH-565: Implementing collecting traces and sending to Datadog

[dmtrscms]

module-name: Updating to add in Datadog integration

Jira Ticket #NDH-565

Problem

We need to start sending trace information from API usage to Datadog

Solution

Updated files to install the python module for Datadog, updated the docker compose file to install the Datadog agent, and provided the necessary configuration for trace information to be sent to Datadog.

Result

API related trace information is sent to Datadog.

Some important notes regarding the summary line:

1. Updating Dockerfile to run the Datadog trace command
2. Installing the Datadog module
3. Creating a container for the Datadog agent
4. Creating environment variables and labels to configure access and parsing of trace information.

Test Plan

Work with @blaine to adapt this to use the AWS container definition. These changes only work locally.

[rmillergv]

HI Demetrius,

two requests, how do other people get the Datadog API key? Can we put that up in AWS secrets please (and put instructions here how to get the API key)

and,

it doesn't have to be exact, but please fill in at least some of thiis PR template

Thanks!

module-name: One line description of your change (less than 72 characters)
Jira Ticket
Problem
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of being the motivation for your change.

Solution
Describe the modifications you've done.

Result
What will change as a result of your pull request? Note that sometimes this
section is unnecessary because it is self-explanatory based on the solution.

Some important notes regarding the summary line:

Describe what was done; not the result
Use the active voice
Use the present tense
Capitalize properly
Do not end in a period — this is a title/subject
Prefix the subject with its scope
Test Plan
(Write your test plan here. If you changed any code, please provide us with
clear instructions on how you verified your changes work.)

[dmtrscms]

My changes didn't save to template! Will update, again. @price, Blaine ***@***.***> set up the API key in AWS. I'll try and find his directions. I just know how to get. 😊 Get Outlook for Mac <https://aka.ms/GetOutlookForMac> From: rmillergv ***@***.***> Date: Wednesday, January 28, 2026 at 8:59 AM To: CMS-Enterprise/npd ***@***.***> Cc: Kellum, Demetrius (CMS/DSAC) ***@***.***>, Assign ***@***.***> Subject: Re: [CMS-Enterprise/npd] NDH-565: Implementing collecting traces and sending to Datadog (PR #368) @rmillergv requested changes on this pull request. HI Demetrius, two requests, how do other people get the Datadog API key? Can we put that up in AWS secrets please (and put instructions here how to get the API key) and, it doesn't have to be exact, but please fill in at least some of thiis PR template Thanks! module-name: One line description of your change (less than 72 characters) Jira Ticket Problem Explain the context and why you're making that change. What is the problem you're trying to solve? In some cases there is not a problem and this can be thought of being the motivation for your change. Solution Describe the modifications you've done. Result What will change as a result of your pull request? Note that sometimes this section is unnecessary because it is self-explanatory based on the solution. Some important notes regarding the summary line: Describe what was done; not the result Use the active voice Use the present tense Capitalize properly Do not end in a period — this is a title/subject Prefix the subject with its scope Test Plan (Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work.) — Reply to this email directly, view it on GitHub<#368 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BYSC4BKQLWU4H3ZVFJVFBKT4JC6EJAVCNFSM6AAAAACTDHAY3GVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTOMJWHA3DGNZYGM>. You are receiving this because you were assigned.

[wbprice]

@dmtrscms how did you validate that this is working locally? I'm mostly seeing a bunch of certificate errors when trying to test this.

[dmtrscms]

@dmtrscms how did you validate that this is working locally? I'm mostly seeing a bunch of certificate errors when trying to test this.

I spun up a local instance of the NPD app, made the modifications, hit the API, then looked at the Datadog dashboard.

I saw the traces and even saw which SQL statements got executed.

[rmillergv]

not run into a dev shop yet that did not love datadog.
Impact is usually negligible on compute time, and in our existing environment is not likely to be noticed at all.

[rmillergv]

key is in AWS secrets: arn:aws:secretsmanager:us-east-1:575012135727:secret:npd-east-dev-datadog-api-key20260122173510575000000001-aT7ojn

this is the key for dev (nonprod).

@wbprice