[0781 PS] Datadog monitors and dashboards

[lisacapaccioli]

Starting with the date of the launch

Rollout of the new form
In order to support gradually releasing the new 0781 mental health form to a small percentage of users at first, then incrementally increasing the user base over time, we need to monitor performance and identify potential issues.

• Number of claims in progress with new 0781 mental health data/form
• Number of successful submissions with new 0781 mental health data/form
• Number of failed submissions with new 0781 mental health data/form

Primary vs. backup path
In order to identify if the new 0781 mental health form tends to have more submission fails, we want to compare the numbers that go down primary vs. backup path.

• Number of submissions with new 0781 mental health data/form down the primary path
• Number of submissions with new 0781 mental health data/form down the secondary/backup path

Compare to old form
We'd like to compare how many claims and/or Veterans are using the old 0781/a online form vs. the new 0781 form.

• Number of successful submissions with new 0781 mental health data/form
• Number of successful submissions with old 0781/a PTSD data/form
• Number of claims in progress with new 0781 mental health data/form
• Number of claims in progress with old 0781/a PTSD data/form

[freeheeling]

Number of claims in progress with new 0781 mental health data/form

• Is the intent to track each instance a user opts in or uploads a form? Can this be accomplished using Datadog?
• Or is this simply a query that is run at launch? If so, if rollout is incremental, when should this value be determined?

Number of successful submissions with new 0781 mental health data/form
Number of failed submissions with new 0781 mental health data/form

• What is being tracked here if we're also going to compare the "Number of submissions with new 0781 mental health data/form down the primary/backup path"? I'm confused by the "successful"/"failed" differentiator.

In order to identify if the new 0781 mental health form tends to have more submission fails, we want to compare the numbers that go down primary vs. backup path.

• The success rate of Form 526 "should not" be impacted by the presence of 0781 data; form 526 transmission is independent of any ancillary form data. Neither would we necessarily be able to determine that new 0781 data were responsible for 526 failing transmission. We could track and report on the incidence of complete 0781 transmission failure with regards to the form version. However, given that we are uploading a PDF, and not transmitting data, I would suspect a processing issue to happen before arriving at the upload stage.

We'd like to compare how many claims and/or Veterans are using the old 0781/a online form vs. the new 0781 form.

• I'm not clear on how this objective will be informative. Those users that have an IPF at the time the feature is enabled globally will always either already have initiated or be conditionally routed through the legacy 0781 flow.