Template formatting: Allow List Syntax in Attention Block

[k-macmillan]

User Story - Business Need

NOTE: Revised 03 FEB 2025 based on 1st attempt to work this ticket. See reverted PR here. Note that this PR transformed the ^ indicator for blockquotes into >, this is not ideal because it is unexpected by clients when used outside of a blockquote.

Services would like to use a list in "personalisation" to be within a code/attention block. The use of lists in this way is not currently supported according to Portal usage, but we do have API support to allow a list passed in the request to become a bulleted list in the email. This feature was not 100% ready for services to use, this is to close the loop and make it Production ready.

• Ticket is understood, and QA has been contacted (if the ticket has a QA label).

User Story(ies)

As a service using VA Notify
I want to use a list to generated a bulleted list within a code block
So that I can simplify templates

Additional Info and Resources

Slack conversation

Example template:

 ^ ##__ Submission details__
 ^
 ^ __Additional forms included__
 ^ ((forms_submitted))
 ^
 ^

forms_submitted and files_submitted should be within a code block but they are being excluded from the code block:

It appears the ^ is not being applied to each bullet point added.

For testing, use template ID, 86820037-1374-4072-9f9f-e74f235170a1, "QA All The Formatting", in the DEV or PERF VANotify service. See that currently, when inserting a list into the personalisation field that is within a block quote, the list is rendered outside of that blockquote. Note that a bulleted list added directly to the blockquote does render the bulleted list within the blockquote directly as shown in the above screenshot, so this must be validated to ensure that it still works.

Acceptance Criteria

• Lists submit inside a code block (denoted with ^) have the entire list inside a code block
• Lists directly added to a blockquote continue to be rendered within the blockquote as expected.
• Update API docs (schema/swagger/postman) is this necessary?
• Notify Sam that this work is done so strike can prioritize their corresponding ticket in a future sprint
• This work is added to the sprint review slide deck (key win bullet point and demo slide)
• Inform this thread in our public channel (November 5th) that this work has been completed

QA Considerations

Permutations of code blocks and lists.

Out of Scope

Updating Portal documentation
Collaboration with Portal is happening in parallel and the documentation will be updated after this is functional in the API.

[mjones-oddball]

@k-macmillan to add an AC regarding the API doc update needed

@kbelikova-oddball Can you find the corresponding Notify Exp ticket for the portal documentation updates? I think Kyle referred to it as advanced usage. Dave would like that ticket linked in the out of scope section so we know where that work is happening.

Additional note from Dave:

In addition we'll want to think about change management/announcement of this functionality to existing customers if this is a feature others need/want beyond the current VA.gov request.

I think above should happen after the portal updates are made so folks have clear instruction on how to use this in the template and API request.

CC @davidconlon

[kalbfled]

This might work without additional changes after #169. The newer version of Mistune handles block quotes with lists the way I expect.

>>> import mistune
>>> print(mistune.html('> this\n>\n>\n> - a\n> - b'))
<blockquote>
    <p>this</p>
    <ul>
        <li>a</li>
        <li>b</li>
    </ul>
</blockquote>

(indentation added)

I contacted Ruben Sun, of the HCD team, to ask if this is still an issue for him. Given our markdown uses ^ to denote a block quote, and given that my preprocessing of input replaces ^ with the standard > before passing the input to Mistune, there's still room for error.

[kalbfled]

I have feedback from the HCD team. Using staging template a4b5df4f-6f29-4b0d-a61e-fb02e1ebdb40, which contains this snippet of markdown . . .

^ ## Submission Details
^ __Form submitted__
^ Disability Claim (VA Form 21-526EZ)
^ 
^ __Additional forms included__
^ ((forms_submitted))
^ 
^ 
^  __Supporting evidence included__
^ For your privacy, we're showing only part of the names of the files you've submitted.
^ ((files_submitted))
^ 
^ 
^ __Date submitted__
^ ((date_submitted))

. . . and this personalization . . .

{
  :email_address=>"sam.stuckey@oddball.io",
  :template_id=>"a4b5df4f-6f29-4b0d-a61e-fb02e1ebdb40",
  :personalisation=> {
    :first_name=>"FAKEMAN_DELAYED",
    :date_submitted=>"November 5, 2024 8:14 p.m. UTC",
    :forms_submitted=> [
      "VA Form 21-4142",
      "VA Form 21-0781",
      "VA Form 21-8940"
    ],
    :files_submitted=> [
      "extXas.pdf",
      "extXas.pdf",
      "extXas.pdf"
    ],
    :date_of_failure=>"November 5, 2024 8:14 p.m. UTC"
  }
}

. . . I expect the output html to have the lists inside the block quote, like this:

<blockquote>
  <h2>Submission Details</h2>
  ...
  <p>For your privacy, we're showing only part of the names of the files you've submitted.</p>
  <ul>
    <li>file 1</li>
    <li>file 2</li>
    ...
  </ul>
  ...
</blockquote>

What actually happens is this:

The underlying html ends the blockquote before the start of the list. The is probably a function of my regex for identifying block quotes.

[kalbfled]

This is taking a little longer than I hoped, but I'm getting there. After tweaking the block quote regex that substitutes > for ^, the problem is that the personalization substitutions are removing the block quote character in front of list items. I have written unit tests that recreate the problem reported by the user.

[kalbfled]

This has been more pernicious than I anticipated. I might have a workable solution, but I have 4 failing unit tests which I will investigate in the morning. If I don't have this ready for review by tomorrow, I'm going to mark it "off track" because QA work remains.