Prototype option 1 for PDF to HTML -pdf2htmlEX + Nokogiri

[meganhicks]

Description As a developer, I want to prototype a solution using pdf2htmlEX to convert PDFs to HTML and process the output with Nokogiri, so that I can determine if this approach produces structured and accessible HTML consistently.

Details on decision narratives here (including mock letter, and examination of 961 letter content).

Hypothetical implementation here.

Acceptance Criteria

1. The service accepts a PDF file as input.
2. pdf2htmlEX successfully converts the PDF into an HTML format while maintaining layout structure.
3. Nokogiri processes the HTML to:
   Convert headings (h1, h2) based on font size.
   Convert lists (ul, ol, li).
   Structure tables with , , .
   Ensure images have alt text.
   Remove absolute positioning styles for accessibility.
4. The service returns a well-structured HTML output.
5. The output is evaluated for consistency across multiple decision narrative PDFs.
6. This protype is compared to the others and a recommendation is made

New Note

The scope has adjusted for the project to VBMS only generated decision letters. This is what you should use to test the prototype.

[jahenkor-navapbc]

Created vets-api branch here: 4170-pdf2htmlex-nokogiri-prototype

[meganhicks]

Issues with this prototype:

1. Excessive Use of span Tags
   Unnecessary span elements wrapping text
   Can cause screen readers to add pauses or misinterpret content
2. Poor Tagging for Accessibility
   Missing proper heading tags (h1, h2, etc.)
   Overuse of div instead of semantic elements
3. Formatting and Spacing Issues
   Relies on span for spacing instead of CSS
   Text alignment may be inconsistent
4. Potential Reading Order Issues
   Text structure may not follow logical reading order
   Extra tags could break up sentences unnaturally
5. Missing or Incorrect Metadata
   No ARIA roles or landmarks for screen readers
   Lists may not be properly formatted with ul and li

[jahenkor-navapbc]

Tools used in this initial prototype:

Solution housed in vets-api:
1. https://github.com/pdf2htmlEX/pdf2htmlEX (Docker image or tool installed on local machine - potential to leverage existing docker image in vets-api for this)
2. https://github.com/fmalina/PDFtranscript/tree/main (Transcribes pdf2htmlEX output into semantic HTML)

Pros:
Paragraph detection, formatting, and proper headings present in semantic html version.

Cons:
May have push back in regards to installing pdf2htmlEX in vets-api docker image

In order to create a prototype that can be housed in vets-api, I'm looking into a ruby wrapper for pdf2htmlEX and a way to transform the transcribe script from python into ruby

The concerns mentioned below have been addressed with pdftranscript:

1. Excessive Use of span Tags
   Unnecessary span elements wrapping text
   Can cause screen readers to add pauses or misinterpret content

2. Poor Tagging for Accessibility
   Missing proper heading tags (h1, h2, etc.)
   Overuse of div instead of semantic elements

3. Formatting and Spacing Issues
   Relies on span for spacing instead of CSS
   Text alignment may be inconsistent

4. Potential Reading Order Issues
   Text structure may not follow logical reading order
   Extra tags could break up sentences unnaturally

5. Missing or Incorrect Metadata
   No ARIA roles or landmarks for screen readers
   Lists may not be properly formatted with ul and li

Mock decision letter (pdf): fake-decision-letter-0a8fd999-999b-45f0-968b-af8d67af85ae.pdf

Html produced by pdf2htmlEX:

Semantic Html produced by pdftranscript, using the output of pdf2htmlEX:

[jahenkor-navapbc]

Closing this ticket out following sprint review/demo of prototype progress. Will begin working on the follow up prototype ticket.