ACE hangs or times out when processing EPUBs with very large package.opf manifest and spine

[bharathydasanm]

I have encountered an issue when processing an EPUB file using ACE where the tool hangs or runs indefinitely due to the size of the package.opf. The file contains:

• Over 20,000 items in the <manifest> (/package/manifest/item)
• Over 1,600 <itemref> entries in the (/package/spine/itemref)

While analyzing the code, I noticed that the issue seems to stem from the parseData function in epub-parse.js (specifically EpubParser.prototype.parseData). It appears that the time taken to process each <itemref> in the <spine> increases significantly, eventually leading to timeouts or the validator running indefinitely.

This is blocking us from validating large-scale EPUB content using ACE.

Request:
Is there any known workaround or possible fix for this issue? Could ACE be improved to handle large EPUB files?

[danielweck]

Hello, would you be able to share (privately) such a large EPUB publication?
It would really help identify the performance bottleneck(s) and hopefully eliminate the source of crash(es).

Basically, DAISY Ace orchestrates the analysis of each (X)HTML document using a fixed-size pool of web browser / webview instances, each of which run the Deque Axe accessibility evaluation engine. DAISY Ace then consolidates the individual results into its final report, which of course also includes the analysis of the package OPF itself, as well as other EPUB-specific constructs like the navigation document / NCX.

There are multiple areas where Ace could potentially allow memory or processor overload, so I would like to try your EPUB at my end to see if I can reproduce issues. Thank you!

[bharathydasanm]

Hi @danielweck ,

Please download the EPUB file from the link below. I have scrambled the contents and images in the file for sharing here ace_test_20250522.epub

While testing with ACE, I noticed that after removing the image references in /package/manifest/item[@media-type='image/png'] (a total of 25,000+ items), the validation completed much faster.

Sample image item
<item id="a0009472cbb830c0aaeb7ff0d51cd9a4" href="images/x107b8420390308f91fd84e8c58fb7e3.png" media-type="image/png"></item>

[danielweck]

Thank you very much for taking the time to scramble text and images. A trusted work colleague named Mickael Menu will shortly request access to the Google Drive file, could you please allow him? He will run some tests in Readium (as I did, in Thorium).

[bharathydasanm]

Yes, provided the access to Mickael Menu

[danielweck]

A possible solution / workaround would be to expose a new command line argument and/or configurable option to disable image processing, more specifically the step that copies resources into a folder exported alongside the JSON and HTML report.

[bharathydasanm]

Hi @danielweck, Hope you're doing well. I wanted to check if there’s any update on this issue, or if I can support in any way to help move it ahead.

[bharathydasanm]

Hi @danielweck,

I investigated the issue and found that a small optimization in epub-parser.js resolves the problem effectively. I applied the fix locally and observed that it not only works correctly but also improves performance. I just wanted to share this, as it might be helpful.

The root of the issue appears to be with DOMParser which causes timeout problems when processing large spine (.opf) documents.

Specifically, the performance bottleneck is in the XPath queries used at lines L295 and L297 these methods are inefficient on large files and slow down data extraction.

To address this, I replaced the XPath-based lookups with a node mapping approach, which greatly improved parsing speed and reliability during data collection. Here's a quick summary of the approach I used.

epub-parser.js

const manifestMap = buildManifestMap(doc, select);
const spineItemIdrefs = getSpineIdrefs(doc);

spineItemIdrefs.forEach(idref => {

    const manifestItem = manifestMap.get(idref);
      ....
      ...
      ...
 });

Mapping function:

function getSpineIdrefs(doc) {
  const spine = doc.getElementsByTagName('spine')[0];
  if (!spine) return [];
  const itemrefs = spine.getElementsByTagName('itemref');
  console.log(itemrefs.length)
  return Array.from(itemrefs).map(itemref => itemref.getAttribute('idref'));
}


function buildManifestMap(doc, select) {
  const items = select('/opf:package/opf:manifest/opf:item', doc);
  const manifestMap = new Map();
  for (const item of items) {
    manifestMap.set(item.getAttribute('id'), item);
  }

  return manifestMap;
}