Packaging includes control characters in HTML

[cmahnke]

I try to wrap my head around serving packages using Vite.
This currently wont't work, since the generated HTML includes control characters and parse5, the HTML parser used by Vite doesn't like those.

Trying to fix this using brute force like this won't work either, since the internal unziper doesn't seem to understand the escaped characters.

let htmlStr = new TextDecoder().decode(html);
/*
// These are parts of the Zip header
htmlStr = htmlStr.replace(/PK\x03\x04/g, "PK");
htmlStr = htmlStr.replace(/PK\x01\x02/g, "PK");
htmlStr = htmlStr.replace(/PK\x06\x06/g, "PK");
htmlStr = htmlStr.replace(/PK\x06\x07/g, "PK");
htmlStr = htmlStr.replace(/PK\x07/g, "PK");
htmlStr = htmlStr.replace(/PK\x05\x06/g, "PK");
// This is the ESC char for ansi codes
htmlStr = htmlStr.replace(/\x1B/g, "");
htmlStr = htmlStr.replace(/\x1F/g, "");
// This is a DOS line ending
htmlStr = htmlStr.replace(/\x1A/g, "");

Is there a known way to work around this?
Can control characters in HTML can be avoided in the future in general?

[cmahnke]

If anyone else has this issue, one can work around it with this stupid trick:

function parseHtmlAndExternalizeScripts(htmlString: string, outputDirectory: string): string {
  try {
    if (!fs.existsSync(outputDirectory)) {
      console.log(`Creating output directory for scripts: "${outputDirectory}"`);
      fs.mkdirSync(outputDirectory, { recursive: true });
    }
  } catch (error) {
    console.error(`Error: Could not create output directory "${outputDirectory}".`, error);
    throw error;
  }

  const root = cheerio.load(htmlString);

  root('script').each((index, element) => {
    const scriptTag = root(element);
    const scriptContent = scriptTag.html();
    const existingSrc = scriptTag.attr('src');
    const hasDataAttribute = scriptTag.attr('data') !== undefined;

    if (!existingSrc && scriptContent && scriptContent.trim().length > 0 && !hasDataAttribute) {
      const uniqueId = crypto.randomUUID();
      const fileName = `script-${uniqueId}.js`;
      const filePath = path.join(outputDirectory, fileName);

      try {
        fs.writeFileSync(filePath, scriptContent.trim());
        console.log(`Extracted and saved script to: "${filePath}"`);

        scriptTag.empty();
        scriptTag.attr('src', fileName);
        scriptTag.removeAttr('type');
      } catch (error) {
        console.error(`Error: Could not write script file "${filePath}".`, error);
      }
    }
  });

  return root.html();
}

Just pass the content of the HTML as string (conversion from UInt8 Array above) an the directory where the HTML file will be saved. Make sure to save the returned HTML string as the needed file (like index.html).

[GarboMuffin]

if we can configure the JS minifier to not use those characters then I'm not opposed to making that change. it probably only costs like 10 bytes in the end. haven't looked at this

though honestly an HTML parser not being able to parse a file that browsers handle fine feels like a bug in that HTML parser

[cmahnke]

I tend to disagree - it's a just a strict parser, not nessecarilly a buggy one. But maybe it's also a worthwhile approach to check at the parser repository first.

[GarboMuffin]

OK I thought about this a bit more. I remembered that the editor uses an XML parser that also chokes on those characters so it probably is just a spec violation that no browser enforces

[cmahnke]

vitejs/vite#20482 has been closed - this should work with the next version. I'm not sure if this can be considered to be closed. But if a XML parser reports this as an issue as well, it might stay open.