Build Workflow Spends Over 20 Minutes on File Compression

[SableRaf]

The p5.js website build workflow takes over 20 minutes, with significant time spent on the compression step (labelled AstroCompress processing in the logs) for html, images, and other files. This duration impacts the developer experience and considerably slows down iterations during development.

See for example: GitHub Action Log

Relevant issues in the repository for Astro's compression library:

1. [performance] Compress in parallel #54 (Help Wanted)
2. Cache #49 (Help Wanted)

Suggestions

• Consider contributing to the upstream fixes in the Astro compression library.

[limzykenneth]

#615 will fix this to a certain extend, I still need to fix some bugs though.

However, npm run build is not meant to be run in the dev environment. npm run dev should suffice for development preview and provides much better dev experience. If there are any notable difference between dev preview and full build it should be considered a bug with the build/dev system and fixed accordingly without need to use npm run build in local environment.

Only CI should run npm run build which will still be slow until Astro can improve parallelization on their end.

[wlami]

Hi @SableRaf and the p5.js-website team,

I noticed your discussion regarding extended build times, particularly during the AstroCompress processing phase. I recently developed a tool called mysong-compress to address similar challenges with my Astro website. This tool offers comparable compression capabilities and includes a caching mechanism designed to significantly reduce build times for larger projects. In my case the build time went from over 10 minutes down to 1 minute.

I invite you to try out mysong-compress in your build process. If you do, I would greatly appreciate it if you could share your build time measurements and output folder sizes for the following scenarios:

• Before Switch: Your current build time and output folder size using AstroCompress.
• mysong-compress First Run: Build time and output folder size during the initial run without an existing cache.
• mysong-compress Subsequent Runs: Build times and output folder sizes for subsequent runs with the cache in place.

Please post your findings in this issue. Your feedback would be invaluable in assessing the effectiveness of mysong-compress and identifying areas for further improvement.

Thank you for considering this, and I look forward to any insights you might provide.

In case you're interested, I can also offer to create a PR integrating mysong-compress for you to have a look.

Best regards,
wlami

[limzykenneth]

@wlami Thanks for the ping. We have a mostly complete solution in #615 that also recreates the compression that astro-compress does. It is achieving the same and a bit better compression than astro-compress and our build time is halved from 20 minutes to 10 minutes.

I haven't tried out your implementation yet but I had a look at the repo and I have a couple suggestion based on what I found while working on our solution.

• A significant part of what makes our build slow during compression is to minify HTML, we have a lot of HTML files. I found that html-minifier-terser is quite slow, especially when it comes to parsing and that it is not very tolerant of failures. @swc/html however is able to achieve better compression at a fraction of the time. (html-minifier-terser takes a little less than 10 minutes, while @swc/html is done in 1 minute)
• Similarly I would assume terser for minification can also be sped up with a more modern minification tool, although that being said, our solution don't minify JS because that is already done by Astro well enough and double processing JS don't give much benefit I think.
• The cache idea is interesting although the plan for us is to pretty much only build in CI and not locally so I guess it may not apply as much. Instead of a custom hash based cache, perhaps using a content addressable cache would be a good way to go instead?

[limzykenneth]

Addressed by #694. For further build speed improvements it likely need to come from Astro, I can optimize the compression a little bit more but it likely won't provide further speed improvements. I imagine once Vite managed to switch to Rolldown, Astro which uses Vite underneath should see good improvements to build and dev speed.

I'll close this for now.