Support gzip compression in `zola serve`

[ajyoon]

While building sites I like to keep an eye on the transfer sizes of pages. This helps me keep sizes under control as I develop. However, because zola serve doesn't use any compression, these metrics in local development do not reflect production. This is especially true with text-based assets like HTML, JS, and CSS, which often compress very well.

If zola serve supported automatic gzip encryption, I would more easily be able to optimize for size where needed.

This could be done a few ways:

• Just enable it unconditionally in zola serve - I can't think of a use-case where this would be problematic, but I could be wrong
• Enable it by default, allowing an opt-out flag like zola serve --no-compression
• Implement it behind an opt-in flag like zola serve --gzip

Based on a glance through https://github.com/getzola/zola/blob/master/src/cmd/serve.rs any of therse approaches seem fairly straightforward to implement and I'd be happy to take a shot at it.

[Keats]

How do you compare what zola serve would use and what you would use in the end? Do you control the compression level?

[ajyoon]

For my use-case, I don't think an exact match of compression settings between local and production is necessary. I mostly just want a close-enough approximation of payload sizes given any production-realistic compression. For my purposes, actually, it wouldn't even matter if zola serve used brotli instead of gzip. (In that sense, I see this issue relates to #1520).

A low-config or no-config approach to this would also help prevent bloat, as I understand zola serve is meant to be fairly minimalist.

[Keats]

I agree it would be nice to have compression done automatically on the output and on zola serve. As you say it would probably be a no config, just compression = none | gzip | brotli.

[ajyoon]

I understand #1520 is concerned with built output as well as served results, but personally I think compressing build artifacts requires more nuance. For instance, would it transform filenames? Would it output both compressed and uncompressed versions? Would it apply to compression-friendly static assets like fonts? Would it use maximum compression ratio efficiency by default, as opposed to servers which often lean toward speed for on-the-fly compression? In my opinion, there are too many factors here to justify including it in a static site generator when it could be implemented by a fairly simple post-processing find command, especially considering that most static site hosting services already perform compression transparently.

I'd be happy to take a crack at implementing zola serve support for compression, but probably not file output as-is.

[Keats]

It's mostly for symmetry. If zola serve can do gzip/broli compression, it makes that zola build can do it as well. As you mention you might not use it yourself but it would be kind of weird to have one but not the other I think. Although there's precedent of the oppposite with minify_html

[mwcz]

I would find it pretty surprising if a tool like Zola compressed build output. That's something that would typically be done by a webserver. There are (rare in my experience) exceptions like nginx's gzip_static which actually wants pre-compressed files, but even so the compression should (I feel) be done by a deployment script which calls gzip or brotli, not by a static site generator. It would be really awkward for anyone who A) wanted zola serve to compress, and B) was deploying to a typical webserver which wants uncompressed files.

If compression could be limited to zola serve, then the configuration option becomes unnecessary. zola serve could look at the accept-encoding request header, and if it contains gzip then gzip the responses. Same for br/brotli, deflate. Whichever compressions zola serve should support. Just gzip would be fine with me.

That would make it a non-breaking change to the default beavior. It sounds like it wouldn't be possible to turn it off, but that can be done in the browser instead, since the browser sends the accept-encoding header. Chrome devtools even has a convenient option for configuring that header.

[Keats]

That's something that would typically be done by a webserver

In the case of static output you might not have a webserver that you can configure. In practice most of the time your platform can do it for you but it can still be nice to have everything ready to serve

[mwcz]

That's something that would typically be done by a webserver

In the case of static output you might not have a webserver that you can configure. In practice most of the time your platform can do it for you but it can still be nice to have everything ready to serve

Exactly, "ready to serve" is a good goal to have! That's exactly why I think zola build shouldn't pre-compress assets. If it emits compressed assets, they are not ready to serve (unless you happen to be using nginx and happen to have a specific non-default configuration enabled). I'm not aware of any other webservers or hosts that can serve pre-compressed assets, but I know that most don't.

Ie, if the build emitted index.html.gz, most web servers and hosts wouldn't serve that when http://site.com/ is requested.

[Keats]

It would be an opt-in option, not the default. I'm not feeling too strongly about that feature though.

[Jieiku]

I actually do this for my own site like this:

zola build
find ~/.dev/abridge/public -type f -regextype posix-extended -regex '.*\.(htm|html|css|js|xml|xsl|txt|woff|woff2|svg|otf|eot|ttf)' -exec gzip --best -k -f {} \+ -exec brotli --best -f {} \;
rsync -zvrh ~/.dev/abridge/public/ web:/var/www/abridge

(nginx serves these directly as mentioned by @mwcz )

sudo nano /etc/nginx/nginx.conf

gzip on;
gzip_vary on;
gzip_proxied expired no-cache no-store private auth;
#gzip_proxied any;
gzip_comp_level 9;
gzip_buffers 64 16k;
#gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml application/xhtml+xml application/x-javascript application/x-font-ttf application/vnd.ms-fontobject font/opentype font/ttf font/eot font/otf;
#gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;