-- Leo's gemini proxy

-- Connecting to perso.pw:1965...

-- Connected

-- Sending request

-- Meta line: 20 text/gemini;

Reduce httpd web server bandwidth usage by serving compressed files


Author: Solène

Date: 22 April 2022

Tags: openbsd selfhosting nocloud


Comment on Mastodon


Introduction


When reaching a website, most web browsers will send a header (some metadata about the requestion) informing the web server that you supported compressed content. In OpenBSD 7.1, the httpd web server received a new feature allowing it to serves a pre-compressed file of a requested file if the web browser supports compression. The benefits are a bandwidth usage reduced by 2x to 10x depending on the file content, this is particularly interesting for people who self-host and for high traffic websites.


Configuration


In your httpd.conf, in a server block add the "gzip-static" keyword, save the file and reload the httpd service.


A simple server block would look like this:


server "perso.pw" {
        root "/htdocs/solene"
        listen on * port 80
        gzip-static
}

Creating the files


In addition to this change, I added a new flag to the gzip command to easily compress files while keeping the original files. Run "gzip -k" on the files you want to serve compressed when the clients support the feature.


It's best to compress text files, such as HTML, JS or CSS for the most commons. Compressing binary files like archives, pictures, audio or videos files won't provide any benefit.


How does it work?


When the client connects to the httpd server requesting "foobar.html", if gzip-static is used for this location/server, httpd will look for a file named "foobar.html.gz" that is not older than "foobar.html". When found, "foobar.html.gz" is transparently transferred to the client requesting "foobar.html".


Take care to regenerate the gz files when you update the original files, remember that the gz files must be newer to be used.


Conclusion


This is for me a major milestone for using httpd in self-hosting and with static websites. We battle tested this change with the webzine server often hitting big news websites leading to many people visiting the website in a short time span, this drastically reduced the bandwidth usage of the server, allowing it to support more clients per second.

-- Response ended

-- Page fetched on Mon May 6 07:42:37 2024