Self-Hosting on ESP32

This capsule is now self-hosted on an ESP32 development board (ESP32-DevKitC V4) connected to a phone charger and Wi-Fi. When the ESP32 application gets notified that Wi-Fi connection is established, it starts a Gemini server and uses the Duck DNS API to set the address of dimkr-esp32.duckdns.org to the response from ifconfig.me. gemini.dimakrasner.com is now a CNAME record that points to dimkr-esp32.duckdns.org, and I copied my certificate to the ESP32.

https://www.duckdns.org

(I might regret this decision and return to my VPS, if the ESP32 setup turns out to be unstable.)

It didn't take long to implement a Gemini server, but this board definitely has some disadvantages compared to a Pi:

You must choose an ESP-IDF version, because they're API-incompatible with each other. The ESP-IDF also dictates the versions of libraries, because some "open-source" libraries included in ESP-IDF are provided as binary blobs: you can't update these libraries.

mbedtls, the default TLS library, supports TLS 1.3 in ESP-IDF 5.0, which is great. However, the sentence "Mbed TLS supports SSL 3.0 up to TLS 1.3" is misleading, because TLS 1.3 support is limited to the client side.

There's more than one way to do many of the things you'd probably want to do in your application. The examples provide at least 3 different ways to set up a Wi-Fi connection and get notifications about the connection status.

ESP-TLS is a super simple wrapper around mbedtls (or wolfssl), which reduces the amount of boilerplate code you need to write if you want to implement a simple TLS client or server. This wrapper is very high-level and doesn't allow you to tweak internal configuration easily.

The HTTPS Server component is implemented as a bunch of callbacks (new session, TX/RX) passed to the HTTP Server component. I haven't found a good, minimal example of a "raw TLS" server.

The TCP/IP stack has an artificial (?) limitation of CONFIG_LWIP_MAX_SOCKETS (<= 16) sockets. That's enough for many Gemini capsules, but not much.

It looks like O_NONBLOCK and FIONBIO don't work: esp_tls_server_session_create() blocks. To overcome this, I had to create a "task" for every incoming connection, and that's OK becaue I don't have to care about scale: the number of sockets is limited anyway, and I know I have enough RAM for 16 stacks.

SO_RCVTIMEO doesn't work as it should: esp_tls_server_session_create() doesn't fail on timeout. The main "task" of my server, which accepts incoming connections, is responsible for killing slow clients and killing the oldest client to make room for a new one. It unblocks esp_tls_server_session_create() by closing the socket.

idf.py and CMake can be painfully slow, even if ccache is enabled. I have a Windows laptop, which I used for a C# course I had to take when I started my current job: I'm transforming the backend of a product that normally runs on Windows servers into a scalable SaaS offering that runs on Linux with PostgreSql. It seems to me that every file system operation that involves many small files is extremely slow on Windows. Later, I moved my workflow to a much older Linux laptop: thankfully, ESP-IDF has a Linux version, and now it takes <5s to build my application.

I looked at some power consumption figures and I'm not sure if the ESP32 is as power efficient as some people think. Maybe I misunderstood what I read, but it looks like power consumption is very close to that of a Raspberry Pi Zero 2 W, which has a quad-core AArch64 CPU and 512 MB of RAM. It will be much harder to recommend the ESP32 when Pi prices drop.

I have concerns about longevity: unlike a Pi, the ESP32 uses on-board flash. I don't like the idea of flashing firmware every time I want to add a post.

4 MB of flash is not much.

RAM is slow: the first version of my Gemini server used a giant blob of concatenated .gmi files and memmem() to search for a magic marker that marks the beginning of a file. It was extremely slow, and now this blob starts with a small array that specifies the offset of a .gmi file, given the CRC32 of its name. This is much faster than the previous solution, even after I zlib-compressed the individual .gmi files and added a decompression step (using miniz) before the response is sent.

https://github.com/richgel999/miniz

More to come later: I'm still cleaning up my code.