-- Leo's gemini proxy
-- Connecting to mediocregopher.com:1965...
-- Connected
-- Sending request
-- Meta line: 20 text/gemini; charset=utf-8
For the past six thousand years I have been the maintainer of a medium-popularity driver for the Redis database.
But this post isn't going to be about Redis, really, or even about Go. It's going to be about Redis's serialization protocol: RESP.
Side note on the versioning: the RESP protocol is currently on its 3rd version, but the 2nd version is still widely used and is the default protocol that the Redis database will use unless you specify otherwise. Version 3 is more of an extension to 2, there's only one mildly incompatible change.
Why do I want to talk about Redis's serialization protocol? Well... cause it's actually a pretty good, general purpose serialization protocol, both for the transfer of data over the wire and storage at rest. There is nothing Redis-specific in its design, and I've taken to using it in place of JSON in some of my personal projects.
If you're interested in the protocol itself and what it looks like I recommend looking through the spec, linked above. It's quite easy to understand. If you've used Redis at all then you're probably already somewhat familiar with the types and datastructures involved, as they are referenced directly in the documentation for each command and probably by your redis driver itself (unless it supports generic marshaling/unmarshaling, like mine ;) ).
At a high-level RESP has the following properties:
Primitive datatypes include strings, integers, floats, booleans, and big numbers.
Supports binary strings without the need for escaping.
Supports compound data structures like arrays, maps, and sets.
Supports streaming (i.e. unknown length) binary strings and data structures.
Human-readable, which is very helpful for debugging.
Dead-simple to implement, which would be a huge plus except...
...basically every language is going to have a high-quality, highly-optimized Redis driver which has already implemented the protocol.
And you know that RESP3 is going to be speedy, because it's used by one of the most speed conscious databases out there. My implementation in Go manages to avoid allocations in almost all cases, which I credit to RESP's design.
There's a specific property of the protocol which is worth mentioning, as I think it's indicative of the design of the protocol at large. That property is this: integers are encoded simply as their ASCII-encoded form, with a predefined prefix and suffix to bound them. For example, the integer `1234` would be:
:1234\r\n
This may seem counter-intuitive, given that it's a binary-safe protocol. It's even more counter-intuitive when you note that ASCII-encoded integers are also used as the length prefix of blob (binary) strings:
$5\r\nHello\r\n
(The `5` in there is the length of the string `Hello`).
Here's the thing: how many bytes does it take to represent the string `5`? Answer: 1. How about the string `8000`? Now you're up to 4. If we were to encode the string length as a 64-bit integer that would be _8_ bytes. So until your string is 10000000 bytes long you're actually saving bytes by just using the ASCII-encoding. There's no significant loss in speed either, since it's quite trivial to turn an ASCII string into an integer via some bit-twiddling. You get the benefits of a traditional varint, but with a huge savings in simplicity.
There's no real gotchas or deal-breakers, but just some things of note which I could point out.
There are a few types and datastructures (besides the ones I noted) that are more specific to a client/server communication protocol, like errors and attributes. Whether or not these are useful is up to your use-case, but you certainly don't have to use them.
While the representation is human-readable for debugging purposes, I wouldn't describe it as human-_writeable_. Therefore I wouldn't recommend RESP for something like a configuration file which needs to be interacted with directly by a human. I would instead consider it in places where you might otherwise consider using BSON, protobuf, or MessagePack.
My love for Redis comes from something beyond its surface level features and benchmarks. There's a certain attitude of design used which I really admire, a willingness to do something different and odd, and to then find the hidden benefits of that approach and build on them. This attitude permeates the entire design of Redis, its serialization protocol included. You can feel it in decisions like using an ASCII-encoding of an integer as its wire representation. It _seems_ like the wrong move, but in practice it's actually pretty reasonable, and comes with some hidden benefits.
I've learned a lot from antirez, Redis's founder and lead maintainer until relatively recently. His blog is worth perusing through if you want more of his odd perspective:
And I hope that with this you'll appreciate this odd-but-great serialization protocol, and perhaps consider it for your uses as well.
========================================
Published 2023-05-29 by mediocregopher
This post is part of a series!
========================================
-- Response ended
-- Page fetched on Wed May 22 00:45:03 2024