2024-03-31 - Specification changes, quarterly release numbering, and more plans

Today I have pushed updates to the official Project Gemini capsule to the effect that the two separate specification documents, one for the Gemini network protocol (very closely descended from that written by Sean Conner during my absence from leadership, with collaboration from many others via Gitlab, to all of whom I am most grateful) and one for the "gemtext" hypertext format (written by me but with the layout and overall style heavily influened by Sean's protocol spec), are now the official and authoritative sources on what Gemini is. Ye olde combined and less formal (but still formal enough to avoid any major interoperability issues for close to five years) has been renamed as a "technical overview", with the clear caveat that in the event it contradicts either of the formal specifications, those specs take precedence. It is, for now, mostly the very same document it ever was, but in the near future it will be lightly edited to include less exhaustive technical detail, which is now supposed to be the domain of the formal specs, and perhaps more explanation about the context and consequences of certain design points, which I still feel to be of wide interest and value, but which is also somewhat out of scope for the formal specs which are just supposed to be raw hard technical truth.

The overall intent now is that the FAQ (particularly the first section) should be accessible to a relatively broad audience and should explain what Gemini does, what it can be used for and why you might want that, without going into any technical detail on how it works, while the technical overview provides that missing detail and should satisfy most curious geeks as to what's going on under the hood, and should be enough to get people started writing basic Gemini software, and the formal specs are where you go to settle arguments or when you're really not sure what to do in corner cases. They are also the documents, or will form the basis of the documents, which may one day in the future be submitted to various standards bodies. In the case of inconsistencies, the formal specifications trump the technical overview, and the overview trumps the FAQ.

My original plan was that I'd make a final update to the combined specification before declaring the separate specifications official, such that nothing actually changed as a consequence of that declaration, it would just be a new authoritative source for exactly the same information. In the end, this felt a bit too much like needless busy work. The actual changes are relatively minor, and have been discussed in this new feed previously in the past, so nobody should be caught by surprise, and there is no desperate sense of urgency behind the changes. Yes, this means that, for now, I have delayed deciding whether or not to "regularise" the protocol specification by making not just a MIME type for status 20 responses mandatory, but for all statuses (that door remains open, though). As per the old mailing list tradition, there is a careful list of all the changes and what they obligate various people to do at the end of this news post.

Both of the two formal spec documents now have version numbers on them, namely 0.24.0. Here I have borrowed a page from the book of Drew DeVault's programming language Hare. Hare shares a clear design goal with Gemini, namely "Hare is...designed to be stable for the long-term: when we’re done with Hare 1.0, the language syntax and semantics will freeze and it can be depended upon indefinitely". This is precisely how I have always wanted Gemini to work (see FAQ entry 4.3.3). The day will come when it won't make sense to talk about Gemini specification version numbers, because the spec will never change again and the majoity of widely used software will be compliant with that eternal spec. But we're in an awkward interim phase now where things are still occasionally changing and its useful to be able to refer to these changes when talking about how up to date various software implementations are. There was always (or at least for a long time) a 0.x.y version number on the old combined specification document for this reason. The last version before this great splitting was 0.16.1, so the jump to 0.24.0 for both of the newly officialised specs represents a discontinuity, but not an inconsistency as such, as newer specs still have higher numbers than lower ones. Why the jump? I've decided to adopt the Hare system of quarterly updates, where "each version number is 0.YY.Q, where YY is the two-digit release year and Q is the quarter (zero indexed) in which it was released". Quarterly spec updates feel like a good solution to keep up some momentum on finalising what still has to be finalised while keeping the pace of change relatively gentle. Of course, there will not necessarily be updates to both or either specification every single quarter.

Also note that both the formal spec documents are now explicitly CC0 licensed, i.e. released into the public domains. I am mellowing on that issue, compared to when I wrote my 2023-10-03 newspost.

My rough plan for April includes, among other things, carefully going through all the currently open issues at the Gitlab repositories where the new specification documents originated, and closing those which were either already adequately dealt with during Sean's reign, or which have been resolved by the appearance of a formal specification for gemtext. Then I will decide which of the remaining issues I want to target for the 0.24.1 release in three months time. Consistent with my previous plan of splitting the roughly year of time until the next "Apollo Day" (see 2024-02-21 newspost) into two six month blocks and not worrying about IRI-related issues for the first block, these target issues will be not be IRI-related. I'll announce the chosen targets in this news feed in the near future.

Summary of specification changes which require software updates

Providing a MIME type for status 20 responses is now mandatory

This was previously optional, with a default MIME type of `text/gemini; charset=utf-8` to be used by clients in the absence of anything provided by the server. If you are the author of a Gemini server which sends status code 20 responses with no MIME type, you MUST change this to regain spec compliance. If you are the author of a Gemini client which does not handle missing MIME types, your client was previously NOT spec compliant for this reason, but it now IS (or rather, it MAY BE, who knows what else you were/are doing wrong) without you having to lift a finger. If you are the author of a Gemini client which did handle missing MIME types, you MAY remove this functionality without becoming non-compliant, but in the spirit of Postel's law I would advocate that you leave it as is.

Definition of status code 44 have changed

Status code 44 was previously specced such that the <META> part of the response header indicated an integer number of seconds which clients should wait before sending subsequent requests. This violated the design principle that simple clients can treat 44 as 40, as it would have resulted in that integer being presented as a human-readable temporary error message. The new network protocol spec does not override the semantics of the (optional for now!) text following a status code of 44. If you are the author of a Gemini server which issues responses with status code 44 followed by a number of seconds to wait, you REALLY OUGHT TO change your code so that the status code is followed by a human-readable error message. Strictly speaking your server is not non-compliant without you changing this, but rather it is sending very cryptic and non-helpful error messages. If you are the author of an automated Gemini client which handled the old status code 44 as it was previously specified, you MUST change your client to implement a request-rate reduction scheme which does not depend on being told how many seconds to wait, and that scheme SHOULD be an exponential back-off.

Port numbers now matter as well as hostnames for TOFU and client certs

The previous spec stated that client certificates should be considered to have a scope defined (in part) by the hostname of the server which requested them, and also that the recommended TOFU scheme for server certificates should associate certificate fingerprints with server hostnames. In both cases, the new network protocol spec refers to the combination of hostname and port. This change does not require authors of Gemini servers to do anything. Authors of Gemini clients which implement client certificates MUST change their handling to regain compliance. Strictly speaking the TOFU scheme described is RECOMMENDED and therefore spec compliance is not affected by whether you change your TOFU implementation or not but, well, I'd RECOMMEND you do. This is a tricky thing to fix retro-actively in theory because you no longer have access to the port number of the server which sent certificates or certificate requests in the past. In practice, assume that it was port 1965 and it's almost certain that nothing bad will happen.

Changes regarding redirects

The new network protocol spec is a lot more explicit about various details related to handling redirects, such as how queries and fragments should interact with redirects. If you are the author of a Gemini client which follows redirects, you MAY need to make changes to regain compliance. Some of you might be surprised that fragments are mentioned at all since they (currently) have no defined meaning for gemtext documents, but remember that there is nothing prohitibing the use of the Gemini network protocol for serving HTML or other content-types for which fragments *do* have defined meanings, so the network protocol as a general tool ought to address these. Also, the new spec states that clients MUST not follow more than 5 consecutive redirects, which was previously just a recommendation in the Best Practices document. If you are the author of a Gemini client which either does not count redirects or does count them but allows more than 5, you MUST change this to regain compliance.

Linebreaks in queries now specified

The new network protocol spec makes it explicit that the query component of a request may be comprised of multiple lines. Various clients and applications have supported this for some time already. The spec says that clients SHOULD encode linebreaks as `%0A` but that handling applications should recognise both `%0A` and `%0D%0A`. If you are the author of a Gemini client which supports multi-line responses to status codes 10 or 11 and which encodes linebreaks as `%0D%0A` you are not strictly required to change this to `%0A` but you probably SHOULD.