2024-01-27 - Updates to in-progress separated specifications

Happy 2024, Geminatus!

Over the past week or so I have some updates to the separated, more formal specifications for both the Gemini network protocol and the "gemtext" markup format. As a reminder, these are non-binding works in progress so nobody has to update their implementations in response to these changes. However, I intend for these to become official and binding by the end of March at latest, so implementers ought to keep an eye on them.

Changes to the protocol specification

In response to it being pointed out that the current specification includes occasional references to the concept of "proxies" (mostly in the context of defining error codes) without actually explicitly defining just what a proxy in the context of Gemini is supposed to be or how one works. So there is now a section dedicated specifically to this subject. There are already multiple implemented and working proxy projects in the wild, mostly in the form of protocol gateways and format translators (such as Agena, a Gopher-to-Gemini proxy, and Duckling and Stargate, both HTTP(S)-to-Gemini proxies), and nothing in the new section is expected to require any change to those projects at all. Gemini-to-Gemini proxies are, to my knowledge, not widely used yet, if at all, but they make sense and there are certain situations where they might be of value to some people. Some clarifications have been added and some important considerations highlighted which should apply to these cases. If you are already developing and/or using such a proxy and anything in this new section would require you to make serious changes, I'd be surprised but I'd like to hear to from you.

The protocol specification has a nice section at the end with concrete examples of various Gemini transactions. I have added one more which demonstrates the "conversational" idiom for server-side Gemini applications where client certificates are used as keys for maintaining state, which allows multiple separate pieces of information to be sent in separate requests each with their own informative prompt, and without requiring specifying semantics for HTTPisms like & and = in the query. At the same time I made a very tiny change to the previously existing example of requesting a client certificate to make it an example where the certificate is not used for keeping state but as a form of access control. In this way the examples now demonstrate that client certificates can be used for multiple purposes.

It has been clarified that when clients collect input from a user in response to a status code of 10 or 11 then clients may offer the facility to enter text composed of multiple lines. I have specified that clients should encode the the linebreaks as `%0A` (LF) but that servers should recognise both `%0A` and `%0D%0A` (CRLF). This encourages clients to adopt the form which seems to already be in the most widespread use, and which uses the fewest of the 1024 bytes allowed in a request URL, but does not oblige any clients already using the longer form to change and encourages application developers to make sure they are supported. I don't anticipate anybody being upset or inconvenienced by this.

Finally, the discussion of both client certificates and TOFU-validation of server certificates has been changed so that the combination of hostname and port number, rather than the hostname alone, is to be used for determining the scope of a certificate. This same change will likely be made to the current official and binding all-in-one specification before the protocol-only specification takes it place - I'm only waiting on being ready to release compliant versions of my own client!

Some other changes have been made but they are "just textual", such as moving around and splitting up some of the TLS section and also removing some text which I considered "too waffly" for the document. That's not to say it was wrong or incorrect or poorly written, but the end goal is to end up with most of the "crunch" of the current all-in-one spec condensed into the two separate ones and most of the "fluff" (which can still be very valuable clarification of contextualisation) in the "light technical tour of the whole Gemini stack" document that I want the all-in-one spec to evolve into once the two separate ones are ready.

Changes to the gemtext specification

The separate gemtext specification was until recently basically just the relevant sections of the all-in-one spec copied and pasted out. Now for the first time there is something like an attempt at a standalone formal document in the same overall style and tone as the network protocol specification. It is not necessarily finished yet, but it's much closer. I have not deliberately written anything which should require any existing gemtext parsers or gemtext documents to be changed in any way.

The biggest change in how gemtext is explained is that the modal nature of a compliant parser is now emphasised more and I have taken advantage of the possibility this allows to describe gemtext in terms of six line types instead of the previous seven. In the all-in-one spec, text lines and preformatted text lines were counted as distinct types. Now they are both just text lines, but the rules for how to handle text lines are mode-specific to achieve the same effect. The rules for how to handle the mode toggling ``` lines are now also mode-specific and this makes also allows for a slightly more succinct representation of the previous behaviour whereby content after the ``` of a mode-toggling line when toggling from "preformated mode off" to "preformatted mode on" may be used as alt text but similar content when the mode is toggled the other way ought to be off. I want to emphasise again that nothing meaningful has actually been changed here and no code updates are required, it's purely a change in how the status quo is described.

At the end of the document is a formal grammar for a gemtext document in augmented BNF form. I'm pretty sure it is not yet perfectly complete and accurate but I'll make sure that it is by the time this becomes the binding specification. For now treat it as a kind of aspirational marker of intent.

Both of these documents will continue to evolve in February and I will also make some more news posts outlining intended changes and soliciting feedback from potentially impacted parties. Please stay tuned!