It was 20 Messidor, year 231 when i wrote this.

Blueprinting Gemini Base Tests

There's been some discussion recently of building compliance tests for gemini servers and clients, as a point of unification. The web had the ACID tests which were a rallying point; the idea is to do something similar for Gemini.

I'm tentatively referring to this project as "Base Tests", because that's the opposite of an ACID test :) And also the literal interpretation.

ACID tests specifically tested web clients for compatibility with JavaScript and HTML standards. The exact format of the ACID tests cannot be translated to Gemini. JavaScript provides a method to programatically control web clients from within the client itself in a cross-client compatible way. There is no cross-client compatible way to programatically control gemini clients -- by design! So gemini's Base Tests will need to get more creative.

Gemini has a lot more diversity of general-purpose file servers than HTTP has. I'm not sure if this was always true, but I don't know of any commonly referred-to compatibility tests for HTTP *servers*, other than "can I load web pages served by this server in chrome?"

Possibly this is a side effect of the relative simplicity of implementing an HTTP server when compared to implementing an HTTP+HTML+CSS+JS (plus everything else in the WWW platform) client.

In any case, Gemini would benefit from both, and these won't be the same.

Client Base Test

ACID worked by running a ton of javascript and then checking the results of that execution in the client state to produce a score.

To test that a Gemini client both implements the network protocol and renders text/gemini documents correctly, we either need an interface (in the abstract sense) for clients to build against (very complicated) or write some sort of exhaustive document served by a bulletproof server; the document should clearly identify how each line should look visually so users can see at a glance if their client passes. Any errors in parsing the network protocol should be obvious to users... that one might be harder than the former.

An aside on the "interface" idea: I'm thinking something like webdriver for gemini clients, which would be ridiculous and absurd but really cool if it existed.

A static, document-based approach would need to include tests for multistep protocol interactions such as following redirects.

Server Base Tests

This one should be easier in the sense that it can be automated. Basically all this would be is a black-box unit test harness for a server which closely implements the spec from a client's perspective. For Gemini file servers you could totally overengineer this - automate generating text/gemini files and ensuring the server responds with the exact file in various forms of reference, url shenanigans, etc. In order to be inclusive of non-file-servers something more generic may be desired.

Getting it right

Demoing this project against actually existing gemini servers may make it tempting to test too much. Here is a list of things that the Base Tests should absolutely never, ever try to test for:

Details of provided TLS certificates, such as hostname or expiration date

(to be continued...)