Documents and Directories

Published 2019-08-20

I think we need to take a step back and start by defining how people will be using gemini to browse and consume content. WWW and Gopher have very distinct models, and it sounds like the arguments for text/gemini and markdown mostly boil down to where gemini falls within this spectrum.

Content Categories

There are two broad categories of content that are worth consideration here:

1. Directories

2. Documents

Directories are what facilitate content discovery on your platform. These pages don't contain much, if any, actual content. They instead serve as jumping boards for users to find what they're looking for. Examples of include google search results, a news index page, or the floodgap "New gopher servers since 1999" list.

Documents are the end goal of why people are using your platform in the first place. They contain the actual information that people want to consume. Examples include a blog post, a news article, or an informational page on 19th century basket weaving.

I don't include non-text media (images, videos, etc.) in the second group only because they are not relevant to any of the points that I want to discuss.

Gopher

Gopher has a clean separation between the two types of content:

directory -> gopher menu

document -> text/plain

Gopher menus are the directories, plain and simple per RFC 1436. The "i" type was later introduced and allows embedding some additional metadata in gopher menus. Some people (including myself) have abused the "i" type to the extreme in order to create a sort of hybrid markup document that can contain both links and content. But I have seen many gopher purists advocate that this is bad practice and a bastardization of what gopher is supposed to be. The current "best practice" seems to be to include any outgoing links in the gopher menu, alongside the document, instead of embedding links in the document itself.

Gopher documents can be any type, although in practice the community has gravitated almost completely toward text/plain with wrapping at around 70 chars. There's nothing in the gopher spec that prevents people from composing phlog posts as HTML documents and serving them over gopher using the "h" type, but I have never seen anyone actually do this. We stick to plaintext because we know that all gopher clients will be able to render it.

It's worth mentioning that this separation between directories and documents leads to a "tree" like structure, where gopher menus are the branches and text files are the leafs.

WWW

The web blurs the lines a bit by using the same format for both content types:

directory -> text/html

document -> text/html

WWW directories are HTML pages with a heavy emphasis on linking to other pages. WWW documents are HTML pages with a heavy emphasis on displaying content.

Of course I'm glossing over a lot of nuance here, since web pages can, and often do, act as both directories and documents at the same time. This has lead to WWW being described as a "web" of documents with no clear hierarchical structure.

Gemini

So how does gemini fit into these two models? Gemini has been described as a mix between gopher and the web, but on which side of the line does it fall?

Scenario 1

This is more-or-less what the current standard is leaning towards today:

directory -> text/gemini

document -> text/plain

text/gemini acts as "a better gopher menu" that is easier to parse, but is too limited in scope to serve any type of complex documents because it lacks basic things like bullets.

In this case, gemini content is primarily structured like gopher space, in a hierarchical tree.

Scenario 2

This proposal is to beef up (or outright replace) text/gemini with some flavor of markdown that can be used to create documents

directory -> some markdown dialect

document -> some markdown dialect

The directory type acts an "a better HTML", but it's difficult to create a markup language that is both expressive enough to satisfy the needs of the document type, while also being simple enough to write a trivial parser for.

In this case, gemini content is primarily structured as a web of documents.

Text reflowing

I have previously said that I feel very strongly that the text/gemini spec should allow clients to control how text lines are wrapped. This enables clients to format text in a manner that is the most accessible to their users.

However, even if I win the argument for text/gemini directories, if people still serve their documents as text/plain, it kind of becomes a moot point. What good is it if gemini directories display nicely, when everyone writes their gemini content as plain/text filled with ascii art and hard line wrapping?

On the flip side, what would happen if we kept text/gemini for directories, but made the recommendation that documents should just be served as text/markdown? It would still be simple to write a gemini client to parse directories, but that client wouldn't be able to render most of the interesting content served over gemini.

Takeaways

I have a lot more that I could say on this subject, but I'm stopping here because I'm getting tired of writing.

The gemini spec should clearly define the content type that should be used for both directories and documents. These could be the same (like HTML) or different (like gopher menu + text/plain). But leaving it ambiguous makes it difficult to frame any discussion around what features the text/gemini format should include.

The gemini design goals (easy to parse, easy to read/write as text, etc.) should apply to both directories and documents. If a gemini client can't effectively render & display the primary document type for content over gemini, what's the point of the client?