-- Leo's gemini proxy

-- Connecting to cadence.moe:1965...

-- Connected

-- Sending request

-- Meta line: 20 text/gemini; encoding=utf-8; lang=en

↩ Home

⬆ Gemlog


Aggregating feeds on Gemini


> 777 words, 25 paragraphs, about 3 minutes to read (300 wpm).

> First published on 2020-12-15.

> This article has been loaded 2280 times.


There are people on Gemini that I like to read posts from. The problem? Keeping up with what's new. Some of them post daily. Some of them post rarely. Checking all of them is tedious.


Existing solutions


CAPCOM


Visit CAPCOM


CAPCOM is Atom, which is XML, which I'll talk about in a couple of paragraphs.


Spacewalk


Visit Spacewalk

"Explanation"


I don't FUCKING understand it.


So I'm going to do my own thing. Let's discuss the technologies that I could use.


RSS


On the Small Web, this problem is solved with RSS. People create XML feeds of their content. Your RSS reader periodically requests everything and compiles it into one chronological listing. It's great for the web. But it doesn't work with Gemini, and there are a few reasons why.


RSS is HTTP


The most obvious reason is that your feed reader can't load gemini:// URLs directly, because it was only designed to work with HTTP/S. Not much else to say on this point. It's a pretty straightforward killing blow. Yeah, I'm sure you could use HTTP proxies and stuff, but this is Gemini, where everything is supposed to be straightforward.


RSS is XML


XML is cool. HTML is cool. Gemini is neither.


Gemini deliberately avoided HTML and XML in its design. While clients technically could load it, the standard format is gemtext, which is based on lines, and parsing it is trivial. For comparison, while parsing XML is a pretty common task and most programming languages offer libraries for it, it's not trivial to somebody to implement their own new parser, or to understand it in a language that does not have fancy constructs like objects, maps, and trees. Gemini lends itself well to deliberately restrictive programming, for fun. RSS does not.


RSS requires publishing tools


I've written about this already in my previous post discussing Gemini.


(You can read it again here if you want to.)


The summary is that the complexity of the web and of HTML means that they need publishing tools, like a static site generator or Wordpress, whereas Gemini/gemtext lend themselves very well to being edited entirely by hand. This, combined with a resurgence in tilde computing, has led to an excellent harmony.


RSS does not work well with this. RSS is XML, and XML does not lend itself to being edited by hand.


Relying on RSS in an aggregator means that, of course, you can only subscribe to people that publish an RSS feed for their articles. Not everyone does this.


We need to rethink feed publishing for Gemini. RSS is not the answer and will never be.


Index pages


Index pages are the answer.


Everyone with a log on Gemini already has an index page that links to all the log entries, includes their title, and the date they were published.


Index pages do not require publishing tools


If you want to, you can use publishing tools to generate your index page. I do that for mine. But it's certainly not a requirement.


People can just as easily manually create an index page and add the link to it whenever they finish writing an article. This is easy for publishers and easy for readers. People do this today.


Anyone can write an index page.


Index pages are Gemini


Index pages are a plaintext, line-based format. There's one link per line. The only parsing that needs to be done is extracting the date and title from the link. Due to the format that almost everyone already uses, without being asked to, this can be done with a regular expression.


Index pages are already feeds


There's no need to provide an XML file if your index page is already an up-to-date, machine-readable list of all your articles.


Index pages are feeds. Let's use them in a feed aggregator.


My feed aggregator


Here's what I managed to create.


It browses index pages, extracts dates and titles from them, and then it all gets ordered into a single feed, which itself is available over Gemini.


The feeds can be updated by calling a script from the command line, which can be added to cron or your favourite scheduler.


I'm somewhat happy with this, but not entirely. It's still a work in progress, and hopefully one day I will be satisfied.


The source code isn't published, and I won't publish it until I'm satisfied enough with how it looks and works. Pester me if you want to use the code yourself and I'll see what I can do.




---


> No replies yet.


Thoughts on this? I'd love to hear them! Write a reply.


---


Looking for more?


[NEWER →] 2020-12-17 - Future of Bibliogram

[OLDER ←] 2020-11-16 - Federation and its consequences have been a disaster for the fediverse

[RANDOM ] 2022-01-28 - Command line computing part 1: Introduction


Or, back to the gemlog index.

-- Response ended

-- Page fetched on Fri Apr 19 18:49:22 2024