-- Leo's gemini proxy

-- Connecting to gemini.dimakrasner.com:1965...

-- Connected

-- Sending request

-- Meta line: 20 text/gemini

New Stuff in Tootik


My tootik instance


tootik is maturing quickly, and I use it for many things: for example, I follow bots that monitor news sites and publish summaries on Mastodon, allowing me to consume news (and replies) in plain text form through a Gemini client.


Thread View


When viewing a post, tootik shows mentioned users, links [...] and a compact lists of replies. Every reply is accompanied by a link to the same view but with the reply as the "post", so one can see replies to this reply. To read all posts in a thread, the *user* (yes, the human user) needs to follow this link recursively and this gets annoying very quickly, because it's easy to forget the context and one needs to find the link (which is not always in the same position on the screen) to follow it. Also, this doesn't scale: if a post has two replies, and both have more replies, it's hard to read one "branch" of replies, navigate back to the "junction" where this "branch" begins and repeat the recursion for other branches until you read everything, unless you have a "map" of the thread.


Now, tootik provides such a "map": posts with nested relies are accompanied by a link to a "thread view" shows all "branches" and sorts "sibling" replies chronologically. It's much easier to follow a conversation (or a heavy debate) now, especially if you use a client that marks already visited links, like Lagrange.


I think at least 3 users asked for this feature, and so far I haven't received negative feedback.


https://github.com/dimkr/tootik/commit/84a388d2bf5993492652b81dcc52441b4256cb84


Polls


tootik now allows users to vote on polls, view a poll results graph and publish Station-style polls.


Poll support is important from a project maturity viewpoint, because it creates many corner cases that must be handled (like the visibility of "vote posts").


https://github.com/dimkr/tootik/commit/0063bd52fb04cf129e708f8377ba343f8e06e78d

https://github.com/dimkr/tootik/commit/ce6fb8da66c4e7cca4ff41be2c23362293724ffe

https://github.com/dimkr/tootik/commit/1871ce83210b6cf04081e9d6905e0550760e2d50


Help Page


The addition of polls adds more "rules" one needs to know when publishing posts, making some kind of documentation that's integral to the tootik UI (as opposed to, the git repo) a necessity.


https://github.com/dimkr/tootik/commit/8bcdcc628044add5b5668c193d42186649ee4476


Shared Inbox


tootik now sets the endpoints.sharedInbox property on users (to the inbox of the special "nobody" user), so a tootik instance with two followers of the same user on another server receives only one request when this user publishes a new public post, or when a poll published by this user receives more votes. This reduces resource consumption a lot, and the number of requests tootik needs to handle has dropped significantly. (I still see some duplicate requests that get filtered; I guess some serveres still have old cache with unspecified endpoints.sharedInbox.)


In addition, when a user of a tootik instance has two followers on another server, both with the same shared inbox, tootik now sends only one request to this server when the the user publishes a new public post. This reduction in the number of outgoing requests speeds up delivery of posts to other servers.


Poll results are sent to servers with followers of the poll author, so a long list of active polls is not a big problem even if the poll author has many followers (assuming they have "shared inboxes" and actually share them).


https://github.com/dimkr/tootik/commit/b52c2b30025a7fa459369922754cf2ab0d2a3006

https://github.com/dimkr/tootik/commit/cde8fff0a9d3eb50c4a7b4e7c9b3014f40d379d6

https://github.com/dimkr/tootik/commit/18dc00c58096767f32df2e3ac64f06d0db772b28


Access Control


Previously, tootik allowed anyone who knows the "secret" hash of a post ID to see the post. Most post IDs contain something that's hard to predict, like a timestamp or a sequence number, but there are many corner cases where one can "discover" this "secret" (for example, through another post).


Now, tootik filters content by the "to" and "cc" fields, and all fixed corner cases are reproduced in newly added tests.


https://github.com/dimkr/tootik/commit/87d9ca2658f610c463d0599408ff68f42f107da8

https://github.com/dimkr/tootik/commit/9265a8b803350175bd124b14cf8cb33f5c1f99f3

https://github.com/dimkr/tootik/commit/2a3dc897a2bcdc702c1f464b7ee1b546379553dc

https://github.com/dimkr/tootik/commit/2fc6a7515784051f6896b091b178518d45675b85

https://github.com/dimkr/tootik/commit/6003654115e572c872b81067e1f5e498c727eb3d


Performance Optimization


The slowest pages (/users and /users/inbox) are still slower than anything else, but they're much faster now. Previously, a big query used to match all posts within a time frame against the user's ID, followed users, followed groups and posts by the user (to show replies), and it's impossible to speed up such a query using an index. The query reads many columns from the posts table, forcing it to do a slow "scan" that checks for many conditions on many posts, and a big index would only slow down insertion of posts without speeding up the query. Now, each "why should I see this post" condition is a separate (and fast) query: search of posts by group can use one index, search of posts by author can use another, search for replies to posts by a specific author can use another, and all queries can run in parallel to maximize utilization of the disk's read bandwidth.


With all these new features, my instance still runs comfortably on a $5/mo VPS. Total RAM consumption (with the underlying OS and evreything) is around the 120 MB mark, and CPU consumption is usually <=5%.


https://github.com/dimkr/tootik/commit/c506d9457b1a4b6805e3bda4abe89ef362d14b2d

https://github.com/dimkr/tootik/commit/01921857d65719315eeb2fb13ab3489f7449f72f


Tests


tootik's test coverage has increased a lot: pretty much every change is now accompanied by tests and bugs are fixed once they have tests that reproduce them (then act as regression tests). It's much harder to write broken SQL now.


guppy:// Support


tootik now supports guppy://, in addition to Gemini, Gopher and Finger. This new protocol will be useful for low latency consumption of public content like news, and every user that consumes "feeds" (like the federated posts feed) over guppy:// instead of gemini:// reduces load on the server. I can see myself using this on the long way to work, if and when Lagrange for Android gains guppy:// support (I'm already using guppy:// on two devices!). TLS handshakes time out often due to dodgy mobile signal, while guppy:// allows pages to be received in a single burst of packets, without a preceding three-way handshake or key exchange.


https://github.com/dimkr/tootik/commit/bfb085ec82a655e04238da023c527d53ba0ff53d

The Guppy protocol homepage

Lagrange Guppy support pull request

-- Response ended

-- Page fetched on Fri May 10 08:07:39 2024