Taking Notes for a Bitrotting Internet

Search is getting worse. You've noticed it. I've noticed it. Everyone can see it plain as day. It's not just slowly getting worse either. I feel like the 2nd, 3rd, &c. derivatives here are all negative.

I wish I had more than anecdata and vibes to explain this but of course any attempt to actually search for why search is getting worse mostly seems to turn up results that are SEO garbage and a couple of short blog posts and just a few articles over the past couple of years

there's this freakonomics podcast about it but they don't really have data either just an attempt at explaining the anecdata

and here's an atlantic piece that attempts to sell us on the idea that actually google has gotten better we just don't realize how nuanced and wise it is

And I'm not here to lament search per se, I'm just laying out my thoughts about what to do about it

There are things like SearX that are meta-engines meant to help with search

https://searx.space/

A friend collected their thoughts in a fediverse thread about alternative search engines

https://mastodon.sergal.org/@slightlyflightyone/109820271784808859

but that's not what I want to talk about this time.

With the integration of large language models (LLMs) into search engines *and* the use of LLMs to generate even more SEO garbage meant to feed affiliate links, I don't think we can even rely on alternatives or meta-engine hacks in the near future

just check these youtube results for examples of all the affiliate marketing mess

So what do we do? I think we have to spend a lot more time building our personal note systems and collaborative wikis/wiki-like group knowledge repositories. The latter is a topic that I don't actually know that much about so I'm going to focus on the former.

So what I use is Denote, which is an emacs based note taking/organizing system

https://protesilaos.com/emacs/denote

But I don't want to talk about denote per se, I want to talk about the general principles of what I think we need for our notes and what we need to do when taking notes.

First, the point here is that I don't think you can ever rely on finding the same thing with a search query twice. I've been bitten by this a lot lately. So if you see something interesting that you want to reference again, you really need to save the link. Second, I also think we have to worry about sites going offline and/or becoming paywalled when they weren't before. So if what you want is to be able to quote something for later, maybe just copypasta that text along with the link in case the link dies. If what you want is, instead, things linked in the article you can pull out those links yourself. This also ties into the prevelance of discord servers as the new forums. It's not uncommon for discords to just shut down suddenly and when they do all that information is just lost. Pre-empt this by saving as much information as possible from those servers.

More broadly, I think we can't rely on just general information lookups from searches either. If you're like me you probably got used to searching for common sysadmin tasks or boilerplate incantations when coding. I think it's best to assume that won't work either, not even for common problems. I think it's not unlikely that we're going to see more proper search get replaced with LLM interfaces, ones that produce plausible looking text but should be presumed wrong until double-checked.

So let me talk about what I think are good qualities in a note taking system:

it should let you deal with both external links and internal links between notes equally easily

you should be able to classify information by tags or other metadata

it should store notes in a text format that can be searched by external tools like grep

it should depend on open-source software

it should let you keep all your data locally

So like I said I used Denote, which is a very very minimalistic system for creating files with a particular meta-data-laden file name convention and links between files that are independent of whether the name of the file changes, letting you update tags or names. I use it with .org or .md files because I like the ability to markup quotes from documents.

But, again, the particular system doesn't matter. I think things like

Obsidian

Logseq

org-roam

tiddlywiki

and I'm sure many others would work just as well. What I absolutely don't recommend are things like Notion: not just because they don't allow local storage of your data but they're also trying to integrate LLMs into their product now too.

I'm not always good at following my own advice but I've been putting in a lot of work over the past year and change into keeping notes and I've also been noticing that whenever I don't save links, quotes, text, &c. myself I can't ever find it again.