Editor Datatypes

Well, I’ve been thinking about some problems I have with Emacs.

Single threaded

And I know that there’s a bunch of ways to run some parts asynchronously, the issue is that many packages and built-in functions don’t and aren’t made/prepared to run that way.

For example a simple replace-regexp that I run so often, just locks up the entire interface, which is especially bad when you use EXWM or have half your programs as Emacs functions.

The issue is that even if you rewrote a part to run the functions asynchronously, there’s not really anything that accounts for that. Maybe the two running functions edit the buffer at the same time, or each reads from the cursor but the cursor is expected to be somewhere and another function or the user moves it. There’s a lot that could happen, most of it not good.

Can’t access Libs

Emacs afaik has no way to just load a library and call functions from it. Which means that even if you do want to extend Emacs with new functionality, you have to bundle it up in an executable and pass around information with pipes.

This prevents one from really going all out and having Emacs be a full-on shell for all your needs which I really want to have. It also seems to be part of the reason why I haven’t found no Wayland compositor progress.

The Elisp is already written though

It would be a shame to lose all the already written code though, preferably it would be possible to run it in some limited capacity. Perhaps that code could be marked or mapped somehow to be used from the new environment.

Fails on large files

Anything above a few thousand lines seems to tank performance a bit, but that’s been fixed up lately, still it has a limit and I feel like, if I have 4 Gigs of memory I should be able to just load a few Megabyte file without bigger problems.

My Thoughts

Soo, after a while, I think the core might want a rewrite, not necessarily that the Emacs core is in any way bad, but I honestly just wanna try writing a text-editor. Just mimic the old ways with a new implementation that mostly already has tooling for all the things I need to do, and of course in pure Lisp cause why do it any other way.

Data Structures

What surprised me was that such a simple idea as insert a character at an arbitrary point and retain history of changes is kinda unsupported by mainstream data structures. Emacs uses something called a Gap buffer[a], which is conceptually simple and even already implemented in core Guile, but doesn’t lend itself to jumps well, since you basically have to copy the entire thing every time you jump a long way. And it doesn’t consider that text is very often worked on as a 2D structures, where newlines are basically not even a character but rather a structural divider. I found something quite interesting that’s called a Rope[b], and that would actually work nicely. It’s actually quite talked about but I didn’t find an implementation and I’m not in the mood to be writing it about now. I found a beautiful port of pfds[c] for Guile Scheme, that contains finger trees[d]. These quite fascinate me and seem to have the properties I want, around supporting random access and at least logarithmic times for splitting and concatenation (which can simulate insertion) even having the n be the size of the smaller tree so a small insertion into a large tree should work well enough. It also has some other features that I have to explore later, and might warrant a reading through of an implementation anyways.

Guile as the base

Guile actually supports a bunch of what I need already, which is why I decided to try using is as the new core. It has a nice module system to load packages into and a whole FFI system for what you can’t write as lisp. It also has premade methods for basically everything one could imagine already, so much of what Emacs can do is already included. Guile also has an alternate language facility where Elisp is already proof of concept implemented, which is perfect because Guile languages can call functions from each other and interop nicely, so theoretically one could load Emacs packages and call them from Scheme, but that’ll take a bunch of extra work anyways.

Async

I’m thinking about running stuff in parallel by default as much as possible, however you can’t have two functions editing the same buffer at the same time, nor at the time some other administrative task is being performed over the buffer, since that could come up with garbage information even if you only read. My thoughts so far are to run one mutating function per buffer, and use the pfds functional property to have an unchanging view of the buffer, accessable to any number of reading function that’ll just get called again with the updated buffer if needed, that shouldn’t be that much of an issue. One might even play around with extra info about the fn to prevent doing that unnecessarily. This way you could basically consider everything idempotent and side-effect free which just sounds so nice.

The UI

For now I’ll just stick with a TUI, the excellent article that explains all about how to be god of the terminal can be found here: on terminal control[a].

[a]