gemini://gemini.hitchhiker-linux.org/gemlog/february

It's been entirely too long since I posted anything to this gemlog, and in fact my online presence has been almost non-existent for a while now across all platforms. When I think about the reasons it's not just that my current projects aren't really Gemini related, or that there's a lot of unfinished work sitting around. Those are factors, yes. It's got more to do with the feeling up until last week that it was something like January the fucking 75th, my job sucks, the hope of getting betterr employment is on hold while the tech sector reboots a bit, and my truck is suicidal. Even my dog seems to be less than chipper these days, although he might just be picking up on our moods.

Living paycheck to paycheck is probably one of the larger causes of my current lethargy. I've been putting off anything that might require spending other than paying bills and buying minimal groceries for so long that it feels neverending at this point. Moving to a smaller house was done in order to help fix this, but we were dug into enough of a hole that it's probably going to take the better part of this year to dig out.

I actually have been busy on a couple of projects in my spare time. Shitbox is my tongue in cheek take on a multi call binary in the vein of Busybox. I started it with the idea that it would eventually go into HitchHiker as a replacement for the current mishmash userland that is made up of utilities ported from BSD, some suckless tools, a fair number of utilities that I wrote myself in C when I started getting serious about programming, and a few tools from Util-Linux that were hard to replace. Shitbox is coming along nicely and covers about 50 commands at this point. It's split into three binaries currently, as I don't like the idea of letting the binaries get too large. I also recently split a few of the library modules into separate crates and made a workspace to promote more parallel compilation, although I'm probably not going to publish this on crates.io and everything is in one repo.

I've done a few things differently in Shitbox than a lot of other Rust projects. Rather than use a crate like `nix` to abstract over Unix interfaces, or even just calling unsafe functions from the `libc` crate for those things that `std` doesn't have an abstraction for, I'm making those particular syscalls directly using the `sc` crate. I had originally created my own syscall module which used inline assembly, until I found sc and realized that the code I had written was almost exactly what they had. The advantage of the `sc` crate is that they also include a nice macro which allows calling one function instead of a different function depending on the number of args.

This relates to one of my beefs with Rust as it is currently implemented. Rust's `std` library links to libc and requires it for most of this low level functionality. This creates a chicken and egg problem for a lot of really low level programming. An example would be Redox, where there is a circular dependency caused by their libc being written in Rust, but the rustc requires libc, drastically slowing down progress on getting the system to a self hosting state. Or using Rust on bare metal microcontrollers, where you're always stuck in a `no_std` environment. A significant portion of `std` could probably be implemented and usable for microcontrollers if not for the libc dependency. It's also a problem that `std` is monolithic. You can't just use one module from `std`, you have to have the entire thing. This is an area where I think Zig gets it right, as you can use almost all of Zig's `std` in almost every environment. It's also lazily built, so you can pick and choose those parts that you need without having to include the entire library. But Zig is still not in a state where I'm ready to use it in production, and I really like a lot of the other things that Rust provides too much to do without them (traits, the borrow checker, great documentation, etc).

As an aside, it was fun dropping down to assembly and learning a little bit about assembly for several different platforms. I was proud of myself for getting it working. And it was really hard to throw that code away in favor of someone else's better implementation.

HitchHiker Linux itself is in a bit of a transitional state right now. I badly want to switch to Musl libc. The musl branch I got compiling first for aarch64, but encountered more problems when targeting x86_64, even though I was building on the same arch as the target. The problems often come down to libtool and pkgconfig, but also sometimes are a result of someone attempting to write their own build system using homebrewed shell scripts. Now, I hate autotools with a passion, but I would much rather deal with autotools than sixty different homebrewed shell scripts. Anyway, it's often difficult to convince the build system that you are, in fact, cross compiling, if the build and target arch are the same, even though the host libc is Glibc and the target libc is Musl. Build systems often try to be smart and in the process show just how dumb they actually are. The compiler knows where it's sysroot is, and I can compile my own C code just by using the appropriate cross compiler and the flags to tell it what libraries to link, and it will know to look for includes in ${sysroot}/usr/include and libraries in ${sysroot}/usr/lib without having to be told, but it's really breathtaking seeing how often this gets broken when people try to overthink things with tools like libtool and pkgconfig, or make the absolutely unforgivable mistake of hard coding absolute paths into their `Makefile.am`.

At any rate, the amount of time I spent fixing build issues was in large part responsible for Shitbox in the first place. I'm tired of patching basic infrastructure to get it to cross compile. And I want HItchHiker to actually be a real alternative with a lot of it's own code, not just a repackaging of the same code everyone else uses.

So after getting the three targets I care most about to compile with Musl (aarch64, riscv64 and x86_64) I set about trying to build things using the latest pkgsrc. Turns out you run into some issues early on. For instance, the ca-certificates package is pretty much required to anything using ssl, which is probably 90% of anything network adjacent at this point and a whole lot of other packages that you wouldn't expect to require any kind of networking. Anyway, ca-certificates has a build time dependency on python-crypto, which has been relatively recently rewritten in Rust, and so requires Rustc running on the new host, which requires llvm. And llvm presupposes that if you're on Linux then you've got the GNU libunwind, which you don't if you're using Musl. There is a standalone libunwind used to this sort of thing, but I haven't getten around to incorporating it into HItchHiker, so that's where I'm currently stuck.

There's also a problem with the version of ex/vi that I was using, so I'm on the lookout for a lightweight vi replacement. I already replaced `ee` for similar reasons using the `kilo` editor as a great little simple text editor for those who don't like vi, but I want there to be a real `vi` in the base system as well because every system should have one. And I'm not going to use Vim at this point, because a huge portion of the Vim user base has already switched to NeoVim so why carry all of Vim in the base system?

February Blues

2023-02-07

Tags for this page