On Sources

I've mentioned that an artifact (a compiled function or data) ingested into harn is stored with all the metadata needed to recreate it. This metadata includes the source code of the artifact.

Note that unlike current systems, I am not satisfied with leaving the source in the middle of some text file in some directory. I want the source for that particular artifact in _my system_, and to that end, I keep a running log of all compiled C code in the 'sources' file.

The 'sources' subsystem

The sources therefore contain source code of all artifacts currently in the system. But it's more that that - it is an append-only immutable log of all artifacts that have ever existed in the system. In fact, it contains a record of all interaction with the system - everything you type on the command line.

For some reason this makes me a little nervous - I am a privacy-minded person. However, I think it is likely that everything you type is recorded somewhere. In this case there is actually a purpose to it. Like an inventor's notebook, you need a record of what you are doing. At least I do.

The other concern is that the file will grow large, but:

Who cares! I am a minimalist when it comes to many things, but in this case it doesn't matter. Your drive is way bigger than the sources file will ever grow.

It's worth it. What is more precious and important than the human time you are putting into the system, as expressed by your keystrokes and all the source code that ever enters the system! If you ever need to see an older version of some function, or repeat some obscure command you typed in, you will be able to find it.

You could compact it and eliminate everything but current versions of what's in the system, if you so desire...

A closer look

The sources file is an immutable log of text entries. The artifact metadata contains a reference (a file offset) to its source. That is how the 'edit' command pulls up the function for editing.

The sources file is a text file - searchable with grep or whatever. Just don't modify it!

In order to be more useful, the system inserts timestamps as comments. This is just a human convenience, so the timestamps are in ASCII. But most of the time, we, humans, don't produce much code, and to keep even 1-minute accuracy would create an awful lot of timestamp lines with nothing in-between. If we stamp each entry, an automated recompile could generate an awful lot of entries in one minute; again, it would be a waste to mark each one.

So the system is a little wiser about timestamps. When a timestamp is written, we keep it around. Next time we write something, we check if the timestamp is the same -- if it is, we don't bother writing it. That's the basic gist of it.

An interactive system thrives on incremental changes, and as we make changes to functions, we leave behind old versions of sources (the system garbage-collects old code). These may be occasionally useful, and the system leaves a comment saying the current source was derived by editing an older version at such and such position. In the future, perhaps, I will make an 'audit' tool to track how an artifact evolves...

And that is what I have to say about the sources today. I should probably go and make it actually do what I promised above -- as of today the sources subsystem is a bit neglected...

=>. index

=>.. home