A build system for those who hate build systems

This is sort of a response to Acidus' post:

And if you want the tl;dr go to the next post with the details on redo.

UPDATE: Apparently the gemini community is already into redo

I'm a developer by trade. A developer with a focus on getting the most work done with the least amount of effort possible (lazy much?). My goal at any job is to get to the point where a good portion of my daily mundane work is automated which leaves me time to focus more on the tasks that require my actual attention and skills. Depending on the role and the type of work, I've been able to streamline this process. I will setup a system where all of the prep work is done by someone else, Marketing for example, and the deliverable they provide slides right into the automation and out pops the solution. To me the development of these types of systems is time well spent. An added benefit of focusing on automation is that I can use the same skill in other aspects of my life.

I was once the treasurer for an association I was a part of. The majority of the job was taking the Excel spread sheet of financial data, breaking down the different pages and generating a human readable version that made the data easier to consume. The first time I did it, boy was that a pain to do. Gathering data from multiple pages, trying to explain what was going on while highlighting areas of interest. That first report took me a few hours and it was sup par.

Of course I was never going to do that process again. I created scripts for converting the Excel to CSV, chopping up the data, generating a graph with ranges that would automatically highlight when there were good and bad things to point out. At the end all I had to do was write a little blurb on the state of the union which was never much more than a few sentences. A shell script, R, Pandoc and I had a nice looking Word Doc that was repeatable and formatted in a very professional way.

A simple build system

I've been a ASM/C developer for a long time so I've written quite a few Makefiles. Because of that I can confidently say...Makefiles suck. If you're doing something simple, something where all the sources are treated the same way it's not that bad. But the moment you want one-off case its a huge pain. The syntax is not the greatest and unless you're writting Makefiles every day you'll be saying "oh crap, that is shell syntax not Makefile."

I was on the hunt for a build system that was simple and straight forward that also allowed me to increase complexity when needed. A monolithic shell script wasn't flexible, Makefiles allowed complexity but are sometimes difficult to grok. And don't get me started on build systems that require you to write XML. I wanted something as easy as a shell script but allowed for complexity and flexibility.

redo by djb (just the spec)

The concept of `redo` is pretty simple. You write scripts for given tasks with a specific naming convention and three arguments are given. There are a few helper methods and the system does all the rest. Oh and you can write the scripts is any language you want.

tl;dr the redo build system

Isn't this still complex?

The thing I like about Acidus' post is that he created a straight forward script that did exactly what it needed. The output didn't need to keep track of dependencies, and wouldn't benefit from the automation to know if intermediate steps can be skipped. When your problem can be solved in a very straight line, go the shortest path.

But a lot of the tedious tasks I tend to run into are those that aren't the exact same task over and over. Its always these slight one off cases. Generating monthly reports, bundling up files generated over a period of time that differ in structure over each backup. Its the nuance that makes things really painful. But over time I've come to realize that each project has a lot of common tasks. I love pandoc and I love writing documents in markdown and outputting them to docx and pdf. With `redo` having a straight forward markdown to docx Do File means I can just toss that into any project and bam it's done.

Here is a list of a few implementations.

apenwarr's Python implementation

Nils Dagsson Moskopp's shell implementation

Gyepi Sam's Go implementation

I actually keep a copy of apenwarr's sh based `do` script in most projects that I use redo just so that I can use it anywhere. Its doesn't support incremental builds but if I'm running on a system that doesn't support the above it just works.

apenwarr's sh do

-- CC-BY-4.0 jecxjo 2023-01-21

$ published: 2023-01-21 21:30 $

$ updated: 2023-01-23 18:12 $

back