2022-03-20 - the ax shell proposal

a newer version of this document is available here:

i love shell programming, but i don't love the languages that are currently out there. sh doesn't have a modern feel and is pretty limited. bash is powerful, but has so many complex rules and gotchas that i still find myself referencing the man page after scripting for almost 10 years. zsh is pretty much the same as bash, but with more features. i've read up on the "nu" shell, while it's very different than existing shells, it introduces complexity in very different ways that i can't agree with. i like the direction the fish shell takes, but i still think a simpler solution can be achieved.

with that said, over the past few weeks i've come up with the basic design for a new shell called "ax". while the design is by no means complete, final or without inconsistencies, i do like the general direction it's taking. this project is something i could see myself working on a year down the road after getting feedback and giving it more thought. the remainder of this article will describe the current design, syntax, and reasonings for the ax shell.

the ax shell

the ax shell (pronounced "axe" or "acks") is an acronym for "[a]nother e[x]cellent [shell]". this shell strives to preserve the unix philosophy, while straying from normal posix shell standards. when i say "unix philosophy", the definition i'm referring to is from doug mcilroy:

"this is the unix philosophy: write programs that do one thing and do it well. write programs to work together. write programs to handle text streams, because that is a universal interface."

a few of the design highlights of the ax shell are:

executables extend functionality, not shell specific builtins, functions, and aliases

text is the only datatype, arrays are actually just text with a null character

simple syntax to allow for easy memorization

unix philosophy on wikipedia

posix reference

a note on the null character

unix doesn't allow the null character to be a part of filenames, env var names or values, or command arguments. but the null character can be part of the stdout of a program or a file's content. the ax shell abuses this small detail to it's advantage. the null character is essential in how arrays work in the ax shell. for example, ax environment variables allow the null character in it's value, but when you convert it to an argument or when exported to a subprocess, only the string up to the first null character is sent to the command or subprocess. also, spaces in file names no longer become an issue when using command substitutions if you are using the null character as a delimiter. i don't think i could come up with an elegant i liked for the ax shell if null characters didn't have this interesting guarantee.

anyways, let's continue with the specification.

reserved characters

here is a list of all reserved characters. there is no escaping supported. instead of escaping, you should use one of the many string options.

builtins, variables, pipes, statements, comments:
!  -- builtin
$  -- create var/array
|  -- pipe stdout
;  -- separate statements
#  -- rest of line is comment ignore null character

parameter strings:
%  -- rest of line is string (ignore null character)
'' -- string (ignore null character)
"" -- string (ignore null character)
`` -- string (ignore null character)
^  -- here doc string (ignore null character)

command substitutions:
*  -- transform one of the captures below to an array
[] -- exec & capture stdout (ignore stderr)
() -- exec & capture return codes (ignore stdout & stderr)
{} -- capture variable

builtins

builtins must always be prepended with a "!". it's easy to tell whether something is an executable or just a builtin with this required prefix. unlike normal commands, builtins can conditionally parse command substitutions. this allows for if and loop constructs to be represented as builtins. there may be more builtins in the future, but the list of builtins cannot be extended by ax shell users.

!out    [arg]...            -- print args to stdout, separating with null character
!cd     [dir]               -- change into a directory
!exit   [err]               -- leave the shell, returning optional error code
!if     [test expr]... expr -- conditional, then is printed to stdout
!loop   [test expr]... expr -- conditional loop, thens are printed to stdout

env vars

the ax shell doesn't distinguish exported variables with local variables. only exported environment variables are supported. here are a few of the environment variables that the ax shell populates for you:

{AX_BIN}    -- the temporary directory that is prepended to the PATH variable

examples

as of now, that's the complete syntax for the ax shell. though the list of builtins is not complete and the syntax is still subject to change, the shell is already very powerful as these examples will show.

# cd into /tmp/dir:
!cd /tmp/dir

# print the text "hello world" with no trailing newline:
!out 'hello world'

# or use an external program to do the same thing:
echo -n 'hello world'

# instead of aliases or functions, add to the path like this:
!out "#!/bin/ax" "!out {2}" | tr \0 \n | tee {AX_BIN}/print-second

# here is how you could make a shortcut for the above alias snippet using standard posix shell commands:
!out ^EOF
    #!/bin/ax
    $AX_BIN [!out {PATH} | awk -F: '{print $2}' | tr -d \n]
    !out "#!/bin/ax" "!out {2}" | tr \0 \n | tee {AX_BIN}/{1}
    !exit 0
EOF | sed -E s/^ +//g | tee {AX_BIN}/alias

# and using the above shortcut, you could end up creating aliases like this:
alias print-second %!out {2}

# if we had a few imaginary shell commands available, you can imagine the alias shortcut to be a bit more elegant:
!out ^EOF
    #!/bin/ax
    $AX_BIN [range 2 *[!out {PATH} | tr : \0]]
    !out "#!/bin/ax" "!out {2}" | tr \0 \n | tee {AX_BIN}/{1}
EOF | trim | tee {AX_BIN}/alias

# here is what a conditional might look like. test is not a builtin, it's a standard unix command:
!if (test -n {VAR}) [echo var is non-empty] [echo var is empty]

# in this example, the program will sleep for 1 second until the variable is set:
!loop (test -z {VAR}) [sleep 1s]

# here is an if elseif statement:
!if (test -n {V1}) [echo V1 is non-empty]
    (test -n {V2}) [echo V2 is non-empty but V1 is empty]
                   [echo both V1 and V2 are empty]

# there is no globbing in ax. instead of "ls */*" like in bash, use a command to get the argument list:
ls *[find -print0 -mindepth 2 -maxdepth 2 -not -path '*/.*']

# or if you have "fd", the above syntax is much more sane to work with:
ls *[fd -0I --exact-depth 2]

conclusion

some things i can think of right now that still need to be hashed out are:

process handling

what to do with standard error

finalizing the list of builtins

i need to spend some time today doing something else rather than working on this blogpost now. you can see that even with a basic syntax, a powerful shell can be constructed. email me at alan@xoc3.io if you have questions or feedback about this. i'll start prototyping this shell when i consider the design to be complete, which will probably not be for many months at least.

-- Response ended

-- Page fetched on Fri May 10 20:02:21 2024