POWERMAN
"In each of us sleeps a genius...
and his sleep gets deeper everyday."

Inferno OS shell for many years gives me only a very negative emotions. And I never realized why some people think Inferno sh is amazing. But as the saying, better late than never - today I decided to carefully learn it, and as a result I’m got insight - it is really an unique thing! Incredibly elegant and simple.

Still, I’ll start with drawbacks, to be more objective. Main - shell is too slow. No one knows why, but all inferno shells (there are more than one, but now I’m talking about /dis/sh.dis) are too slow. This is very strange, because usual speed of Limbo applications (with JIT) is between C speed and speed of fast scripting languages like Perl. Because of this there is no sense in writing real applications in sh. But write limbo app for every trivial micro-task is overkill too, so we have to use sh anyway for starting scripts and other similar things. The second major drawback - the inconvenience of using a text console, no command history, no auto-complete, no comfortable editing of command line is very frustrating (but I’ve just been told about the utility rlwrap, emu running through rlwrap -a seems able to solve this problem). The third - syntax of this shell is very unusual because it use unpaired quotes, and this break attempts to highlight syntax of inferno sh scripts using existing highlight implementations for any other shell, even for similar Plan9’s rc (because there is no existing syntax highlight implementation for inferno shell yet). That’s annoying, and I’ve planned to solve this issue today by implementing syntax highlight rules for Vim, that’s why I began carefully learning it… but as result instead of implementing syntax highlight rules I’m writing this article. :)

So, which building blocks we have for our scripts?

Functionality supported by /dis/sh.dis "out-of-box" is impressive! There is no even conditional operators and loops! No functions! So, what we have then, and how to live with that? You’ll see soon. Here is what we have:

  • execute apps (including pipes and i/o redirections)

  • strings, lists of strings, and command blocks

  • environment variables

  • file name patterns (*, ?, […])

  • and few builtin commands and substitution commands

Perhaps seeing the last point you think, "Aha! Some special commands. Certainly everything else is done by them, all the banal." … But no, your guess is wrong. There are only three builtin commands used often enough: exit, run and load; as for substitution commands it’s quote and unquote. The run just execute given script in current shell (same as . and source commands in bash).

But the load - yeah, this is the bomb! It let you load into the current shell additional modules, implemented in Limbo, which may add any functionality to shell - if, for, functions, exceptions, regular expressions, math operations, etc. There is also unload command, which let you dynamically unload these modules. :) However, even without a loadable modules shell is complete and functional - and to prove that I have implemented if and for in "bare sh" (you’ll see them at end of article)!

Oh, there is something else I forgot to mention. (Do you think now finally time for the "aha!"? Nope.) Shell also support comments. Which start with #. :)

Execute apps

Many things are similar to any *nix shell:

  • run apps (.dis files) and scripts (text files with shebang #! in first line)

  • commands separated by ;

  • run commands in background with &

  • pipes between commands using |

  • i/o redirections with >, >> and <

    ; echo one two | wc
        1       2       8

But there are also a lot of differences.

Advanced i/o redirections

  • define file descriptor number (to redirect stderr or opening new file descriptors in addition to default 0, 1 and 2)

    • cmd <stdin.txt >stdout.txt >[2]stderr.txt

    • cmd >[1=2] (redirect stdin to stderr)

    • cmd <[3]file.txt (command started with extra fd 3 opened for reading from file)

    • cmda |[2] cmdb (instead of piping cmda’s stdout pipe it’s stderr, while stdout will go on screen)

    • cmda |[1=2] cmdb (again, pipe cmda’s stderr, but connect it to cmdb’s stdout - opened for reading, of course, - instead of stdin)

  • open fd both for reading and writing

    • cmd <>in_pipe <>[1]out_pipe (stdin and stdout both opened for reading and writing but to different files)

  • non-linear pipes

    • cmd <{first;command;block} >{second;command;block} (both command blocks are run simultaneously with cmd, and cmd will get two params - file names like /fd/NUMBER, which are connected using pipes to stdout of first command block and stdin of second command block)

Last feature is most interesting. For example, standard command to compare two files cmp with help of this feature can compare output of two other commands instead of files: cmp <{ ls /dir1 } <{ ls /dir2 }.

$status

Inferno uses unusual way to implement process’s exit status. In traditional OSes any process return integer when exits: 0 if everything is ok, and non-zero if error happened. In Inferno any process either just exits or raises unhandled exception, which value is a string with error description.

There is an agreement by which error description should begin with the "fail:" if this error is just exit status - i.e. the application doesn’t crashed, but simply wants to return error to the process who started this application (usually the shell). If error description won’t begin with "fail:", then after application exit it’s process will be kept in memory (in "broken" state), to make possible investigating crash reason using debugging tools (same as core dumps in *nix, but instead of saving them to disk they are kept in memory).

So, after command exits it "exit status" (exception text) will be available in environment variable $status (with prefix "fail:" removed; if there was nothing after "fail:" in exception text then $status will be set to "failed").

Strings, lists of strings, command blocks

Inferno shell work only with strings and lists of strings.

String

String is either a word without spaces and some special symbols, or any symbols in single quotes. Only symbol which can be escaped in such a string - single quote itself, and it’s escaped by doubling it:

; echo 'quote   ''   <-- here'
quote   '   <-- here

There is no \n, \t, variable interpolation etc. in strings. If you need to add newline symbol inside a string - either just add it as is (by pressing Enter) or concatenate string with variable which contain this symbol. Inferno have app unicode(1) which can output any symbol by it’s code, so we can use it when need some special symbol in string (statement "{command} will be described later, but for now you can think it’s same as `command` in bash):

; cr="{unicode -t 0D}
; lf="{unicode -t 0A}
; tab="{unicode -t 09}
; echo -n 'a'$cr$lf'b'$tab'c' | xd -1x
0000000 61 0d 0a 62 09 63
0000006

Double quotes doesn’t used for strings (btw, paired double quotes in inferno shell doesn’t used at all).

Command blocks

Command blocks are surrounded by braces, and such blocks processed by shell simple as single string (which include surrounding braces).

; { echo one; echo two }
one
two
; '{ echo one; echo two }'
one
two

Only difference from usual strings in single quotes is: sh will check syntax in command blocks and slightly reformat it.

; echo { cmd | }
sh: stdin: parse error: syntax error
; echo { cmd | cmd }
{cmd|cmd}
; echo {
echo     one
echo two
}
{echo one;echo two}

Lists of strings

Lists of strings is just 0 or more strings separated by spaces. They can be surrounded by parens, but in most cases this is optional.

Any shell command is just list of strings, where first string in list is command name or command block, and all other strings are their params.

; echo one two three
one two three
; echo (one two) (three)
one two three
; (echo () (one (two three)))
one two three
; ({echo Hello, $1!} World)
Hello, World!

Operator ^ used to concatenate lists of strings. It can be applied to two lists with same amount of items (items will be concatenated by pair), or one of lists must contain just one item (which will be concatenated with every item of other list). As a special case, if both lists contain only one item - get the usual concatenation of two strings.

; echo (a b c) ^ (1 2 3)
a1 b2 c3
; echo (a b) ^ 1
a1 b1
; echo 1 ^ (a b)
1a 1b
; echo a ^ b
ab

Usually one shell command must fit in single line - there is no special symbol to define "command continuation on next line". But it’s ease to have multiline command - just open on previous line string (with single quote) or command block (with open brace) or list of strings (with open paren), or end previous line with list concatenation operator.

Environment variables

Variables in inferno shell looks deceptively similar to the traditional shells, but do not let that fool yourself!

; a = 'World'
; echo Hello, $a!
Hello, World!

/env

Environment variables in OS Inferno implemented different to traditional OSes. Instead of adding new POSIX API to work with environment variables and providing them to every process using scaring things like int execvpe(const char *file, char *const argv[], char *const envp[]); in Inferno they are just files in /env directory. It gives some interesting features, such as the application can provide access to its environment variables to other applications on the network - just export (using listen(1) and export(4)) it’s /env.

It’s possible to create/delete/modify environment variables by creating, removing and changing files in /env. But you should take in account shell is keeping in memory copy of all environment variables, thus changes you’ll do using files in /env will be seen by apps you start after that, but not by current shell. When you modify environment variables in shell (using operator =) files in /env will be updated too.

In shell, names of variables may contain any symbols, including ones which not allowed in file names (like ., .. and including /). Variables with such names will be available only in current shell, but started apps won’t see them.

Lists

Another important difference - all variables contain only lists of strings. You can’t set variable to string - it’s always will be list of single string.

To delete variable set it to empty list (either explicitly with a=() or just with a=).

  • $var is fetching list of strings from variable var, and there can be zero, one or many strings

  • $#var is fetching amount of items in $var list

  • $"var will concatenate (using single space as separator) all items in $var list into single string

    ; var = 'first   str'  second
    ; echo $var
    first   str second
    ; echo $#var
    2
    ; echo $"var
    first   str second

As you see, output of $var and $"var doesn’t differ, because echo output it’s params using single space as separator, and $" concatenate list items in same way. But first echo command received two params while last one received only one param.

Always where you implicitly concatenate value of variable with anything shell will inject explicit operator ^ (which, as you remember, work with lists, not strings). It is convenient, although while you not get used to it may be surprising:

; flags = a b c
; files = file1 file2
; echo -$flags $files.b
-a -b -c file1.b file2.b
; echo { echo -$flags $files.b }
{echo -^$flags $files^.b}

Assigning to list of variables works too. If right side of assignment contain more items than variables in left side - last variable will receive tail of list, while all previous variables will contain lists with only one item.

; list = a b c d
; (head tail) = $list
; echo $head
a
; echo $tail
b c d
; (x y) = (3 5)
; (x y) = ($y $x)
; echo 'x='^$x 'y='^$y
x=5 y=3

List of all params for script or code block always available in variable $*, and each single param available in $1, $2, ….

Scope

Every command block define new local scope. You can set values to variables using operators = and :=. First will change value of existing variable or create new variable in current scope. Second always create new variable, hiding previous value of same-named variable (if any) until end of current block.

; a = 1
; { a = 2 ; echo $a }
2
; echo $a
2
; { a := 3; echo $a }
3
; echo $a
2
; { b := 4; echo $b }
4
; echo b is $b
b is

References to variables

Name of variable is just a string after symbol $. And it can be anything which result in a string - like string in single quotes or another variable.

; 'var' = 10
; echo $var
10
; ref = 'var'
; echo $$ref
10
; echo $'var'
10
; echo $('va' ^ r)
10

Substituting command output

Actually, this topic is related to "execute apps" section, but it was necessary to describe lists of strings first, so I had to postpone it until now.

So, meet the unpaired quotes I mentioned before, which all of time break syntax highlight.

  • `{command} (execute command, and return it’s output to stdout as list of strings; output split into list using symbols found in variable $ifs; if that variable not defined then three symbols are used: space, tab and newline)

  • "{command} (execute command, and return it’s output to stdout as single string - by concatenating all lines of output)

As example let’s see how we can load file list using ls. Keeping in mind file names may contain spaces default behaviour of `{} doesn’t suitable for us - we’ve to split ls output only by newline.

; ifs := "{unicode -t 0A}
; files := `{ ls / }
; echo $#files
82
; ls / | wc -l
     82

Actually, this can be done easier with file name patterns, which expanded by shell into list of file names:

; files2 := /*
; echo $#files2
82

Builtin commands

There are two types of builtin commands: normal commands and substitution commands.

Normal commands executed just like usual apps:

; run /lib/sh/profile
; load std
; unload std
; exit

Substitution commands executed using ${command params} and they return (not output to stdout, but return - just like when you access variable value) usual list of strings - i.e. they should be used in params for commands or in right side of assignment operator. For example, command ${quote} escape given list of strings into single string, while ${unquote} do the reverse operation.

; list = 'a  b'  c d
; echo $list
a  b c d
; echo ${quote $list}
'a  b' c d
; echo ${unquote ${quote $list}}
a  b c d

Implementing our own if, for and functions

As I promised, now I’ll show how to implement these very useful things in "bare sh". Of course, you don’t have to do this in real world, loadable module std will give you all you need, and it’s much more useful. But, nevertheless, this implementation is of interest to demonstrate the possibilities of the "bare sh".

Everything is implemented using just:

  • variables

  • strings and lists of strings

  • code blocks and their params

Let’s implement "functions"

As I mentioned, any shell command is simply a list of strings, where the first string is the command name and the rest strings are it parameters. A command block is simply a string, and you can save it in a variable. And any command block receives parameters in $*, $1, etc.

; hello = { echo Hello, $1! }
; $hello World
Hello, World!

Moreover, we can even do function curring just like in functional programming. :)

; greet = {echo $1, $2!}
; hi    = $greet Hi
; hello = $greet Hello
; $hi World
Hi, World!
; $hello World
Hello, World!

Another example - we can use command block params to get list item by it’s number:

; list = a b c d e f
; { echo $3 } $list
c

Let’s implement "for"

I havn’t tried to implement full-featured and easy to use if, I wanted to implement for and thus implemented minimalistic if because it was needed for for (loop must stop at some point, and it’s hard to do this without conditional operator).

; do = { $* }
; dont = { }
; if = {
    (cmd cond) := $*
    { $2 $cmd } $cond $do $dont
}
; for = {
    (var in list) := $*
    code := $$#*
    $iter $var $code $list
}
; iter = {
    (var code list) := $*
    (cur list) := $list
    (next unused) := $list
    $if {$var=$cur; $code; $iter $var $code $list} $next
}
; $for i in 10 20 30 { echo i is $i }
i is 10
i is 20
i is 30
;

Misc

By default shell fork namespace when start new script, thus that script won’t be able to change parent’s namespace. This isn’t suitable for scripts which goal exactly is change parent’s namespace. Such scripts must start with #!/dis/sh -n.

Builtin command loaded will output list of all builtin commands from all loaded modules. Builtin command whatis output information about variables, functions, commands, etc.

If you create variable $autoload with list of shell module names, then these modules will be automatically loaded by all starting shells.

Shell have support for syntactic sugar: && and ||. These operators provided by "bare sh", but they converted to commands and and or which are absent from "bare sh" - they are from module std (so to use && and || you should load std first).

; echo { cmd && cmd }
{and {cmd} {cmd}}
; echo { cmd || cmd }
{or {cmd} {cmd}}

Limbo apps may receive command line param with shell code block and can easily execute it using sh(2) module.

Way how Inferno shell handle variables and lists of strings is making your life much easier, because you need to escape everything only once and then you don’t have to think about how to escape same things once again when you use these values somewhere.

Resume

This small article is nearly complete reference guide for Inferno shell. I mean, it describe all functionality of shell, and with all details and examples. If you’ll read sh(1) you’ll see what I didn’t mention very few things: variables $apid, $prompt, few builtin commands, sh options and full list of special symbols which can’t be used in strings without single quotes.

If we forget for one second about advanced i/o redirection features, then using just:

  • strings with trivial escaping rules

  • command blocks (which are actually usual strings)

  • lists of strings with one operator ^

  • variables with one operator =

  • access to variables using $var, $#var and $"var

we have full-featured shell! It’s truly full-featured even in "bare sh" variant, what was just proven by implementing functions, if and for using only these simple features listed above (btw, I’ve also find a way to implement exceptions - like /bin/false :) - but it’s a hack using run and I didn’t included it in this article).

Next, when we begin loading modules into shell, its features and usability increases in order of magnitude! For example, here is what we get from sh-std(1) module:

  • several conditional commands (and, or, if)

  • several commands to test conditions (!, ~, no)

  • several loop commands (apply, for, while, getlines)

  • commands to define functions of both types (fn, subfn)

  • commands to work with exceptions and exit status (raise, rescue, status)

  • substitution commands to work with lists and strings (${hd}, ${tl}, ${index}, ${split}, ${join})

  • etc.

But all these additional commands doesn’t make sh syntax even a bit more complicated, it’s still very trivial and elegant!