Shell blindness

I wonder how many programmers realize just how weird shell scripting is compared to pretty much every other programming activity one might engage in. I hadn’t really thought on it too deeply before, but I started digging into shell more deeply several months ago. Previously, I had been reasoning about it as just sort of an archaic programming language, and that usually got me where I needed to go, though usually with some frustration along the way.

The thing is, shell scripting really only bears a passing resemblance to general purpose programming languages. It has control flow, variables and arrays, but that’s about it. You can’t compile it. It invokes system programs by design – anything longer than a few lines is probably non-deterministic. Any individual executed command’s output may change drastically at any moment (through updates, redirections, etc). As a result of that, you can’t reasonably model its behaviour; you can only test it. Like tapping commands into the prompt directly, it is interpreted line by line. The OS will change, bits and pieces will move around – the environment changes piecemeal.

Different shells have different, completely (or worse, subtly) incompatible syntax. There’s a standard that remains unimplemented (unimplementable? undesirable?) despite valiant efforts, and a smattering of documents and tools in the OS building community to help navigate this minefield.

Despite this, shell scripts are the glue of modern *nix operating systems, deeply embedded in bootstrapping and service management. The POSIX specification (and man pages of every shell) actually spells it out quite clearly: the shell is a command language interpreter.

It’s a little embarrassing to admit that, despite considering myself 50/50 on the developer/sysadmin front, I’ve spent most of my career only using enough shell to bootstrap myself into a higher level language. Sometimes this has certainly been necessary, but I suspect a percentage of it would have been better (or just as well) served by pure shell. I guess it’s fair to call it a sort of schlep blindness – shell blindness.

“The most dangerous thing about our dislike of schleps is that much of it is unconscious. Your unconscious won’t even let you see ideas that involve painful schleps. That’s schlep blindness.”

Wikipedia actually posits something about shell scripting which is interesting: “Shell scripts often serve as an initial stage in software development.” To people used to building applications with frameworks like rails, this sounds strange. In fact, I have encountered relatively few developers who either really understand, or enjoy working in, shell.

It’s interesting to think that there are probably a swathe of programs one could write in carefully crafted shell that would work, without dependencies, across a wide variety of unix-style operating systems, and be compatible with systems stretching decades into the past.

To that end, I wrote shlint (shell lint) to try and help me get a better grasp on how to write portable shell.

Github statistics – while not representative of the entire software development world, are certainly a reasonable representation of developers I’ve worked with – suggest that shell is actually the fifth most used language, 8% of code hosted on Github being shell. That is not inconsiderable – It’d be interesting to know how much of it is duplicated shell boilerplate/libraries of one sort or another.

Ryan Tomayko provides an excellent introduction to the world of shell programming in this talk. Definitely worth watching if you’re not someone who regularly writes shell.