Web Excursions 2022-05-11
Unix Command Line Conventions Over Time
Early 1970s
In the beginning, in the first year or so of Unix, an ideal was formed for what a Unix program would be like:
it would be given some number of filenames as command line arguments, and it would read those.
If no filenames were given, it would read the standard input.
It would write its output to the standard output.
There might be a small number of other, fixed, command line arguments.
Options didn’t exist.
This allowed programs to be easily combined: one program’s output could be the input of another.
There were, of course, variations.
The
echo
command didn’t read anything.The
cp
,mv
, andrm
commands didn’t output anything.However, the “filter” was the ideal.
Options
Fairly quickly, the developers of Unix found that many programs would be more useful if the user could choose between minor variations of function
The command line option was added.
This seems to have resulted in a bit of a philosophical discussion among the developers.
Some were adamant against options, fearing the complexity it would bring
To make command line parsing easy to implement, options always started with a single dash, and consisted of a single character.
Multiple options could be packed after one dash, so that
foo -a -b -c
could be shortened tofoo -abc
.
If not immediately, then soon after, an additional twist was added: some options required a value.
The syntax for values was a little complicated: the value could follow the option letter as part of the same command line argument, or be the next argument.
At this point, command line parsing became more than just iterating over the command line arguments.
it was now the case that one often needed to check the manual, or experiment, to find out how a specific program was to be used.
Later on, Wikipedia says 1980, the C library function getopt was written.
It became part of the Unix C standard library.
It implemented the command line parsing described above.
It was written in C, which at that time was quite a primitive programming language, and this resulted in a simplistic API.
Part of that API is that if the user used an unknown option on the command line, the
getopt
function would return a question mark (?
) as its value.Some programs would respond by writing out a short usage blurb.
This led to
-?
being sometimes used to tell a program to show a help text.
Long options
During the 1980s some changes to command line syntax happened.
The biggest change here was long options: options whose name wasn’t just a single character.
the single dash clashed with the “clumping together” of single character option.
A further complication to parsing the command line was that single-dash long options that took values couldn’t allow the value to be part of the same command line argument.
because a simple C command line parser would have difficulty figuring out what was the option name and what was the option’s value.
The GNU project
The GNU project was first announced in 1983.
GNU introduced another long option syntax, I believe to disambiguate
Initially, GNU used the plus (
+
) to indicate a long option, but quickly changed to a double dash (--
).I believe it was also GNU that introduced using the equals sign (
=
) to optionally add a value to a long option.GNU further allowed options to occur anywhere on the command line, not just at the beginning.
This made things more convenient to the user.
wrote a C function,
getopt_long
, to unify command line parsing across the software produced by the project.The GNU changes have largely been adopted by other Unix variants.
GNU also added standard options: almost every GNU program supports the options
--help
,--version
, and--mail=ADDR
Double dash
Apparently the double-dash was supported already in about 1980 in the first version of getopt in Unix System III.
Around this time, a further convention was added: an argument of two dashes only
(--)
as a way to say that no further options to the command being invoked would follow.I believe this was another GNU change, but I have no evidence.
The double dash is more useful for other situations, such as when invoking a program that invokes another program.
[e.g., in]
$ cargo run -- --version
, without the double dash, you would be telling cargo to report its version.
around the late 1980s, subcommands were a response to many Unix programs gaining a large number of “options” that were in fact not optional at all, and were really commands.
the user was required to use one of them, but not both.
I believe the oldest program that uses subcommand is the version control system SCCS, from 1972,
but I haven’t been able to find out which version added subcommands.
Another version control system, CVS, from 1990, seems to have had them the beginning.
Later version control systems, such as Subversion, Arch, and Git, follow the subcommand pattern.
Version control systems seem to inherently require the user to do a number of distinct operations,
which fits the subcommand style well,
and also avoids adding large numbers of individual programs (commands) to the shell, reducing name collisions.
The main command may have options (often called “global options”), but so can subcommands.
When options can occur anywhere on the command line, is
--version
a global option, or specific to a subcommand?
To solve this, some programs require global options to be before the subcommand, which is easy to implement.
Others allow them anywhere.
Everything seems to require per-subcommand options to come after the subcommand.
If we were re-designing Unix from scratch, and didn’t need to be backwards compatible, we could introduce a completely new syntax that is systematic, easy to remember, easy to use, and easy to implement.
Announcement: I’m Going to Miss You, But I Am Taking a Sabbatical
I’ve been writing here for more than 24 years, nearly half my life — I need a breather.
With the stress of the separation, my new living situation, and not seeing my kids every day, I felt a little like I was dying too.
Recently however, my fiddle leaf fig has been struggling again. - My plant is not ok.
And neither am I — I feel as off-balance as my tree looks.
I’m burrrrned out.
I appreciate so much what I’ve built here at kottke.org —
I get to read and learn about all sorts of new things every day, create new ideas and connections for people, and think in public —
and I feel incredibly lucky to be able to set my own schedule, be my own boss, and provide for my family.
But if you were to go back into the archive for the past several months and read the site closely, you’d see that I’ve been struggling.
I’ve tried thinking about these and many other questions while continuing my work here, but I haven’t made much progress; I need time away to gain perspective.
Does what I do here make a difference in other people’s lives? In my life?
Is this still scratching the creative itch that it used to?
And if not, what needs to change?
Where does kottke.org end and Jason begin?
Who am I without my work?
Is the validation I get from the site healthy?
Is having to be active on social media healthy?
Is having to read the horrible news every day healthy?
What else could I be doing here?
What could I be doing somewhere else?
What good is a blog without a thriving community of other blogs?
The plan, as it currently stands, is to take 5-6 months away from the site.
I will not be posting anything new here.
I won’t be publishing the newsletter.
The goal of my time away from the site is resting, resetting, recharging, and figuring out what to do going forward.
“I need some space to think and live and have generative conversations and do things, and then I’ll make something, but I can’t tell you what it is just yet.”
That’s the sort of energy I need to tap into for a few months.
There’s a passenger ferry that goes from Cape Cod to Nantucket
and there’s a stretch of time in the middle of the journey where you can’t see the mainland behind you and can’t yet see the island ahead — you’re just out in the open water.
That’s what I need, to be in that middle part —
to forget about what I’ve been doing here for so many years without having to think about where I’m going in the future.
I need open water and 5-6 months feels like the right amount of time to find it.
This is probably a good time to admit that I’m a little terrified about taking this time off.
There’s no real roadmap for this, no blueprint for independent creators taking sabbaticals to recharge.
The US doesn’t have the social safety net necessary to enable extended breaks from work (or much of anything else, including health care) for people with Weird Internet Careers.
I support a lot of individual writers, artists, YouTubers, and bloggers through Substack, Patreon, and other channels,
and over the years I’ve seen some of them produce content at a furious pace to keep up their momentum,
only to burn out and quit doing the projects that I, and loads of other people, loved.
With so many more people pursuing independent work funded directly by readers & viewers these days, this is something all of us, creators and supporters alike, are going to have to think about.