Tools to know when on a unix command line

When I was a consultant (oh, so long ago), it was critical to have knowledge of various command-line unix tools. The main reason for this was that the administrators of the machine would only grant ssh access to someone like a consultant. In addition to this, you were not allowed to install any software either. This situation caused me to become more fluent in various command-line tools that I normally would not have used.

  1. vi – Typically, emacs is not installed on a corporate unix machine unless it is a lowly development box. So, when on a box using ssh, the main editor availabe is vi. Vi is actually quite powerful once you get used to it. Several years of consulting later, and I now install gvim on any PC that I own or work on.
  2. perl – Thankfully, perl is a standard install on any unix box. This is critical for any batch oriented work that is more complex than you want to do in a shell. In this case, I am talking about using perl for mostly text processing or various scripting activities. I am not talking about the newer object oriented perl stuff available since the 5.0 days.
  3. ksh and sh – I am lumping these two shells together as they are normally readily available and have minor differences. If you learn how to script in either shell, you can do some amazing things. This is especially true if you start hooking in things like perl and some of the other tools in this list. The reason both shells are listed is that some “strict” administrators feel that ksh has too many bugs to install on their machine.
  4. sed (and sometimes awk) – sed is fantastic for basic search and replace editing functionality. Given that you do not have a GUI editor available, sed can do some quick text transformations very quickly for a large number of files. It is also handy if you have some “template” scripts where you use sed to customize the script before actually running it. Awk is useful in some very limited circumstances if you know perl. If you don’t know perl, awk is very good at processing “structured” files in a batch mode.
  5. ftp – ftp is a fundamental tool on a unix box. I am including this not because you need ftp itself when on the unix box, but learning the standard tools and their command-line options is critical. For example, what are the command-line options for ftp that turn off basic prompting as well as prompting for a username? Well, -i is used to turn off basic prompting and -n turns off prompting for the username and password. Once you have those options, you can script ftp using the USER command at the ftp prompt. This allows you to write production ready scripts that transfer files between various production systems.
  6. sudo – I won’t go into the possible security issues with something like sudo, but it allows you to run commands as the root user if you have the proper permissions. This is a little harder to get access to, but in a development environment it can be extremely useful.
  7. screen – I have to thank a DBA I used to work (yes Shawn, I am talking about you) with for introducing me to this. Similar to nohup, screen allows you to run a command and come back to it later. The power of screen is that it keeps the entire session open, and you can switch to it like it is just another application or session. This beats the hell out of trying to get the syntax of “nohup mycommand > mycommand.log 2>&1 &” correct.
  8. wget – If you are working on web development or some sort of server listener, wget allows you to retrieve a URL from the command-line. You do have to jump through some hoops to get this to do what you want, but it can be extremely useful.
  9. grep – If you know you have a file that has some specific text in it, but you do not know which file it is, grep is the tool. Because grep also uses regular expressions, you can find almost anything in a text file. If you pipe (|) this through various other commands you can do some amazing things.
  10. find – I continuously lose my place when playing admin on a unix box. Is a tool installed in /usr/sbin or one of the share directories, or somewhere else? Find is the standard searching tool on unix. The only problem with it is a slightly arcane set of command line options and syntax. I still can not get the syntax of find correct in certain situations.
  11. cut – Cut is extremely useful if you have a text file in a known format (delimited or fixed-width). Cut allows you to break the text file into little pieces and process it later. I have used cut in various places where I get a delimited file and need to parse the data out of it. If you do not have the patience to learn awk, cut is an excellent replacement.

So, I know two of these tools are not as useful as some of the others (sh and awk), but they will become invaluable if you are in the wrong situation. I know this list isn’t all encompassing, and I am sure that I missed something important. So, just dump some info down in the comments and maybe I will change the list if there is something I would agree with.