The Command Line

A Motivating Example

Quick, create six folders to hold songs for these six artists, using your “Finder” or “Explorer”.

How long did that take?

Here’s how you do it on the Command Line:

mkdir 'Charli XCX' Halsey 'The Weeknd' Khalid 'Lady Gaga' Beyoncé

Note that sometimes quotes are needed. See why?

Why is the Command Line So Cool?

People that often use the command line are sometimes called power users. Why is the command line powerful?

• Hands stay on the keyboard: no physical “context switches” to a mouse or trackpad (so it’s often faster)
• Text is really dense: lots of information in (very) few characters
• You’re not limited to what’s on the menus
• It might be the only way to control a remote computer

You can do almost anything from the command line. You don’t need to launch many separate window-based applications. With the command line, you can:

• Create, delete, rearrange, edit, and search files (without using the finder)
• Start, stop, and inspect your running programs (without a task manager)
• Get information such as the current time, how much disk space is left on your system, etc.
• Connect to, control, and copy files between, devices and remote computers (without a fancy graphical tool)

These notes do not cover native Windows commands

We are only covering the command language of Bash, which is native to the Mac and the most popular shell on Unix systems. If you are using Windows, you can install Git Bash or other Bash-enabled terminal, no problem! The basic ideas of command line usage, though not the actual commands and shortcuts themselves, will transfer over just fine to native Windows command shells such as cmd and PowerShell.

The Basics

Your command line application carries out a dialog between you and the user. At a prompt, you type in a command and hit Enter (usually). The command executes, perhaps with a response from the system, and then you get a new prompt. Here is a sample session. In this example, the prompt is $, but on your system it might be different:

$ date
Mon Jan 20 17:40:16 PST 2020
$ echo Hello     how  are you
Hello how are you
$ grep qi /usr/share/dict/words 
Iraqi
Iraqian
qintar
Saqib
$ host www.google.com
www.google.com has address 172.217.11.68
www.google.com has IPv6 address 2607:f8b0:4007:802::2004
$

Nice! We’ve just seen four commands:

• date, which prints what the computer thinks to be the current datetime.
• echo, which echos back what you type (though not always exactly).
• grep, which finds things in files. Here we looked for the character sequence “qi” in a file of words (sorry, this file is not on Windows), and four words were found. (More about files coming up soon.)
• host, which looks up IP addresses for domain names (okay sorry for the network lingo, but hey that was faster than firing up graphical tool, right? Oh also, Git Bash doesn’t have the host command.)

The set of available commands differs between systems, but there’s a couple hundred that are common.

• Pretty good list of all macOS commands
• Pretty good list of all typical Unix commands
• In Git Bash, use the command ls -1F /bin | grep '\*$' | grep -v '\.dll\*$' | sed 's/\*$\|\.exe\*$//g'

Commands are super flexible. They have arguments and options. Try:

ls
ls /bin
ls -l /bin
ls -lFG /bin
ls -lG

The arguments and options for each command are listed and explained in great detail in something called the command’s manpage.

The File System

We’re going to learn how to navigate and manage files using some cool commands and a bunch of powerful keyboard shortcuts, through an in-class exercise of making a travel photo library. But first, there are things about the computer’s file system we just need to know.

About the file system

A few brief notes before we get started:

• A computer’s file system has a root folder which contains files and other folders. Each folder contains files and other folders. The name of the root folder is /. Here’s an example:

  

• The file system can also be viewed as a tree. The example above looks like this:

    /
    ├── Library
    │   ├── Fonts
    │   │   ├── Farisi.ttf
    │   │   ├── GillSans.ttf
    │   │   └── Sana.ttc
    │   └── Keychains
    │       └── System.keychain
    ├── Users
    │   ├── jessica
    │   │   ├── homework
    │   │   │   ├── hw1.py
    │   │   │   └── hw2.txt
    │   │   └── resume.pdf
    │   └── kimiko
    │       ├── IMG0001.jpg
    │       └── IMG0002.jpg
    └── bin
        ├── date
        ├── hostname
        ├── kill
        └── sleep
  

• The absolute pathname of a file or folder “starts at” the root folder, and works its way down, for example:

      /bin
      /bin/kill
      /Users/jessica/resume.pdf
      /Users/jessica/homework/hw1.py
      /Library/Fonts/Farisi.ttf
      /Library/Keychains
  

• When using the command line, there is always a current folder, also known as the working directory that you are “in” at the moment.
• Files and folders always have a relative pathname, whose name starts not from the root, but from the current folder. For example, if the current folder is /Users/jessica, then the following are relative pathnames for other folders and files:

      resume.pdf
      homework
      homework/hw1.py
      ..                                  (parent folder)
      ../kimiko
      ../kimiko/IMG0001.jpg
      ../..
      ../../bin
      ../../bin/hostname
      ../../Library
      ../../Library/Fonts/Sana.ttc
      .                                   (current folder)
      ./resume.pdf                        (pretty cool, right?)
  

• Every logged in user has a home folder, or home directory. Its name is ~. You can use this to make pathnames, too. If your home folder was /Users/kimiko, then you can always refer to your picture 1 as ~/IMG0001.jpg, regardless of the current folder.
• File names with spaces are okay, but they need to be quoted or have their spaces prefixed with a backslash. When typing filenames in commands, the following names are equivalent:

      Tokyo\ Tower.jpg
      "Tokyo Tower.jpg"
  

“Folder” or “Directory”?

Doesn’t matter. Same thing. You can say either.

An in-class exercise

We will make a travel photo library, together in class. (For homework, you will clean up and build out a movie library, starting with a very messed up an incomplete set of starter files.)

In this exercise we will learn:

• How to start the terminal application
• How to create folders and files (mkdir, touch)
• How to list the contents of a folder (ls, find, tree)
• How to navigate within the file system (pwd, cd)
• How to move, copy, and delete folders and files (mv, cp, rm, rmdir)
• Some incredible and cool keyboard shortcuts TAB ⌃A ⌃E ⌥← ⌥→ ⌃L ⌘K ⌃R

Can’t find the exercise content?

We’ll be doing it in class, that’s why! Take good notes.

Special Characters

Not every character is legal in a filename; some have special meaning within Bash. If you really, really must put them in a filename, you probably can, but in order to use the file, you’ll need quotes around the file name, and even then there’s no guarantee things will work (at least not without some serious worry.) Here are some good things to know:

1. If at all possible, stick to letters, digits, dashes, dots, and underscores only! Try to avoid anything else.
2. NEVER use /, as that separates path components.
3. NEVER use quotes, as these are the way you save yourself from special characters.
4. TRY TO AVOID spaces, as they can get irritating, though just quote them if you see them.
5. NEVER use braces ({ and }) because these get expanded.
6. NEVER use asterisks, question marks, or square brackets (* and ? and [ and ]) because these get expanded.
7. NEVER use &, |, <, > because these are used to combine commands and do redirections (whatever those are...).
8. This list of characters to avoid or never use is not complete. Just remember the first rule.

What does “get expanded” mean?

Braces and the asterisk are used to generate text! It might appear complicated, but really it’s pretty amazing. Braces make arbitrary text. This is called brace expansionHere are examples:

$ echo {b,r,spl,ch}at
bat rat splat chat
$ echo {2..16}
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
$ echo b{a,e,i,o,u}t
bat bet bit bot but

The asterisk, question mark, and square brackets participate in filename expansion: they generate filenames only (which must exist). Though the actual rules are complicated, * matches any string, ? matches a single character, and [ ]matches a single character from the enclosed pattern, and its pattern language can be quite powerful. We won’t get into the details, but will just look at examples:

$ mkdir scratch
$ cd scratch/
$ touch dog.txt rat.jpg bat.txt animals personal.html
$ ls
animals        bat.txt        dog.txt        personal.html  rat.jpg
$ ls ?at*
bat.txt  rat.jpg
$ ls *.txt
bat.txt  dog.txt
$ ls *o*
dog.txt        personal.html
$ ls *[sgv]
animals  rat.jpg

Summary of file commands

We learned these in our in-class exercise.

Other Commands

You don’t use the command line only for managing your files. There are tons of other things to do, from starting and stopping programs, to networking, to controlling devices. Please browse this awesome online reference of commands. It covers Linux, but there is a Mac Version too. Most of these commands work in Git Bash too. You can also find a list at Wikipedia.

Some of the cooler commands to learn are:

• date
• cal
• file
• cat
• sort
• uniq
• tail
• less
• zip
• unzip
• curl
• which
• sudo

If you are into details, details, details, see The Bash Reference Manual.

Keyboard Shortcuts

To use the command line like a pro, and save tons of time, you’ll be happy knowing at least of few keyboard shortcuts. Some of the more useful ones are:

But there are dozens more! Here is a great list of keyboard shortcuts. You can also, if you have tons of time and really, really want to know the details, read the section on command line editing in the Bash Reference Manual.

Operating on the Commands

These operations on commands should be a part of any command line users’ knowledge. They can be learned over time.

command1 | command2

(The pipe) Send the output of command1 into the input of command2.

command1 && command2

(AND) Execute command1, then ONLY if command1 succeeds, proceed to command2.

command1 || command2

(OR) Execute command1, then ONLY if command1 fails, proceed to command2.

command1 ; command2

(SEQUENCE) Execute command1, then execute command2.

command1 < file

(REDIRECT STANDARD INPUT) Read input not from the keyboard, but from the given file.

command1 > file

(REDIRECT STANDARD OUTPUT) Output not to the terminal, but to the given file.

command1 >> file

(REDIRECT AND APPEND TO STANDARD OUTPUT) Output not to the terminal, but append it to the given file.

command &

(BACKGROUND) Execute command as a background task. Do this when the command would take a long time to run, or is supposed to run in the background. After launching a command this way, control of the terminal is immediately returned to you. The program you launched will show up in your system list of processes (assuming it hasn’t finished or been killed).

Examples (meant to be executed in order):

• Create a new file called hello.py with a little Python program in it:

  $ echo 'print("Hello")' > hello.py
  

• Add a new line to hello.py:

  $ echo 'print("World")' >> hello.py
  

• Uppercase the file hello.py

  $ tr '[a-z]' '[A-Z]' < hello.py
  

• Compile the Java program Greeter.java, then ONLY if the compilation proceeds, move on to running it.

  $ javac Greeter.java && java Greeter
  

• Run the tree program, but if it fails (including by not existing), run find

  $ tree || find .
  

• List a few of the recent git commands you ran:

  $ history | grep 'git '
  

• Download the text of the book Emma to a new file called emma in the current directory:

  $ curl http://www.gutenberg.org/files/158/158-0.txt > emma
  

• Find the 25 most frequently appearing words in the book, printing them together with their number of occurrences, in descending order:

$ cat emma | tr '[A-Z]' '[a-z]' | sed -E $'s/[^a-z]+/\\\n/g' | grep [a-z] | sort | uniq -c | sort -rnk 1 | head -n 25

More!

Don’t be afraid to get too “advanced.” Instead embrace what can be done. Check out what you have time for, and ignore, for now, what seems a little arcane. There will be time. When you’re ready, here are some good places to go for details, tips and tricks, and fun times:

• The Bash Reference Manual
• Wikipedia article on Bash
• Bash Cheat Sheet
• macOS Command Reference
• Command Line Fu
• A cool presentation
• Terminus Game
• Cmd Challenge

Try this command:

$ nc towel.blinkenlights.nl 23

Bonus Material: Built-In Folders

You’ll notice a lot of existing folders and files, even on a brand-new machine. A number of these folders are “standard.” The standard folders are operating system dependent. For fun, here are the ones you’ll see in most Linux-based systems (based on this thing called the FHS):

Review

Make sure you can discuss or explain each of the following:

• Why is the command line useful?
• mkdir, cd, pwd
• touch
• ls, find
• mv, cp
• rm, rmdir
• cat
• open
• Relative vs Absolute paths
• .
• ..
• ~
• Autocomplete (TAB)
• History navigation (Up and down arrows)
• Line navigation (Ctrl-A, Ctrl-E, Option-arrows)
• Clearing (Ctrl-L, Option-K)
• | (The pipe)
• &&

Summary