The Command Line

A Motivating Example

Quick, create six folders to hold songs for these six artists, using your “Finder” or “Explorer”.

How long did that take?

Here’s how you do it on the Command Line:

mkdir 'Charli XCX' Halsey 'The Weeknd' Khalid 'Lady Gaga' Beyoncé

Note that sometimes quotes are needed. See why?

Why is the Command Line So Cool?

People that often use the command line are sometimes called power users. Why is the command line powerful?

• Hands stay on the keyboard: no physical “context switches” to a mouse or trackpad (so it’s often faster)
• Text is really dense: lots of information in (very) few characters
• You’re not limited to what’s on the menus
• It might be the only way to control a remote computer
• You don’t need thousands of separate window-based apps to clutter your workspace

On the command line you can easily:

• Create, delete, rearrange, edit, and search files (without using the finder)
• Start, stop, and inspect your running programs (without a task manager)
• Get information such as the current time, how much disk space is left on your system, etc.
• Connect to, control, and copy files between, devices and remote computers (without a fancy graphical tool)

These notes do not cover native Windows commands

We are only covering the command language of Bash, which is available on Macs and on Unix systems. (We’ll discuss the more modern Zsh toward the end of these notes.) If you are using Windows, you can install Git Bash or other Bash-enabled terminal, no problem! The basic ideas of command line usage, though not the actual commands and shortcuts themselves, will transfer over just fine to native Windows command shells such as cmd and PowerShell.

The Basics

Your command line interface runs through just another application on your computer. It might be called “Terminal” or “The Shell” or “A Shell” or “Command” or “PowerShell”. How you launch it varies from system to system.

Your command line application carries out a dialog between you and the user. At a prompt, you type in a command and hit Enter (usually). The command executes, perhaps with a response from the system, and then you get a new prompt. Here is a sample session. In this example, the prompt is $, but on your system it might be different:

$ date
Mon Jan 20 17:40:16 PST 2020
$ echo Hello     how  are you
Hello how are you
$ grep qi /usr/share/dict/words 
Iraqi
Iraqian
qintar
Saqib
$ host www.google.com
www.google.com has address 172.217.11.68
www.google.com has IPv6 address 2607:f8b0:4007:802::2004
$

Nice! We’ve just seen four commands:

• date, which prints what the computer thinks to be the current datetime.
• echo, which echos back what you type (though not always exactly).
• grep, which finds things in files. Here we looked for the character sequence “qi” in a file of words (sorry, this file is not on Windows), and four words were found. (More about files coming up soon.)
• host, which looks up IP addresses for domain names (okay sorry for the network lingo, but hey that was faster than firing up graphical tool, right? Oh also, Git Bash doesn’t have the host command.)

The set of available commands differs between systems, but there’s a couple hundred that are common.

• Pretty good list of all macOS commands
• Pretty good list of all typical Unix commands
• In Git Bash, use the command ls -1F /bin | grep '\*$' | grep -v '\.dll\*$' | sed 's/\*$\|\.exe\*$//g'

Commands are super flexible. They have arguments and options. Try:

• ls
• ls /bin
• ls -l /bin
• ls -lFG /bin
• ls -lG

The arguments and options for each command are listed and explained in great detail in something called the command’s manpage.

The File System

We’re going to learn how to navigate and manage files using some cool commands and a bunch of powerful keyboard shortcuts, through an in-class exercise of making a travel photo library. But first, there are things about the computer’s file system we just need to know.

File System Basics

A few brief notes before we get started:

• A computer’s file system has a root folder which contains files and other folders. Each folder contains files and other folders. The name of the root folder is /. Here’s an example:

  

• The file system can also be viewed as a tree. The example above looks like this:

    /
    ├── Library
    │   ├── Fonts
    │   │   ├── Farisi.ttf
    │   │   ├── GillSans.ttf
    │   │   └── Sana.ttc
    │   └── Keychains
    │       └── System.keychain
    ├── Users
    │   ├── jessica
    │   │   ├── homework
    │   │   │   ├── hw1.py
    │   │   │   └── hw2.txt
    │   │   └── resume.pdf
    │   └── kimiko
    │       ├── IMG0001.jpg
    │       └── IMG0002.jpg
    └── bin
        ├── date
        ├── hostname
        ├── kill
        └── sleep
  

• The absolute pathname of a file or folder “starts at” the root folder, and works its way down, for example:

      /bin
      /bin/kill
      /Users/jessica/resume.pdf
      /Users/jessica/homework/hw1.py
      /Library/Fonts/Farisi.ttf
      /Library/Keychains
  

• When using the command line, there is always a current folder, also known as the working directory that you are “in” at the moment.
• Files and folders always have a relative pathname, whose name starts not from the root, but from the current folder. For example, if the current folder is /Users/jessica, then the following are relative pathnames for other folders and files:

      resume.pdf
      homework
      homework/hw1.py
      ..                                  (parent folder)
      ../kimiko
      ../kimiko/IMG0001.jpg
      ../..
      ../../bin
      ../../bin/hostname
      ../../Library
      ../../Library/Fonts/Sana.ttc
      .                                   (current folder)
      ./resume.pdf                        (pretty cool, right?)
  

• Every logged in user has a home folder, or home directory. Its name is ~. You can use this to make pathnames, too. If your home folder was /Users/kimiko, then you can always refer to your picture 1 as ~/IMG0001.jpg, regardless of the current folder.
• File names with spaces are okay, but they need to be quoted or have their spaces prefixed with a backslash. When typing filenames in commands, the following names are equivalent:

      Tokyo\ Tower.jpg
      "Tokyo Tower.jpg"
  

“Folder” or “Directory”?

Doesn’t matter. Same thing. You can say either.

An in-class exercise

We will make a travel photo library, together in class. (For homework, you will clean up and build out a movie library, starting with a very messed up an incomplete set of starter files.)

In this exercise we will learn:

• How to start the terminal application
• How to create folders and files (mkdir, touch)
• How to list the contents of a folder (ls, find, tree)
• How to navigate within the file system (pwd, cd)
• How to move, copy, and delete folders and files (mv, cp, rm, rmdir)
• Some incredible and cool keyboard shortcuts TAB ⌃A ⌃E ⌥← ⌥→ ⌃L ⌘K ⌃R

Can’t find the exercise content?

We’ll be doing it in class, that’s why! Take good notes.

Special Characters

Not every character is legal in a filename; some have special meaning within Bash. If you really, really must put them in a filename, you probably can, but in order to use the file, you’ll need quotes around the file name, and even then there’s no guarantee things will work (at least not without some serious worry.) Here are some good things to know:

1. If at all possible, stick to letters, digits, dashes, dots, and underscores only! Try to avoid anything else.
2. NEVER use /, as that separates path components.
3. NEVER use quotes, as these are the way you save yourself from special characters.
4. TRY TO AVOID spaces, as they can get irritating, though just quote them if you see them.
5. NEVER use braces ({ and }) because these get expanded.
6. NEVER use asterisks, question marks, or square brackets (* and ? and [ and ]) because these get expanded.
7. NEVER use &, |, <, > because these are used to combine commands and do redirections (hang on these will be explained soon).
8. This list of characters to avoid or never use is not complete. Just remember the first rule.

Expansion

Wait what does “get expanded” mean?

Braces and the asterisk are used to generate text! It might appear complicated, but really it’s pretty amazing. Braces make arbitrary text. This is called brace expansionHere are examples:

$ echo {b,r,spl,ch}at
bat rat splat chat
$ echo {2..16}
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
$ echo b{a,e,i,o,u}t
bat bet bit bot but

The asterisk, question mark, and square brackets participate in filename expansion: they generate filenames only (which must exist). Though the actual rules are complicated, * matches any string, ? matches a single character, and [ ]matches a single character from the enclosed pattern, and its pattern language can be quite powerful. We won’t get into the details, but will just look at examples:

$ mkdir scratch
$ cd scratch/
$ touch dog.txt rat.jpg bat.txt animals personal.html
$ ls
animals        bat.txt        dog.txt        personal.html  rat.jpg
$ ls ?at*
bat.txt  rat.jpg
$ ls *.txt
bat.txt  dog.txt
$ ls *o*
dog.txt        personal.html
$ ls *[sgv]
animals  rat.jpg

Summary of file commands

We learned these in our in-class exercise.

Other Commands

You don’t use the command line only for managing your files. There are tons of other things to do, from starting and stopping programs, to networking, to controlling devices. Please browse this awesome online reference of commands. It covers Linux, but there is a Mac Version too. Most of these commands work in Git Bash too. You can also find a list at Wikipedia.

Some of the cooler commands to learn are:

• date
• cal
• file
• cat
• sort
• uniq
• tail
• less
• zip
• unzip
• curl
• which
• sudo

If you are into details, details, details, see The Bash Reference Manual.

Keyboard Shortcuts

To use the command line like a pro, and save tons of time, you’ll be happy knowing at least of few keyboard shortcuts. Some of the more useful ones are:

But there are dozens more! Here is a great list of keyboard shortcuts. You can also, if you have tons of time and really, really want to know the details, read the section on command line editing in the Bash Reference Manual.

Operating on the Commands

These operations on commands should be a part of any command line users’ knowledge. They can be learned over time.

command1 | command2

(The pipe) Send the output of command1 into the input of command2.

command1 && command2

(AND) Execute command1, then ONLY if command1 succeeds, proceed to command2.

command1 || command2

(OR) Execute command1, then ONLY if command1 fails, proceed to command2.

command1 ; command2

(SEQUENCE) Execute command1, then execute command2.

command1 < file

(REDIRECT STANDARD INPUT) Read input not from the keyboard, but from the given file.

command1 > file

(REDIRECT STANDARD OUTPUT) Output not to the terminal, but to the given file.

command1 >> file

(REDIRECT AND APPEND TO STANDARD OUTPUT) Output not to the terminal, but append it to the given file.

command &

(BACKGROUND) Execute command as a background task. Do this when the command would take a long time to run, or is supposed to run in the background. After launching a command this way, control of the terminal is immediately returned to you. The program you launched will show up in your system list of processes (assuming it hasn’t finished or been killed).

Examples (meant to be executed in order):

• Create a new file called hello.py with a little Python program in it:

  $ echo 'print("Hello")' > hello.py
  

• Add a new line to hello.py:

  $ echo 'print("World")' >> hello.py
  

• Uppercase the file hello.py

  $ tr '[a-z]' '[A-Z]' < hello.py
  

• Compile the Java program Greeter.java, then ONLY if the compilation proceeds, move on to running it.

  $ javac Greeter.java && java Greeter
  

• Run the tree program, but if it fails (including by not existing), run find

  $ tree || find .
  

• List a few of the recent git commands you ran:

  $ history | grep 'git '
  

• Download the text of the book Emma to a new file called emma in the current directory:

  $ curl http://www.gutenberg.org/files/158/158-0.txt > emma
  

• Find the 25 most frequently appearing words in the book, printing them together with their number of occurrences, in descending order:

$ cat emma | tr '[A-Z]' '[a-z]' | sed -E $'s/[^a-z]+/\\\n/g' | grep [a-z] | sort | uniq -c | sort -rnk 1 | head -n 25

More!

Don’t be afraid to get too “advanced.” Instead embrace what can be done. Check out what you have time for, and ignore, for now, what seems a little arcane. There will be time. When you’re ready, here are some good places to go for details, tips and tricks, and fun times:

• The Bash Reference Manual
• Wikipedia article on Bash
• Bash Cheat Sheet
• macOS Command Reference
• Command Line Fu
• A cool presentation
• Terminus Game
• Cmd Challenge

Try this command:

$ nc towel.blinkenlights.nl 23

Bonus Material: Built-In Folders

You’ll notice a lot of existing folders and files, even on a brand-new machine. A number of these folders are “standard.” The standard folders are operating system dependent. For fun, here are the ones you’ll see in most Linux-based systems (based on this thing called the FHS):

Choosing Your Shell

On Macs and Unix systems, the actual shell (command line interface) program run in your Terminal window is up to you. On modern Macs, the default shell is zsh. But bash is still available. The OG shell, sh is also available. Old rarely used anymore shells include csh, tcsh, and ksh. Oh, and there is also fish.

Though these notes specifically covered bash commands, almost everything applies exactly to zsh. Bash and zsh differ mainly in configuration, something not covered in these notes, but things you will need to know to fully customize your system and someday become a true power user.

Recall Practice

Here are some questions useful for your spaced repetition learning. Many of the answers are not found on this page. Some will have popped up in lecture. Others will require you to do your own research.

1. Why is a command line often faster than graphical interface programs?
   Because hands stay on the keyboard, text is dense, you’re not limited to what’s on the menus, it might be the only way to control a remote computer, you don’t need thousands of separate window-based apps to clutter your workspace.
2. What are some tasks that are facilitated by command line interfaces?
   Create, delete, rearrange, edit, and search files, start, stop, and inspect your running programs, get information such as the current time, how much disk space is left on your system, connect to, control, and copy files between, devices and remote computers.
3. What are common names for the program that runs a command line interface?
   Terminal, Shell, Command, PowerShell
4. What shell command creates three folders named A, B, and C?
   mkdir A B C
5. What does the date command do?
   Prints the current date and time
6. What does the echo command do?
   Prints its arguments
7. What does the grep command do?
   Searches for a pattern in a file
8. What does the host command do?
   Looks up IP addresses for domain names
9. What does the ls command do?
   Lists the contents of a folder
10. What is the file system?
    The set of folders and files on a computer and how they are arranged
11. What is the root folder?
    The topmost folder in the file system
12. What is the absolute pathname of a file?
    The pathname starting from the root folder
13. What is the relative pathname of a file?
    The pathname starting from the current folder
14. What is another name for the “current folder”?
    Working directory
15. What is the home folder?
    The main folder for a logged in user
16. What is the touch command commonly used for?
    Creating files
17. What do pwd and cd do?
    pwd prints the current working directory, cd changes the current working directory
18. What does the mv command do?
    Moves or renames files
19. What does the cp command do?
    Copies files
20. What does the rm command do?
    Removes files
21. What does the rmdir command do?
    Removes folders
22. What are some characters that cause command line expansion?
    Braces, asterisks, question marks, square brackets
23. What does the command echo b{a,e,i,o,u}t do?
    Prints bat bet bit bot but
24. What does the command touch card{0..99}.txt do?
    Creates 100 files named card0.txt through card99.txt
25. How does one create three files named a, b, and c?
    touch a b c
26. How do we display the names of all files in the current folder whose names start with c?
    ls c*
27. What does the cat command do if only one argument is given?
    Prints the contents of a file
28. How do we represent the current folder, the parent folder, and the home folder?
    .
..
~
29. What command is used to run a program associated with a file?
    open
30. In most shells, how to you move to the beginning and end of a line?
    Ctrl+A and Ctrl+E
31. What do the up and down arrows do in a shell?
    Navigate through the history of commands
32. What does the TAB key do in a shell?
    Autocompletes filenames
33. What is meant by the composite command f ; g?
    Run command f, then run command g
34. What is meant by the composite command f && g?
    Run command f, then run command g only if f succeeds
35. What is meant by the composite command f || g?
    Run command f, then run command g only if f fails
36. What does ls .. > hello do?
    Writes the list of filenames in the parent folder to a file named hello
37. What does echo 'print("World")' >> hello.py do?
    Appends the text print("World") to the file hello.py
38. What is meant by the composite command f | g?
    Run command f, then run command g with the output of f
39. What does curl do?
    Makes a network request

Summary