Asynchronous Programming in JavaScript

What, me wait?

Background

JavaScript programs are generally single-threaded. Conceptually, program execution consists of running code blocks in response to events. All code blocks are run to completion; we never run two or more code blocks together at the same time. (However, a good JavaScript engine will know how to use multiple processor cores and may perform various optimizations—and language features like WebWorkers do exist—but the one-block-at-a-time-on-a-single-thread model is true for everyday JS.)

This is probably the simplest async program:

console.log("First");
setTimeout(() => {console.log("Third")}, 0);
console.log("Second");

This program reliably and deterministically prints First then Second then Third.

setTimeout(f, t) simply schedules the function f to run after at least t milliseconds. Importantly, note that f is only scheduled by the setTimeout call: it does not even get a chance to run until the currently executing block of code finishes! Only after the script finishes, printing First and Second, can the delayed function be run to print Third.

Also note: JavaScript code never gets interrupted. If we had a long running script that makes a call to setTimeout(f, 50), the call returns immediately and the script keeps going. The script will not be interrupted in 50 milliseconds. Even if the script takes 10 seconds to complete, the scheduled function will just have to wait. (In other words, 50 means “at least 50”.)

Callbacks

Let’s start with an example. In Node’s fs module there is a readFile function, defined by:

function readFile(filename, options, callback) {
  // Schedules some code to run. This code will read the file then invoke
  // callback(...) with the file data that was read and/or error info.
}

which you call like this:

// Here is how we call the async function. We defined the callback as an
// anonymous function thought that wasn’t strictly necessary. The first
// argument to the callback is an error object if reading the file failed
// or null otherwise. The second argument holds the lines of the file that
// was successfully read. The body of a callback almost always begins with
// an error check.
fs.readFile('stuff.txt', 'utf-8', (err, data) => {
  if (err) throw err;
  console.log(data); // (for example)
});

Note that the call to readFile only schedules the code to read this file and therefore returns almost immediately. So readFile doesn’t actually read the file. This simple example:

f();
readFile('myfile', dealWithIt);
h();

actually invokes h() before anything gets read. Only after the file is read will we finally invoke dealWithIt(err, data).

So this is pretty cool: because readFile is an async function, we get to call h() without waiting for the file to be read! With asynchronous programming we almost never have to “wait.” This often helps the processor stay busy and helps gives such systems higher throughput.

Exercise: Define the term “synchronous function” and give examples.

Sometimes you have to do a number of asynchronous operations one after the other. What does this look like?

function runJob(configFile, outputFile) {
  initializeConfiguration(configFile, (err, configInfo) => {
    if (err) throw err;
    fetchData(configInfo.whatever, (err, rawData) => {
      if (err) throw err;
      processData(rawData, (err, processedData) => {
        if (err) throw err;
        analyze(processedData, (err, analysis) => {
          if (err) throw err;
          report(analysis, outputFile, (err) => {
            if (err) throw err;
            console.log(`Report written to ${outputFile}.`);
          });
        });
      });
    });
  });
}

Do you find that code pretty? Many people don’t. Nesting callbacks like this is called callback hell.

The simplest way to avoid callback hell is probably to just make the callbacks explicit:

function runJob(configFile, outputFile) {
  initializeConfiguration(configFile, handleConfiguration);

  function handleConfiguration(err, configInfo) {
    if (err) throw err;
    fetchData(configInfo.whatever, handleFetch);
  }

  function handleFetch(err, rawData) {
    if (err) throw err;
    processData(rawData, handleProcessing);
  }

  function handleProcessing(err, processedData) {
    if (err) throw err;
    analyze(processedData, handleAnalysis);
  }

  function handleAnalysis(err, analysis) {
    if (err) throw err;
    report(analysis, outputFile, handleReport);
  }

  function handleReport(err) {
    if (err) throw err;
    console.log(`Report written to ${outputFile}.`);
  }
}
This stuff is important

Make sure you understand callbacks before looking at more advaned features.

Promises

Instead of writing an async function that accepts a callback as a parameter, your async function can return a Promise to which you apply callbacks. In the simplest case, instead of the classic:

myFunction(args, callback);

you write:

myFunction(args).then(callback);

At first glance, that doesn’t seem like much, but promises have one huge plus in their favor: they are composable, so when you have lots of async operations, things are going to look much cleaner. Examples first and explanations later. Remember the example from before? Here it is assuming each of the async functions return promises:

function runJob(configFile, outputFile) {
  initializeConfiguration(configFile)
  .then(configInfo => fetchData(configInfo.whatever))
  .then(rawData => processData(rawData))
  .then(processedData => analyze(processedData))
  .then(analysis => report(analysis, outputFile))
  .then(() => console.log(`Report written to ${outputFile}.`))
  .catch(err => { throw err; });
}

In this code above, initializeConfiguration schedules some async work and returns a Promise object. Promise objects have two methods, then and catch, which attach callbacks to the promise. The cool thing is calling then just returns another promise. So the chaining we end up with is pretty sweet.

Exercise: Can you simplify the fourth and fifth lines in the example above?

Okay, so what exactly is a promise? How do you make one? How do they work internally?

What is a Promise?

A promise is an object representing a value which may or may not be ready yet, or may never be ready. It may be in one of three states:

At the most basic level if we have:

Promise p = ...;
the promise is in the pending state. Calling:
p.then(f, g)

means (roughly) if/when the promise p gets fulfilled, f (if it is a function) gets called; and if/when p gets rejected, g (if it is a function) gets called. Note that:

But what’s cool is that p.then returns another promise, so we can write nice chains.

What happens if the promise is already fulfilled or rejected before a handler gets attached? (Good question if you thought to ask it!) Well don’t worry: the handler will get called right away.

Constructing Promises

The promise constructor takes in a single argument (known as the “executor”), which is a function of two arguments, generally called resolve and reject. The executor body should call resolve on success and reject on failure:

const promise = new Promise((resolve, reject) => {
  setTimeout(() => {
    if (Math.random() < 0.5) resolve("Yes"); else reject("No");
  }, 250);
});

promise
  .then(message => { console.log(`ok: ${message}`); })
  .then(() => { console.log ('Who knew?'); })
  .catch(message => { console.log(`fail: ${message}`); })
  .then(() => { console.log('Done'); });

Run this code multiple times to see different outputs.

Note you never have to call the executor function directly, ever. The Promise constructor calls it for you, passing internal functions (for resolving and rejecting) to your executor.

In practice, you will write your own async functions using a pattern something like this:

function myFunction(args) {
  return new Promise((resolve, reject) => {
    // ...
  });
}

So remember, async functions either (1) accept your callback or (2) return a promise.

Understanding Chaining

Lets say we have:

p1.then(f1, g1).then(f2, g2).then(f3, g3)

which we will rewrite to the completely equivalent:

p2 = p1.then(f1, g1);
p3 = p2.then(f2, g2);
p4 = p3.then(f3, g3);

Now the way promises are set up to work is:

if p1 is fulfilled
    if f1 is a function
        if f1() succeeds
            fulfill p2
        else
            reject p2
    else
        reject p2
if p1 is rejected
    if g1 is a function
        if g1() succeeds
            fulfill p2
        else
            reject p2

This is why rejections can propagate through a series of simple thens, and why after a successful catch, we can resume the chain of thens.

More Promise Functionality

You can do other cool things:

Promise.all([p1, p2, p3, ..., pn])
Produces either (1) a fulfilled promise, if all of the promises fulfill, or (2) a rejected promise when just a single one of the promises are rejected. In the former case the fulfilled value is an array of the values for which p1...pn were fulfilled. In the latter case the rejection reason is the rejection reason from the first of p1...pn to be rejected.
Promise.race([p1, p2, p3, ..., pn])
Produces a filled or rejected promise based on the first fulfilled or rejected promise from p1...pn, using the fulfilled value or rejection reason from it.
Promise.resolve(value)
Produces a fulfilled promise (with the given value)
Promise.reject(reason)
Produces a rejected promise (with the given reason)

Gratuitous examples:

> Promise.resolve(100).then(x => {console.log(x);})
Promise { <pending> }
100

> Promise.resolve('dog').catch(x => {console.log(x);})
Promise { <pending> }

> Promise.reject('dog').catch(x => {console.log(x);})
Promise { <pending> }
dog

Async and Await

Modern JavaScript has two new keywords that make writing async code even more elegant.

So remember from above how an async function returns a promise? Well you can actually mark a function with the keyword async, which means:

> async function div(x, y) {if (y === 0) throw 'divBy0'; else return x/y;}
undefined
> div.constructor
[Function: AsyncFunction]
> p = div(35, 7)
Promise { 5 }
> function show(x) { console.log(x); }
undefined
> p.then(show)
Promise { <pending> }
5
> div(5, 0).catch(show)
Promise { <pending> }
divBy0

An async function can contain an await expression, where await p suspends execution of the function until promise p is settled (i.e., fulfilled or rejected) and then resumes. This is NOT synchronous blocking! Other code can run during the time the async function is suspended, just like in a JavaScript generator.

If p is fulfilled, the resolved value is produced, otherwise the await expression will throw the rejected value. So convenient.

In the running example, assuming the utility functions are all marked async, we can just write:

async function runJob(configFile, outputFile) {
  const configInfo = await initializeConfiguration(configFile);
  const rawData = await fetchData(configInfo.whatever);
  const processedData = await processData(rawData);
  const analysis = await analyze(processedData);
  await report(analysis, outputFile);
  console.log(`Report written to ${outputFile}.`);
  // No need to catch and rethrow anything, because all the awaits will throw if necessary!
}

TL;DR

Well, pretty much everything we saw above is explained by Wassim Chegham’s brilliant nine-second explanation:

Sweet, right? Okay now go write correct, robust, async code of your own.