October 17, 2018

Since the introduction of async/await in Node v8, I've been using it a lot to write cleaner code. And if you've used other programming languages in the past, you probably already know how to use it.

In this article I'll discuss the basics of async/await for the newcomers, but also some more advanced tips and tricks that you should consider when using async/await in production.

Where did we come from?

To correctly understand what we are doing in the modern async/await, I'll give a short introduction of where we came from.

In Javascript, all code is executed in sequence, on a single thread. And while the execution of Javascript is blocking, the I/O operations are not ("asynchronous non-blocking I/O model"). This is because the I/O operations are handled by the underlying engine. Examples of I/O operations are: fetching API data over the internet, reading a file from disk or querying data from a database.

Synchronous

The worst way we can do I/O operations in Node, is synchronous. Let's see an example first:

const fs = require('fs')

const contents = fs.readFileSync('some-file.txt', 'utf8')
console.log(contents)

What happens here is that a file is being read into contents, but while it does so, code execution stops until file reading is completed. No other Javascript code will be executed in the meantime.

Callbacks

To make sure you can continue executing your code, we can make use of a callback. A callback function is executed when the I/O operation finishes.

const fs = require('fs')

function callback (err, contents) {
    console.log(contents)
}

fs.readFile('some-file.txt', 'utf8', callback)
console.log('so some other work')

As you can see, we're calling the asynchronous function readFile to which we pass the callback function callback. It is saved to be executed later once the underlying I/O operation is finished and does not block the execution of Javascript code.

While the underlying engine is reading the data from the file, code is continuing to the next statement and does some more work for us. When reading from the file is finished, the engine calls the callback function, which prints the contents of the file to the console.

Callback Hell

It's good to have asynchronous code, and callbacks is a great way to achieve this. But as you've guessed, there is also a downside to callbacks. This downside is called "the callback hell".

When you're working with asynchronous code, you've probably had situations where you'd want to wait executing a function before another is finished. For example, read file A, compress file contents, write to new file B and ultimately calculate size difference between file A and B. This would look something like this:

readFile('file-A.txt', function(err1, contents) {
    // handle error err1, or continue
    compressData(contents, function(err2, compressed) {
        // handle error err2, or continue
        writeFile('file-B.gz', compressed, function(err3, result) {
            // handle error err3, or continue
            compareFileSizes('file-A.txt', 'file-B.gz', function(err4, diff) {
                // handle error err4, or continue
                console.log('File size difference: ' + diff)
            })
        })
    })
})

console.log('so some other work')

I think you can already spot the problem here. This callback hell, or also called Christmas tree effect is not a clean way to code. I did not even handle all the possible errors yet, and this code already looks messy with callback in callback in callback in callback. When this code block grows any bigger, it will be more difficult to debug it also.

Promises

A Promise is an object that wraps an asynchronous operation and notifies when the operation was successfully finished, or failed with an error. This sounds exactly like a callback, but instead provides it's own methods.

readFile('file-A.txt')
    .then(function (contents) {
        // compress data
    })
    .catch(function (err) {
        // handle error
    })

It breaks up the callback into 2 separate functions. A .then() for a successful result and a catch() for when an error occurs. Instead of nesting callbacks, you can chain the then() functions and pass the result into the next promise. Another advantage is that the chain of promises can have a single error handler.

Reading the file, compressing the contents, writing the next file and calculating the difference can be re-written like this:

readFile('file-A.txt')
    .then(compressData)
    .then(compressed => { return writeFile('file-B.gz', compressed) })
    .then(result => { return compareFileSizes('file-A.txt', 'file-B.gz') })
    .then(diff => {
        console.log('File size difference: ' + diff)
    })
    .catch(function (err) {
        // handle error
    })

This already looks much better, but we're not there yet.

Async / Await

The "async/await" syntax is an easier way to work with Promises. An async function actually returns a Promise. This means that we're still writing non-blocking code, the Promise way, but with an easier syntax.

async function compressCompareFile () {
    const contents = await readFile('file-A.txt')
    const compressed = await compressData(contents)
    const result = await writeFile('file-B.gz', compressed)
    return compareFileSizes('file-A.txt', 'file-B.gz')
}
compressCompareFile()
    .then(diff => { console.log('File size difference: ' + diff) })
    .catch(err => { })

All these (async-)functions that perform file actions return a Promise. The await pauses the async function, waits until the Promise resolves and then continues to the next statement. In the background everything is still asynchronous and we're not blocking code executing elsewhere.

Async

The async keyword is placed in front of a function, and simply means that the function performs asynchronous operations and returns a Promise. If you return a value from the function, then JavaScript automatically wraps it into a resolved promise with that value.

async function calculate () {
    return 1234
}
// ... is the same as:
async function calculate () {
    return Promise.resolve(1234)
}

Await

The await literally makes Javascript wait until the Promise resolves and returns the value that would normally be available in the .then() of the Promise. This waiting does not block any other code or operations, and does not cost any CPU resources.

Note: the await keyword can only be used inside an async function. If it's used inside a regular function, a syntax error will occur!

// works only inside async functions
const result = await calculate()

Error Handling

If a Promise resolves normally, then await returns the result. But in case of a rejection, it throws the error, just like there was an error thrown at that line of code. This means that we can handle errors with a standard try..catch block.

try {
    const contents = await readFile('file-A.txt')
    const compressed = await compressData(contents)
    const result = await writeFile('file-B.gz', compressed)
    return compareFileSizes('file-A.txt', 'file-B.gz')
} catch (err) {
    // handle error
}

Top level

Since we can only use the await inside of an async function, we can not use it at the top level of our code. In this case it's normal practice to use .then()/.catch(). Also, when there are no try..catch error handling blocks inside your async functions, any errors that occur will bubble up to the main module inside the .catch() of the top call to the async function.

async function calculate () {
    return 1234
}
calculate().then(result => {
    // got result
}).catch(err => {
    // handle error
})

Performance

Now you must be thinking that you know all about async/await. You've maybe tried it out in your own code by putting await in front of every Promise call and transforming Promise-functions into async-functions. And if it works, you know that you're code is asynchronous and non-blocking. So what can go wrong?

Dependency

In the previous examples, we read a file, compressed it's contents and written it back to disk. These 3 functions depend on each other and have to be executed in order. First read the file before we can compress the data. And we can only write something back to disk, when compression is finished.

Let's simplify this procedure into a single function that we want to repeat. We have a folder with 3 files that we all want to compress. The code should look something like this:

const resultA = await compressFile('file-A.txt')
const resultB = await compressFile('file-B.txt')
const resultC = await compressFile('file-C.txt')
const diff = await compareFileSizes('*.txt', '*.gz')

So what it wrong with this code? It runs fine, right? In this example, the code runs line-by-line. Compress file A, wait until it finishes, compress B, wait, compress C, wait and finally calculate the difference in size.

We're actually creating a performance issue here, because the 3 files are compressed 1 by 1. The 3 compress functions have no relation to each other, so it would be better to start compressing the files at the same time. They can run in parallel, and when they all finish, we can calculate the difference.

const promiseA = compressFile('file-A.txt')
const promiseB = compressFile('file-B.txt')
const promiseC = compressFile('file-C.txt')
await promiseA
await promiseB
await promiseC
const diff = await compareFileSizes('*.txt', '*.gz')

The 3 compression functions are now all started and can run at the same time. Since we're not calling them with the await anymore, the Promise object is returned from the async function. We can use it later in code to see if the've been resolved.

We could also replace the await statements with a single Promise.all for easier reading.

const promiseA = compressFile('file-A.txt')
const promiseB = compressFile('file-B.txt')
const promiseC = compressFile('file-C.txt')
await Promise.all([promiseA, promiseB, promiseC])
const diff = await compareFileSizes('*.txt', '*.gz')

Summary

We went all the way from synchronous Javascript code to modern async/await. I hope that the many examples in this article have given you an impression on how you can really simplify you code. It makes it easier to read and eventually easier to debug. And if you already know Promises, you know all about async/await.