Invoking Web APIs

Web APIs are probably the most common APIs of them all. Few people get the pleasure of writing them, but many developers will need to invoke them.

What is an API?

Here’s what Wikipedia says:

An application programming interface (API) is an interface or communication protocol between different parts of a computer program intended to simplify the implementation and maintenance of software. An API may be for a web-based system, operating system, database system, computer hardware, or software library. An API specification can take many forms, but often includes specifications for routines, data structures, object classes, variables, or remote calls.

Many programming languages provide “APIs” for their standard libraries or third-party libraries. These APIs are generally documented with detailed descriptions of the functions or classes they provide. Examples: Java, JavaScript, Ruby, Go. These are not Web APIs, though.

What is a Web API?

A Web API is one that provides (resources and) services via HTTP or HTTPS. Yep, that’s all it is.

HTTP

If you are unfamiliar with HTTP or HTTPS, they are (at the most basic-level) request-response protocols. HTTPS is just HTTP with security at the transport layer. You’re welcome to check out my notes on HTTP or see Wikipedia.

In many circles, the term “API” refers to a Web API.

Someone who knows as much as anyone about APIs is Kin Lane.

There are many thousands of Web APIs out there. For each API there are three big questions:

Is it free?

Does it require authentication?

Is it CORS-enabled?

Example Web API: Studio Ghibli

The Studio Ghibli API is a free and very small API, requiring no authentication, with good documentation. A great API to start with! Browse the documentation now.

Try out some of the suggested endpoints, such as https://ghibliapi.herokuapp.com/films/58611129-2dbc-4a81-a72f-77ddfc1b1b49.

Example Web API: Jikan

A free API. There are tons of endpoints. All of them are GET endpoints, all return JSON. Here are just a few ot them:

GET /search/anime?q=titan
GET /anime/{id}
GET /anime/{id}/episodes
GET /anime/{id}/episodes/{page}

Big Lists of APIs

Some nice folks have made nice lists of APIs for you! Check out: this one, this one, this one, and (especially) this one.

Fun ones: Google Maps, imdb, Twitter, GitHub, Spotify.

Invoking Web APIs with curl

curl is easy to use on the command line. Quick example first:

$ curl -i 'https://api.jikan.moe/v3/search/anime?q=Death+Note&type=anime&page=1&limit=1'
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Sun, 01 Mar 2020 22:55:01 GMT
Content-Type: application/json
Content-Length: 743
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: *
Cache-Control: private, must-revalidate
ETag: "5cc615d7da28ed9ff381e648c4c0507b"
X-Request-Hash: request:search:463fa8decae648e3d43d1d40d046d9cd2ff6a415
X-Request-Cached:
X-Request-Cache-Ttl: 432000
Expires: Fri, 06 Mar 2020 22:55:01 GMT
Vary: Accept-Encoding
X-Cache-Status: MISS

{"request_hash":"request:search:463fa8decae648e3d43d1d40d046d9cd2ff6a415","request_cached":false,"request_cache_expiry":432000,"results":[{"mal_id":1535,"url":"https:\/\/myanimelist.net\/anime\/1535\/Death_Note","image_url":"https:\/\/cdn.myanimelist.net\/images\/anime\/9\/9453.jpg?s=b89e80691ac5cc0610847ccbe0b8424a","title":"Death Note","airing":false,"synopsis":"A shinigami, as a god of death, can kill any person\u2014provided they see their victim's face and write their victim's name in a notebook called a Death Note. One day, Ryuk, bored by the shinigami lifesty...","type":"TV","episodes":37,"score":8.63,"start_date":"2006-10-04T00:00:00+00:00","end_date":"2007-06-27T00:00:00+00:00","members":1924927,"rated":"R"}],"last_page":20}

That example used the -i option of curl to display the status line and headers. Without the -i you see only the response.

Here’s another example, where I've piped the output of curl into the jq command to prettyprint the JSON. The command line can be so cool. Keep practicing and learning it.

$ curl 'https://api.jikan.moe/v3/search/anime?q=Ajin&type=anime&page=1&limit=1' | jq
{
  "request_hash": "request:search:d6f2df3ea084db0a57ff65258b37cf964f8ab249",
  "request_cached": true,
  "request_cache_expiry": 423870,
  "results": [
    {
      "mal_id": 31580,
      "url": "https://myanimelist.net/anime/31580/Ajin",
      "image_url": "https://cdn.myanimelist.net/images/anime/13/77968.jpg?s=3838c24a0ecafc453e980e7da1a8eb32",
      "title": "Ajin",
      "airing": false,
      "synopsis": "Mysterious immortal humans known as \"Ajin\" first appeared 17 years ago in Africa. Upon their discovery, they were labeled as a threat to mankind, as they might use their powers for evil and were incap...",
      "type": "TV",
      "episodes": 13,
      "score": 7.57,
      "start_date": "2016-01-16T00:00:00+00:00",
      "end_date": "2016-04-09T00:00:00+00:00",
      "members": 362127,
      "rated": "R"
    }
  ],
  "last_page": 20
}

Invoking Web APIs Client Side

Every modern web browser implements the fetch function. Read Mozilla’s documentation on fetch now, it’s pretty good.

From Vanilla JS

Here is a complete example of a simple HTML page that hits the Jikan API on load to list the id numbers and titles of the episodes of the (first season) of the anime Shingeki No Kyojin:

aot.html
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Document</title>
  <style>table {border-collapse: collapse} th, td {border: 1px solid black}</style>
</head>
<body>
  <h1>Shingeki no Kyojin Season 1 Episodes</h1>
  <table>
    <tr><th>Episode</th><th>Title</th></tr>
    <tbody id="episodes"></tbody>
  </table>
  <script>
    addEventListener('load', () => {
      const url = 'https://api.jikan.moe/v3/anime/16498/episodes';
      fetch(url).then(r => r.json()).then(response => {
        response.episodes.forEach(episode => {
          const row = document.createElement("tr");
          const idCell = document.createElement("td");
          idCell.textContent = episode.episode_id;
          const titleCell = document.createElement("td");
          titleCell.textContent = episode.title;
          row.appendChild(idCell);
          row.appendChild(titleCell);
          document.querySelector("#episodes").appendChild(row);
        })
      })
    })
  </script>
</body>
</html>

Hmmm, this would look better in React, right?

From React

Here’s the previous example in React. Note how API invocations should go into effects:

Invoking Web APIs Server Side

From Node

From Python

Postman

Wrappers

Many times an API provider will provide you with a library (sometimes called a “wrapper” or an “SDK”) containing a set of functions that internally make the API call (via fetch, for example).

Understanding Authentication

Many APIs require that at least some, and maybe all, of their endpoints, be authenticated. That means that some information in the request must be present that identifies the requestor as a valid “user.” This information could be:

In a well-designed API, this information cannot be easily guessed or stolen (via, say, MitM attacks). Some of the big ideas in authentication are:

CORS

Useful API Vocabulary

Make yourself some flashcards with the following terms and their definitions:

Web API
...
REST
...
Endpoint
...
HTTP
...
HTTP Request
...
HTTP Response
...
HTTP Method
...
HTTP Header
...
HTTP Status Code
...
Cache
...
Fetch
...
Synchronous
...
Asynchronous
...

Summary

We’ve covered:

  • ...