HTTP APIs – connect the web elegantly

What’s an HTTP API?

Credit: https://openclipart.org/detail/77383/shopping-cartblue

Good question. “API” stands for Application Programming Interface. In general, it’s a way for programmers to connect their software to an existing piece of software or hardware.

When hearing the term “API” many people think of a particular kind of API, but in reality the term covers just about any kind of software interconnection. Graphics programming libraries, database access, browser extensions, news and weather feeds, and so on — these are all APIs in one form or another.

HTTP APIs (or web APIs) let your software retrieve and modify information on a web server. For example:

Choices to make up front

There’s not just one kind of web API. You have browser APIs to interact with and control a web browser, modify its display or behaviour using JavaScript or a plugin architecture. Then you have server APIs, which typically allow a third-party developer to work with information stored on a remote server, sometimes from web browsers, sometimes from another server, sometimes from another bit of software.

I’ll be talking specifically about server APIs via HTTP. There are a lot of commonly-used technologies: REST, SOAP, JSON, RPC, XML. They all have various pros and cons, but I’ll use Python, Flask (a popular microframework for Python), REST and JSON to demonstrate the concepts.

Setting up your API

Assuming you have Flask installed already, here’s a simple single-file website that works:

from random import randint
from flask import Flask, jsonify

app = Flask(__name__)

@app.route("/api/hello")
@app.route("/api/hello/<int:max_num>")
def hello(max_num=10):
    """ Return a Hello World message along with a random number """
    return jsonify(message="Hello World!", number=randint(1, max_num))

if __name__ == "__main__":
    app.run()

Some tutorials for other laguages:

Building a great API

URL structure

To make life easier for anyone using your API, you’ll need a good URL structure. This will make your API easier to work with, and save users a lot of time.

For example:

http[s]://apiv1.example.com/product-categories/
http[s]://apiv1.example.com/product-categories/3487/
http[s]://apiv1.example.com/product-categories/3487/specials/

Or

http[s]://api.example.com/v1/product-categories/

Data & response structure

Next you should have a sensible, consistent data structure. How exactly you structure your response depends on your requirements, but it should return useful information, and an appropriate HTTP status code. Common status codes would be 200 for a successful request, 403 for request denied (eg. invalid credentials), 404 for resource not found and 500 for a server error.

A basic message-only response:

{ "message": "Purchase OK" }

A bad message-only response:

{ { "status": {"message": "Purchase OK" } }

What’s bad about this? There is a superfluous layer of nesting, and needless extra keys.

A message containing a list of shopping basket items:

{ "items": [ { "id": 123, "description": "Parcel tape", "quantity": 1 },
             { "id": 456, "description": "Wrapping paper", "quantity": 1 },
             { "id": 789, "description": "Label", "quantity": 3 } ] }

Similarly, you could do things very wrong:

{ "ids": [123, 456, 789],
  "descriptions": ["Parcel tape", "Wrapping paper", "Label"],
  "quantities": [1, 1, 3] }

While it might look like we avoided the needless use of keys and made our data structure tidy and compact, it’s much harder to guarantee that the shopping basket ids, descriptions and quantities have been added in the correct order.

Try to make your data structures sensible, consistent, and readable to the human eye. Group related information information together.

It’s also worth making your response structures consistent across different API endpoints — for example, avoid calling a message “message” in one endpoint, “response” in the next, “note” in another.

Error handling

This is another essential. When there is an error, you need to make sure that your API handles it gracefully, and informs anyone who needs to know about it. A typical approach is to issue an error message in your JSON response in conjunction with an appropriate HTTP status code.

If it’s a serious error that’s not the API user’s fault, it’s also a good idea to send an email to your development team both advising of, and describing, the error so they can fix the fault. Include the exception name and a stack trace at a minimum. Depending on your requirements, you might also want to log it to disk.

Python’s logging module (version 2 | 3) is great for handling this.

Authentication and Access Control.

Depending on what you’re doing, this may or may not be necessary. For a public API, you probably don’t need to worry about it. If you’re using a local web server on a locked-down embedded device, it’s probably not an issue. But if you want to limit your audience or the volume of requests, you’ll need some way to restrict and allow access.

There are several common methods for doing authentication. It’s often a tradeoff between security and convenience. Weigh up how sensitive your API’s information is. If it’s financial information you’ll want it pretty tight, but if it’s a small list of products which can be viewed by the public anyway then it won’t be such a pressing concern.

One easy & simple way is to include an API username and key in the request URL, and check this against a known list. For example:

https://apiv1.website.domain/member/123/details?user=bob&key=P1neapple

When a request is made to this URL you’d compare the user (bob) and key (P1neapple) to a list of allowed key/user combinations, and then compose an appropriate JSON response, or an HTTP 403 if they’re not allowed in. You could also omit the username entirely and simply use a long and unique key for each API user.

While easy and convenient, this method is not very secure as your user credentials will be stored in any place which logs the request URL. You should never use URL-based authentication when transferring sensitive information.

To make authentication more secure you could place API credentials in the request header, or use OAuth.

Some annoying things to avoid

Simplistic credential management: An entry in a settings file is fine if you only have one or two users but if you’re supporting quite a few then it could get unwieldy very quickly. Consider storing API credentials in a database to make managing access for multiple users easier in the long run.

No documentation: Make sure you have some. Besides making life easier for anyone who uses your API, its presence shows that you’ve put a lot of thought and effort into your API. Your target audience is developers who use your API, but remember they’re humans first. Keep your documentation simple and to the point.

User experience & needless requests: Your API users are likely to be developers, but they’re still users. Consider how they might expect to use it. Is there a sensible way you can provide all the information they’re likely to need without needing to issue multiple requests?

Sluggishness: If you’re returning lots of information speed can be an issue. This can come from any number of areas and you’ll need to run the appropriate benchmarks and tests, but in most data-heavy APIs you should look hard at your database queries – return only the columns/fields you need and make good use of JOINs.  If you’re using django (another Python framework), learn to love .select_related() and .prefetch_related()!

Any kind of API is a user interface for programmers. Even though it’s graphical, basic UI principles still apply: clarity, consistency, visibility, feedback, avoiding surprising behaviour, and a sensible mapping between what the user sees and what happens under the hood. Remember that the end-users are human, and you’re on the right track.

16 September 2016 by andrew