Returning informative API Errors

When building an API it’s natural to put most of the focus into building a beautiful golden path for your API’s usage. Builders often don’t like to consider the failure cases. And so errors are often under designed.

But errors in an API are not just an edge-case, they a crucial part of the functionality, and should be treated like a core feature to be proudly shared and documented with users. Failing clearly and concisely is arguably more important than any other aspect of API design.

Errors should:

Be as detailed as possible.
Provide context as to exactly what went wrong and why.
Help humans find out more information about the problem.
Help applications decide what to do next.
Be consistent across the API.

HTTP Status Codes

The journey to great errors starts with status codes. Status code conventions exist to specify what type of error occurred. Make sure that you match conventions, because tools will make decisions automatically based on the status code, like automatically refreshing access tokens on a 403, or retrying the request on a 500.

There are a lot of status codes, but here are the most common ones you’ll see in API responses:

400 Bad Request - The request was invalid or cannot be served. The exact error should be explained in the error payload.
401 Unauthorized - The request requires an authentication token.
403 Forbidden - The server understood the request, but is refusing it or the access is not allowed.
404 Not Found - There is no resource behind the URI.
405 Method Not Allowed - The request method is known by the server but has been disabled and cannot be used.
406 Not Acceptable - The requested media type is not supported.
408 Request Timeout - The server timed out waiting for the request.
409 Conflict - The request could not be completed because of a conflict.
410 Gone - The resource is no longer available and will not be available again.
412 Precondition Failed - The server does not meet one of the preconditions that the requester put on the request.
413 Payload Too Large - The request body is larger than limits defined by server. The server might close the connection or return an Retry-After header field.
414 URI Too Long - The URI requested by the client is longer than the server is willing to interpret.
415 Unsupported Media Type - The request entity has a media type which the server or resource does not support.
416 Range Not Satisfiable - The client has asked for a portion of the file, but the server cannot supply that portion.
417 Expectation Failed - The server cannot meet the requirements of the Expect request-header field.
418 I’m a Teapot - The Network Working Group were particularly bored one day and did an April fools joke.
429 Too Many Requests - The user has sent too many requests in a given amount of time.
500 Internal Server Error - The server has encountered a situation it doesn’t know how to handle.
501 Not Implemented - The request method is not supported by the server and cannot be handled.
502 Bad Gateway - The server, while acting as a gateway or proxy, received an invalid response from the upstream server.
503 Service Unavailable - The server is not ready to handle the request.
504 Gateway Timeout - The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server.

HTTP status codes can convey a lot of assumptions, but they cannot possibly cover all situations, so it’s important to add something for the human developers to see what’s wrong.

Application Errors

An error like 400 Bad Request is generally used as a vague catch-all error that covers a whole range of potential issues. Responding with a status code and telling your user the request was “bad” is not particularly useful on its own. You need to include additional data in the API response to explain exactly what has occurred, including additional data which is useful for:

humans - so that the developer building the integration can understand the issue.
programs - so that program behavior can be correctly triggered of the error.

Imagine that you’re building a carpooling app, and you need to plan a trip between two locations. What happens if the user inputs coordinates that which ait’s not possible to drive between, say England and Iceland? Below is a series of responses from the API with increasing precision:

HTTP/1.1 400 Bad Request

A not very helpful error response, the user will have no idea what they did incorrectly.

HTTP/1.1 400 Bad Request

"error": {
    "message": "Trip is not possible, please check start/stop coordinates and try again."
}

This message could be passed back to the user which will allow them to figure out how to address the issue, but it would be very difficult for an application to programmatically determine what issue occurred and how to respond.

HTTP/1.1 400 Bad Request

"error": {
    "code": "trip_not_possible",
    "message": "Trip is not possible, please check start/stop coordinates and try again."
}

Now this includes data that can help our users know what’s going on, as well as an error code which let’s them handle the error programmatically if they would like to.

So, we should always include both API error messages, as well as API error codes. Let’s take a closer look at the best practices for each

API Error Messages

API Error Codes

The use of an Error Code is well established in the API ecosystem. However, unlike status codes, error codes are specific to an API or organization. That said, there are conventions you can follow to give your error code a predictable format.

Stripe’s error codes have a nice easy to understand structure. We recommend that you stick to this structure, or adapt it to fit your own APIs structure. we could switch to an error object and pop a code property in:

HTTP/1.1 400 Bad Request

{
  "error": {
    "code": "trip_too_short",
    "message": "This trip does not meet the minium threshold for a carpool or 2 kilometers (1.24 miles)."
  }
}

if (error.code === 'trip_too_short')

Complete Error Objects

If you include a code and a message you are off to a great start, but there’s more you can do to turn errors into a handy feature instead of just a red flag.

Here’s the full list of what an API error should include:

Status Code: Indicating the general category of the error (4xx for client errors, 5xx for server errors).
Short Summary: A brief, human-readable summary of the issue (e.g., “Cannot checkout with an empty shopping cart”).
Detailed Message: A more detailed description that offers additional context (e.g., “It looks like you have tried to check out but there is nothing in your cart”).
Application-Specific Error Code: A unique code that helps developers programmatically handle the error (e.g., cart-empty, ERRCARTEMPTY).
Links to Documentation: Providing a URL where users or developers can find more information or troubleshooting steps.

You can build your own custom format for this, but let’s leave that to the professionals and use existing standards: RFC 9457 - Problem Details for HTTP APIs (opens in a new tab). This is being used by more and more API teams.

{
  "type": "https://signatureapi.com/docs/v1/errors/invalid-api-key",
  "title": "Invalid API Key",
  "status": 401,
  "detail": "Please provide a valid API key in the X-Api-Key header."
}

This example of an error from the Signature API (opens in a new tab) includes a type, which is basically the same as an error code, but instead of an arbitrary string like invalid-api-key the standard suggests a URI which is unique to your API (or ecosystem): https://signatureapi.com/docs/v1/errors/invalid-api-key. This does not have to resolve to anything (doesn’t need to go anywhere if someone loads it up) but it can, and that covers the “link to documentation” requirement too.

Why have both a title and a description? This allows the error to be used in a web interface, where certain errors are caught and handled internally, but other errors are passed on to the user to help errors be considered as functionality instead of just “Something went wrong, erm, maybe try again or phone us”. This can reduce incoming support requests, and allow applications to evolve better when handling unknown problems before the interface can be updated.

Here’s a more complete usage including some optional bits of the standard and some extensions.

HTTP/1.1 403 Forbidden
Content-Type: application/problem+json

{
 "type": "https://example.com/probs/out-of-credit",
 "title": "You do not have enough credit.",
 "detail": "Your current balance is 30, but that costs 50.",
 "instance": "/account/12345/msgs/abc",
 "balance": 30,
 "accounts": ["/account/12345", "/account/67890"]
}

This example shows the same type, title, and detail, but has extra bits.

The instance field allows you to point to a specific resource (or endpoint) which the error is relating to. Again URI could resolve (it’s a relative path to the API), or it could just be something that does not necessarily exist on the API but makes sense to the API, allowing clients to report a specific instance of a problem back to you with more information that “it didn’t work…?”.

The balance and account fields are not described by the specification, they are “extensions”, which can be extra data which helps the client application report the problem back to the user. This is extra helpful if they would rather use the variables to produce their own error messages instead of directly inserting the strings from title and details, opening up more options for customization and internationalization.

Best Practices

Handling errors in API design is about more than just choosing the right HTTP status code. It’s about providing clear, actionable information that both developers, applications, and end-users of those applications can understand and act upon.

Here are a few more things to think about as you’re designing errors in your API.

200 OK and Error Code

HTTP 4XX or 5XX codes alert the client, monitoring systems, caching systems, and all sorts of other network components that something bad happened.

The folks over at CommitStrip.com know what’s up.

If you return an HTTP status code of 200 with an error code, you are confusing every single developer and every single standards based tool that you or your clients are ever likely to use, even if you have not thought of it yet.

It’s the difference between thinking of HTTP as a dumb pipe to move data up and down, and you are going to do all of the work of detecting success or failure in your code, or leveraging HTTP as a network protocol with a bunch plethora of built in and extensible functionality which standards based tooling can all build upon for the benefit of everyone.

Single or Multiple Errors?

Should you return a single error for a response, or multiple errors?

Some folks want to return multiple errors, because the idea of having to fix one thing, send a request, fail again, fix another thing, maybe fail again, etc. seems like a tedious process.

This usually comes down to a definition of what an error is. Absolutely, it would be super annoying if you get one response with an error saying “that email is in a bad format” and then another with “the name you sent has unsupported characters” when that information could have been answered at once, but you don’t need multiple errors to do that.

The error there is that “the payload is invalid”, and that can be a single error. We don’t want to lose that context, so lets pop in some validation messages, which is a common pattern.

{
  "type": "https://example.com/probs/invalid-payload",
  "title": "The payload is invalid",
  "details": "The payload has one or more validation errors, please fix them and try again.",
  "validation": [
    {
      "message": "Email address is not properly formatted",
      "field": "email"
    },
    {
      "message": "Name contains unsupported characters",
      "field": "name"
    }
  ]
}

This is a single error, but it contains multiple validation errors, which sets a decent expectation of being able to progress when this is all fixed.

This method is preferred because it’s impossible to preempt things that might go wrong in a part of the code which has not had a chance to execute yet. For instance, that email address might be valid, but the email server is down, or the name might be valid, but the database is down, or the email address is already registered, all of which are different types of error with different status codes, messages, and links to documentation to help solve each of them where possible.

Should You Use a Standard?

When it comes to standards for error formats, there are two main contenders:

RFC 9457 - Problem Details for HTTP APIs

The latest and greatest standard for HTTP error messages. There only reason not to use this standard is not hearing about it, and you’ve just fixed that. It’s new, released in 2023, replacing the RFC 7807 which came out in 2016 but it pretty much the same thing.

It has a lot of good ideas, and it’s being adopted by more and more tooling, either through web application frameworks directly, or as “middlewares” or other extensions.

This helps you avoid reinventing the wheel, and it’s strongly recommended that you use it if possible.

JSON:API Errors

JSON:API (opens in a new tab) is not so much a standard, but a popular specification used throughout the late 2010s. It focuses on providing a common response format for resources, collections, and relationships, but it also has a decent error format (opens in a new tab) which a lot of people like to replicate even if they’re not using the entire specification.

Pick One

There has been a long-standing stalemate scenario where people do not implement standard formats until they see buy-in from a majority of the API community, or wait for a large company to champion it, but seeing as everyone is waiting for everyone else to go first nobody does anything. The end result of this is everyone rolling their own solutions, making a standard less popular, and the vicious cycle continues.

Many large companies are able to ignore these standards because they can create their own effective internal standards, and have enough people around with enough experience to avoid a lot of the common problems around.

Smaller teams that are not in this privileged position can benefit from deferring to standards written by people who have more context on the task at hand. If you are Facebook then certainly roll your own error format, but if you are not then RFC 9457 will point you in the right direction, and implementations make it easy.

Ambiguity in error code?

404 is drastically overused in APIs. People use it for “never existed”, “no longer exists”, “you can’t view it” and “deactivated”, which is way too vague. That can be split up into 403, 404 and 410 for different meanings.

If you get a 403, this could be because the requesting user is not in the correct group to see the requested content. Should the client suggest you upgrade your account somehow? Are you not friends with the user whose content you are trying to view? Should the client suggest you add them as a friend?

A 410 on a resource could be due to the resource being deleted, or it could be due to the user deleting their entire account.

Sometimes being more specific about these different use-cases can help, but sometimes it can leak sensitive information. For example, GitHub prefer to return a 404 for a private repository that you do not have access to, instead of a 403, because a 403 would confirm the existence of the repository. You maybe don’t want people knowing that github.com/acme/your-secret-repo exists, so it’s better to not give out any hints.

Retry-After

As an API designer you want to let API consumers know if they should retry a failure, and if so, when. The Retry-After header is a great way to do this.

HTTP/1.1 429 Too Many Requests
Retry-After: 120

This tells the client to wait two minutes before trying again. This can be a timestamp, or a number of seconds, and it can be a good way to avoid a client bombarding your API with requests when it’s already struggling.

Learn more about Retry-After on MDN (opens in a new tab).

NOTE

Help your API consumers out by enabling retry logic in your Speakeasy SDK.