API Advice

API Experts - Why an API Platform is important

Nolan Sullivan

August 4, 2022

This is the first in a series of conversations that we'll be having with leading API development engineers and managers about how they're approaching API development within their organizations. If you or someone you would like to sit down and talk to us, please get in touch (opens in a new tab).

Tl;dr

• API platforms are important for turning API development into a repeatable task for internal developers. Repeatability is critical for exposing a consistent experience to external developers.
• An API platform should handle all the API ops (every aspect of API creation that's not the business logic). Ideally, you should be able to put server code in, and get a production-ready API out.
• It's important to formalize an API strategy early, ideally long before you've launched anything publicly. A little upfront planning can avoid a pile of tech debt later.
• For most companies who sell to developers, APIs are at the heart of DevEx. API platforms can often become the foundation for the larger DevEx team.
• Webhooks are great and should be considered any time there's a "push" use case.

Introduction

Roopak Venkatakrishnan is a director of engineering at Bolt. He is responsible for managing the platform & infrastructure org, as well as the merchant org working to democratize commerce by providing one-click checkout to sellers. Prior to Bolt, Roopak was at Spoke (acq. by Okta) and held senior positions at Google and Twitter.

What is an API Platform

Nolan: You've been working on building out Bolt's API platform. Could you explain what the purpose of an API platform is? If it is successfully built, how does it impact the day to day work of the developers in the company?

Roopak: Yeah, simply put, you want to give your internal developers a great experience when building APIs. And then when you ship an API as a product, you want to make sure the API is holistically thought through; that certain standards are followed throughout. So for example, when an external developer already uses two of your APIs, and then they go on to use a third API that you add later: the errors look the same, the endpoints, the style, the response objects, everything is the same. Generally, once they start working with your product, there should be no surprises. It should look similar and holistic.

So that's one part where an API platform can start helping. It can define a nice pattern for the teams, their product, and make sure APIs are treated holistically across the company. But the second, more interesting part of it, especially for internal teams, is there's going to be a lot of shared components. Things that you don't want to just be using. As an example, you don't want everyone to be building their own rate limiting. It could also be the way you do your docs, it could be guides, it could be, authentication. Every team shouldn't be trying to build their own, it should just be handled for you.

So what I would say is that an API platform should take away all the API ops, basically everything that isn't business logic. Let product developers handle the business logic and then the rest of it is handled by the API platform. An assembly line for APIs, raw business logic goes in, and a productized API comes out.

Nolan: Bolt is an API as a product company, but for lots of companies APIs are a secondary interface. How do you think this changes the need for an API platform?

Roopak: Interestingly, Bolt was not, until recently, an API-driven company. In the sense that, when I joined, one of the things that bothered me about our APIs was that it was just different all over the place. The way we handled authentication, the way we did rate limits, the way our docs were published, our errors, even our observability, across different APIs, it was inconsistent.

To go back to the question, whenever you start providing an API externally, you should have an API platform, and in fact, I think essentially every company does have an API platform. They just don't realize it's an actual thing that they maintain or provide. Usually, it's just two developers, who are kind of managing this in the style guides or runbooks they provide to the rest of the team. It's very similar to say, maintaining Dev & Prod in a company's early days, right? Like, you don't have a developer productivity engineer, when you are eight engineers, but someone still does it 30% of the time, you're just not accounting for it.

And I think you should just be thoughtful and realize that at some point, you do need to start accounting for it and be like, "hey, if we don't do this, we're just going to end up in a bad place." Ultimately, someone needs to be thinking about all these API concerns. Because, here's the worst part of it, this is not something you can go back and fix, right?

Here's the problem with an API, you've released it, and then you're stuck with it. If you want to make a change, you have to beg every single external developer to make the change. Let's just say you add a field to an API without realizing it. Even if it's not documented, you cannot remove it anymore. You know why? Because the minute it's out there, somebody's started using it. So, it's actually one of those things you should think about earlier, more than anything else, because it's no take backs.

How to design API architecture

Nolan: Bolt's public APIs are RESTful, but you also provide webhooks. What're your thoughts on when to use different frameworks?

Roopak: I'm biased. I really, really like web hooks. I would say, in some sense, it's easier to offer web hooks to start off with, than to offer APIs. For example, I have built a lot of stuff off of some companies' web hooks before I started using their APIs. But I do think it's important that you offer both.

I think that web hooks are the push and APIs are the pull. Web hooks should be the way you notify customers about changes in your system. And I believe that almost any system which has an API is going to have changes in their system that they want to notify their customers about.

Nolan: If someone was designing their API architecture today, what advice would you give them?

Roopak: I have an anecdote. Spoke, the previous company I worked at was very interesting. I was an early employee, so we were building from the ground up. From day one, we said, "Hey, we're going to build our entire app on a public API". We're just not going to publish the documentation. So that way, whenever we want to become an API driven system, we are already there. I thought it was a genius idea!

But it was really tough. At some point, we finally realized that it was slowing us down. Because to make changes to a public API, you have to be really thoughtful, and you know, an internal endpoint not so much. So, we tried doing it all public, but it didn't end up working out.

And when we published our actual public API, it ended up that we didn't just take everything from what we already had, they were almost there, but still required changes. So, we published a style guide for public APIs. We said, "Okay, this is how our external APIs are gonna look."

But it was an interesting learning for me, I like that we did it. I just think that, in the end, the style guide approach kind of worked for us, at least for a while, we were small enough that we were not adding too many endpoints. Eventually though, it just becomes harder. Because the more engineers you have, there's just someone who's just not going to follow the guide. Someone is going to make a mistake, and then you need to have a group of people who review the API changes, and then it just starts getting more and more expensive. And that's why API platforms are ultimately necessary.

The API Platform team at Bolt

Nolan: When did you create the API platform team at Bolt, and how did you know it was the right time to start it?

Roopak: Our APIs had existed for the people who needed them, but we weren't necessarily trying to be API-first. But at some point, we realized, you know, the kinds of customers we want, they actually need a good API. We can't publish something shitty, because that's just not going to work. So that's sort of when we realized we needed a formalized API platform.

I think the answer to the right time is, it's like, if you're a company, which is offering an API as a product, you should think about this on day one. But if you're starting to offer an API as a secondary interface, and even if you don't have a dedicated team around it, you should have a group of people who sit and then think through this and then say, "Hey, how are we going to make sure what we expose is what we want to expose." You better think through this, you can't do it after the fact. It has to be before. Otherwise, you're going to pay back a lot of tech debt.

Nolan: How would you define the mandate of the API platform team at Bolt

Roopak: It's actually slowly growing, right? Initially, it was, help our engineering teams do the basics of shipping an API. The team started with building our API tooling: detecting backward incompatible changes, helping with our Doc's, and so on and so forth. That is growing, it's going to be every part of the API lifecycle: key rotation, authentication – rate limiting is its own beast. Every part of API tooling is going to be something that this team does.

But the team is actually formally called DevEx, which is developer experience, because that's the ultimate goal.This team is going to to be the team that interfaces with developers every which way. One day, it may be a fully-fledged developer portal, but today it's mainly focused on tooling to produce consistently great APIs for our customers.

Nolan: What is something your team recently accomplished that you would like to shine a light on? Anything you're particularly proud of?

Roopak: We recently shifted API Doc generation to be much more in line with our code. We're creating the API reference directly from our server code, and having it live right beside the code. I'm very excited about that, mainly because I believe that API docs are super important; it's the end-product people see. You can have a beautiful API, but if you have bad API docs, you're going to be set a couple of steps back.

So we're using Open API 3.0. And we've constructed the whole thing where all the models are separate. Even when you're starting to build an API, you can be like, "Oh, what are the models that we use in our different APIs", and you can actually look them up. Here's the request models that we use, all the various different things. So you can look at them per request, and then you can look at what we use overall. And it's all right beside the code, which I'm personally a fan of because it encourages developers to think about what the customer will see if they are making a change to the code base. You can't just write the API code in isolation and throw it over the fence to someone else to write the API reference and documentation. The end-product is just not going to work well that way.

Nolan: What are currently the API platform team's biggest goals?

Roopak: Oh there's a lot of different things. I'm not going to jump into everything. We're redoing some of our authentication, our key rotation, all of that. The other big goal is improving how quickly we get our partner developers being able to use our APIs. That's one of the metrics that we're starting to track. How soon can a developer get a key and make their first successful API call? And there are so many more tiny things to improve the internal experience for developers that we're trying to get done.

Nolan: If you ask developers building public APIs at Bolt what their biggest challenge is, what would they say?

Roopak: Well, we do ask our developers this, and until recently, it was things like: I don't know where the docs are, how to edit them. It was very complicated. So we picked that up. I think, probably right now, it would be something like, I don't know what our definition for a good API is. And you know, that's something that we need to come up with, and evangelize. Like, we're redoing errors right now as an example. We just haven't published a comprehensive guide yet. And right now, it's much more informal, like you get guidance from this team. So I would say that's probably the next thing. We know how we want endpoints to look. We want to make it clear to our developers what's a good API, so new APIs are in-line with the rest of Bolt's existing APIs.

Nolan: Do internal APIs factor into an API platform team's remit?

Roopak: Right now it's handled differently. We're still trying to figure out how we want to do this. We want our API platform to focus on the external stuff first. That's more important to get in shape. And then internal stuff.

It's interesting, as long as the APIs are performing, even if the API change is backwards incompatible, it's not the end of the world. You can create an entirely new endpoint, and then make every service transition over because you can ping the team and be like, "hey, switch over," and then you can get rid of the old endpoint. Sure, it might take a week, and it's a little bit of a pain, but we're shipping a lot of new external APIs. So we want to make sure we got that in shape. So internal, not yet.

I do think there is an ideal world where I would like an API platform to manage both internal and external. Build a framework, maybe internally use something like gRPC. And, you know, the platform would help other teams generate internal docs for their endpoints, so that any team can easily provide a good DevEx internally. But, we're not big enough for that yet. As you become a much bigger company, you do want internal teams to be treated similarly to your external customers. But, you know, it all depends on the size of the company.

Nolan: A closing question we like to ask everyone: any new technologies or tools that you're particularly excited by? Doesn't have to be API related.

Roopak: Ah, this is a very hard question, because for me, this changes every two days. I try out loads of new tools and then some of them just stick. So I can tell you all the random things that I have been trying out and using recently. I cannot pick a favorite because it's too hard! I got an invite to Arc (opens in a new tab), the new browser, so that is something which I've been playing with. I have switched my terminal to use Warp (opens in a new tab), and I use Graphite (opens in a new tab) for code review. I think Graphite is just friggin great. This is on the tooling side. On the development side, it's been things like Temporal (opens in a new tab) and Tailscale (opens in a new tab). And finally, I mean, this is not a new tool, but I'm moving all my personal projects that I build onto Cloudflare (opens in a new tab), trying to make everything run on workers and pages and whatever else they offer.