Architecting APIs with James Higginbotham

Cal: Hi, welcome to the Frontier podcast. My name is Cal Evans. I’m your host, and my special guest today is my friend James Higginbotham. Say hi to everybody, James.

James: Hi, everybody.

Cal: Now, James. You are–I love saying things like this: You are a world renowned API expert. You teach API to different companies. You’ve got a book out. Now, you co-wrote that with our friend Keith?

James: That was the first book that we wrote. Yeah, we published that on Leanpub. This is a new one where I kind of built upon those principles, and it’s published through Addison Wesley. So I’m the sole author as part of Vaughn Vernon’s signature series with Addison Wesley.

Cal: Good plan. Getting rid of Keith. That’s always scum. <laugh>

James: Yeah, that was the plan all along. It worked out great.

Cal: Well, great. We’re gonna talk today about API architecture, because everybody loves to use the term(s) architect and architecture, and actually, speaking of Keith; Keith and I talked not long ago about being a software architect and what that is, but today I want to talk about API architecture in specific and how that differs from certain things and how people can do that. So now, before we get started talking about architecting APIs, though, let’s talk a little bit about what it takes to be an API architect–in your mind, what’s the number one skill a developer has to master before they can start architecting APIs?

James: So just building on what you and Keith spoke about, API architect, that role is starting to merge heavily in a lot of organizations and it builds on the skills of a software architect, but really extends a bit further. The primary reason for that is a lot of API architects tend to focus heavily on systems or on the enterprise and what goes on inside an organization or inside a product company. With an API architect role or API architecture, we really have to extend that out beyond just the organization. So it goes beyond a single system, code base, or ownership of both sides of software. Even if they’re different teams, they’re often times within the same organization, and traditional software development or SaaS development. With API architecture, we don’t control the entire end-to-end solution.

Oftentimes, our API consumers are outside of our control, they’re outside of our organization. So that number one ability is to move beyond a single system, code base, or even end-to-end ownership of a solution, and really start to think about what are the dynamics, and you know, side effects that can come up a as a result of the decisions that were being made about an API and how that can, you know, impact customers, partners, or internal developers, all together. So that, to me, is the number one skill. It’s about distributed computing at an extreme, because you don’t own everything like you might have traditionally owned within an organization.

Cal: Interesting. So with a software architect, your role is really to see the big picture, but with an API architect, it’s really the big picture plus because not only you have to see all of how your internal systems come together, but you’ve also got to see how customers are going to potentially use these. Am I restating it correctly?

James: That’s absolutely correct. So it requires the API architect to have a sense of collaboration, of communication, of being able to extend their thinking and their purview, really, to one that goes outside of the organization, outside the business, outside the product. So that’s that ‘plus’ side of things. And it really is pushing the bounds of what architects have to do today, because we can’t just say, ‘I’m gonna pick this particular serverless tech,’ you know, a technology like AWS Lambda. And ‘I’m gonna combine that with some node and I’m gonna combine it with these’ and everything else. There is a sense of collaboration and communication that has to exist of not just what I’m selecting internally and how those things come together, but how is that facilitating, for better or worse, the discussions and the communication lines between my organization and an individual developer outside of it or another organization that’s working with us as well.

Cal: Interesting. And you know, every time I start talking about high-level development skills and stuff like that, the C word comes up: communication. I got into computers–I have told this story a hundred times–I got computers because I didn’t like people, and I figured computers would be easier to deal with. Much to my surprise, now I’m a professional author, a professional speaker, a professional podcaster, basically a professional communicator because I got into computers. So, it all rolls back to: you’ve got to know how to communicate your ideas and your concepts, and you gotta be able to understand what other people are communicating to you. Well, that’s very interesting, and thank you for that because now we have a foundation to build on. We understand what the terms mean. So once I’ve added communication and being able to see the big picture plus to my talent stack, I’m ready to start architecting APIs. I guess the question is: what’s the difference between designing an API and architecting an API?

James: Yeah. From my perspective, really the design initially focuses on the contract of the API and the associated nonfunctional requirements that we’re often concerned about as an architect. So, thinking about, do I wanna do REST? Do I wanna use GraphQL GRPC? Do I wanna combine them in some way? We’re seeing that a lot in organizations now where they may have a foundation in REST, but they may supplement it with GraphQL to build an experience API or to knit different REST APIs together. They may use some GRPC behind the scenes for performance optimizations, service-to-service communication, or maybe there’s another API style that’s emerging, or maybe you’re in devices. I work with a client that deals heavily in network devices, and they have their own set of networking standards that have existed for a while to support API interoperability between devices that is more efficient, more effective than some of these other API styles.

So whatever you’re selecting, it’s that idea of how, what does the API need to do? What are the outcomes it’s providing, and what is the design style or styles that I need to arrive at and deliver for that contract? So that requires us to have a deep understanding of HTTP and other protocols, as well as the standards, practices and patterns of our organization. Now, when we talk about architecting an API, now we’re really extending that. So not only are we talking about design, but we’re expanding that bubble, that sphere to include the ownership of an API and those ownership concerns. Who’s gonna be using it? Are they external to consumers? Are they partners? How are we onboarding them? How are we securing that API? How are we testing to make sure all those nonfunctional requirements have been adhered to? How does it fit into the overall technology ecosystem? You know, all the choices that we make and that we support internally to deploy, monitor, and manage our systems in production inside of an organization, large or small? How does that API deliver a great or terrible developer experience? How do all those elements fit in?

So it really expands it beyond just the design and the code to all the other nonfunctional and functional interactions of that API. So it’s a much broader set of skills and concerns than just purely designing and implementing an API with your favorite programming language, web framework, or, you know, whatever else that you’re considering.

Cal: Interesting. Thank you for all of that, that kind of helps gel all this together. I do wanna ask, I know my listeners and our viewers probably already understand all these terms, but I’m old and I’m not up on the latest hip terms. GRPC–and I assume ‘RPC’ is ‘remote procedure call’, but what is that?

James: So GRPC originated with Google. It was a solution that enabled Google to leverage code generation, to quickly stand up services and tie them together. It’s built on top of HTTP2, so it supports some bidirectional communication that HTTP offers–HTTP2 offers–as opposed to HTTP 1.1, and it leverages a lot of co-generation, it–for, for those that have been around, like us, that have been around for a while, and they’ve seen CORBA or SOAP–it’s reminiscent of that, but using newer technologies, because there’s a lot of co-generation, there’s a lot of, you know, generation of stubs and skeleton on the server and client sides and things like that. But it started there and it’s gained a lot of traction because of the inherent performance. HTTP2 has some performance improvements.

GRPC leverages that, it builds on top of protobuf, which allows us to define the schema definitions and this, you know, definition format, this protobuf3 format, and then it can co-generate everything and optimize it over the wire. So it’s binary over the wire and then it’s marshaling and unmarshaling on both sides. So a lot of people are using that, or starting to explore that opportunity for microservices inside of organizations, where they want to leverage that back and forth communication in a high-performance manner and not worry about set up and tear down of individual HTTP connections with 1.1, or trying to leverage HTTP 1.1, keep alive in some of those other kind optimization techniques and really drive, uh, performance with HTTP2. So it’s a fascinating protocol. It’s still early. Some have adopted it. Some haven’t, it doesn’t necessarily have great support in the browser. So it tends to be more, you know, within the data center kind of solution.

Cal: Very cool. You mentioned the old schools, CORBA and stuff like that. I’m gonna date myself. Before we had the term API, I was building com objects for IIS using Fox pro and accessing my Microsoft SQL server as my backend. And we didn’t even have the term API at that point. We just, we knew this is what we had to do to get the data off the database and into the web. So good times, good times.

James: Yeah, I remember those times, those were fun. Those were when things were simple, CGI scripts and just queries and turning things into HTML. And we didn’t have all this, you know, really complex systems that we have today.

Cal: Yep, yep, yep. Just static, uh, not static, but good old ASP cloud, not even net, but ASP classic. Okay, so, we talked a little bit, you mentioned some things like API usability and user experience or developer–I assume that you mean developer experience–because I don’t know a lot of users that actually use APIs. What makes an API good? Is it the usability? Is it technical exactness? Is it adoption? Or business value created? Or is it something else?

James: I think it’s a combination of those things. So APIs are really interesting in that they do have the need for an excellent developer experience. So a developer that sits down needs to use it. It’s just like any library for your programming language of choice. You wanna know how to use the library and use it effectively and get up and running as quick as possible and see, is it gonna solve this problem? Is it gonna do it well? So there is a usability for the developer side, what we would call developer experience or DX. Sometimes it’s abbreviated as DevX. There is also usability from the end user, not because the end user talks to the API directly, but they’re talking to it indirectly. So the user experience is driven a lot by the design of the API.

Am I gonna have to make a bunch of small incremental steps? Can I optimize that with one big step? You know, can the UI developers design something that flows better for the user, and can they do that with or because of the design of the API, or are they gonna get stuck in it? So really, when you define what is a good API or what is an effective API, it’s about delivering outcomes for the jobs to be done. So if we think about the end user, what is their problem or situation that they’re in, and what is the job they need to get done so they can produce that outcome? And we do that over and over and over again. So how do we facilitate that with an effective API design? Then also, how do we facilitate the integration of that API by the developers? If we’re just taking our raw data stores and wrapping it with an API and pushing it across the wire, we have the potential of exposing our internal implementation decisions to that developer that needs to integrate with it and perhaps, even all the way to the user.

So going back to some of those previous days of some of those terrible APIs, or terrible user interface designs from the client server days with lots of buttons across the bar and thousands of menu options, to really terrible web interfaces today that we can have. Web design has really taken on, you know, a world of its own. There’s a lot of really experienced human computer interaction (HCI) experts that are driving great user experience efforts inside of web interfaces, which is fantastic to see. But, you know, likewise, we can either have an API that delivers those jobs to be done effectively, or makes it really hard on the end user, or the developer, or both. So we wanna make sure that we keep those in mind, and then we compliment that with looking at the business value delivered. So if we’re purely looking at the business value, we could selfishly design an API that delivers business value, but doesn’t solve people’s problems. And likewise, the opposite of that, we could solve people’s problems, but not deliver for business value or market value.

So there are a lot of elements to go into what is a great or effective API, you know, there’s no one thing that can describe that, but all those things come together and that’s the role of the API architect to find that balance and figure out how all that comes together.

Cal: Very cool, very cool. Yeah and, you know, you’re talking about the old school with all the buttons. You, you really need to stop looking at my code. Okay. It’s just, it’s embarrassing for both of us.

James: The first step is admitting a Cal. <laugh> So, you know, we gotta, we got you to admit it, we’re on our way.

Cal: Oh yeah, now, if you see anything on the web that looks good and has my name on it, I guarantee you it’s because the lovely and talented Kathy got involved. There’s a reason I’m a backend programmer.

James: Yeah, likewise, likewise for me too. Backend development is my passion and it’s been very difficult to build a UI. So that’s why I gravitated toward APIs. I get to focus on the back end, but I have to, for an effective API, collaborate with front end designers to make sure that that API’s gonna meet the needs.

Cal: Yep, absolutely. Okay, I’m gonna ask you one of the secrets of the API industry. Once I have created my API, I have built it. How do I get them to come? How do I make people use it? Or how do I drive adoption? That’s what I’m trying to say. How do I drive adoption for my API? Because, you know, I’ve seen, I used to read programmable web all the time and I said, ‘good APIs come and go’. And I thought, this is really cool. I’m gonna do something with this. And I never did. And apparently nobody else ever did either.

James: Yeah, there’s kind of two sides to it. A lot of organizations that I consult to, I’m helping them with the design and the approach of how to manage their internal APIs, APIs that help internal developers get things done faster. The idea that, you know, we want to consume an API first and build only if necessary.

And, you know, as developers, we always want to kind of build it ourselves. We have that not-invented-here syndrome that we’ll oftentimes battle internally, but within a larger organization, we can’t afford to spend time re-doing everything all the time. Build another API, do the same thing someone else has done. So from that perspective, internal adoption is driven a lot of times by messaging inside the organization, through various messaging channels. Some large organizations have newsletters and they have centralized portals or, you know, automated emails that keep people up to date with things. They even sometimes have dedicated staff that their job is to message and communicate and help people understand what’s out there.

So making things discoverable, using keywords in your API documentation and having great API documentation so that when I’m searching for a keyword, I may not find it in the title; I might find it in the description of API because I’m trying to do something the API’s description says, ‘Hey, I have this capability.’ And then you stumble upon it and take a look at it and figure out if it’s a good fit or not. If you’re releasing, like you were talking about programmable web, these public APIs, that in itself is a discipline that’s very much along the lines of product marketing. APIs require product management, product marketing, just like any other software as a service solution. Sometimes that product marketing can be tailored towards your customers of a SaaS where you have an API and you just wanna let people know that they can automate with it, integrate it with other tools that are common in the marketplace and so on.

But it also sometimes means just, you know, rolling up your sleeves and getting involved with the community. Developers are very, very much by nature, uh, skeptical of anything they see, because they feel like the code that they wrote is gonna be far more performant and effective than what someone else wrote. ‘Surely that person can’t do it as effectively as I can’ or can’t, you know, didn’t do it. You know, ‘it’s not as elegant as what I would come up with.’ So we are by nature skeptical. And so that requires us to establish trust with the developer community, that we’re, that our API, you know, helps to solve that particular area. So anything from helpful articles to insights into how you design things or why certain things were done to the back end details of how it might be implemented, all those things are kind of really interesting to developers, as well as building awareness and just saying, ‘Hey, you know, there’s a lot of awareness that needs to be built about this API, so let me target some articles and get on podcasts and, you know, share links and articles through newsletters and other channels.’

But one of the things that I’ve seen really work well is partnering with platforms like Heroku and others. Those platform-as-a-service ecosystems where they have an add-on ecosystem. SiN grid that, you know, now owned by Twilio, originally started as just a REST API to keep you from having to deal with the email protocols of SMTP pop3 and imap, right? Those are very effective protocols, but they’re very low level. They don’t let me think about ‘how do I send an email campaign or an individual email? A ‘forgot password’ email? How do I just get that up and running in my app quickly?’ SendGrid solved that problem. And one of the first instances that I found them was through Heroku.

I had a Rails app. I was deploying it. I needed to be able to send emails. I didn’t wanna stand up my own email server or use some, you know, try to double it up with my personal email server or something else. I wanted to be professional. I wanted to look like it’s coming from, you know, my own corporation. And I didn’t wanna have to worry about spam rules and things. I wanted to kinda have that turnkey solution built in, making sure that I’m on, you know, appropriate approval lists and stuff so my email isn’t classified as spam. So here I look in the Heroku add-on marketplace, and here’s a company called SendGrid, who are they? Let me look, oh, they’ve got a nice little easy to use REST API. And they even have some libraries in Ruby and I can drop ’em into my Rails app and I can tie it all together.

I enable the add-on, and in Heroku, it sets up all my environmental configuration. And all I have to do is write a little bit of code and my API key’s injected into my real time environment in Heroku, and if I want to go from the free tier to the paid tier, I just enable that and Heroku bills me. And, you know, SendGrid gets a portion of that rev share based on how they’ve established that. So there’s opportunities to gain traction that way until you can, you know, grow the business sufficiently enough to do that necessary product marketing at a larger scale, like you saw Twilio and Stripe and so many other software-as-a-service companies that are powered by APIs do. You know, sometimes that’s not as easy as in the early days. So leveraging those ecosystems is really valuable as well.

And so there’s a combination. So if you’ve got internal APIs, you have internal messaging to do. If you have external facing, you know, developer APIs, you have product marketing there to do. And that adoption takes time. And our friend, Keith, has written a lot about that, ‘cause he’s really good at that: marketing APIs and marketing products in a digital world and getting people’s attention and tracking that and managing that over time. It’s a discipline in and of itself. And so we have to factor that into the API. We can’t just slap an API on a solution and expect everybody just to start magically using it. They need to know why, what they can do. You need to kind of get them thinking creatively and build trust with it.

Cal: Now you mentioned internal versus external. And I really hadn’t even thought about it, but, you know, internal, most people go, ‘oh, you know, we just stand up and holler over the cubicles: Okay. I got a new API,’ but in organizations like Microsoft, you know, they have hundreds and even thousands of internal APIs, and just driving adoption internally can be a task in and of itself. So I had not even thought of that. That’s very interesting.

James: Yeah. There’s a discipline around that too, you know, and the SOA days, in the service owner and architecture days, we had service catalogs, and that’s where we went and found our wisdom. And, you know, did all the stuff that we did then. So that need still exists, but more for an internal perspective, how do I, if I’ve built this service, I’ve built this API, how do I make people aware of it? Discovery of an API’s important. There are often times focused teams that work on standing up a catalog and encouraging teams that have built APIs to register their API on that catalog and encouraging developers to go check that catalog before they build, to see if something already exists. And that exists a lot in larger enterprises. So in a smaller group go into a slack channel, or, as you said, yell across a cube wall, and let people know that something’s out there now, to go get a Wiki page or whatever, and go pull all the information for it and connect to it. In a larger organization, it’s in and of itself yet another discipline, it’s part of an holistic API program to make sure that APIs can be found, onboarded easily, you know, and, and leveraged.

Cal: Now that was a great little offshoot. I kind of wanted to mix things up, but let’s get back to building them now. Something that I’ve talked about: I’ve got a friend of mine who used to be big into APIs. He’s since moved on to crypto, but he had basically an API shopping cart system before they were really popular. And he and I had long talks about how to manage upgrading APIs. And we did, we went back and forth. I know there are some companies–I’m not gonna pick on anybody cuz I don’t want anybody to think, you know, this is bad or this is good–but there are some companies that will simply put ‘V1’, ‘V2’, ‘V3’ in the the URI. Whereas, other companies will use HTTP headers to denote which version you are talking to, or you request which version you’re talking to. Do either of those strike your fancy or do you have another way that you use to handle upgrading what is basically a contract with the application?

James: Yeah, there’s first, just kind of back up a bit and then I’ll talk about that, is there is, you know, an important approach that we have to recognize as API architects. One is that there’s versions, which we can classify as the major versions. Those are the things that the consumers are opting into, and you may use version numbers in URL or use ‘accept headers’ to indicate, you know, based on the version of the resource representation that you’re asking for, in that header. I personally fall into the header camp as more of my preference, but I fully recognize and encounter more URI-based versioning like ‘/V1’, in the URI just because tooling supports it easier. It’s easier to take someone who’s not quite as familiar with the technical side of APIs and maybe they’re using an API GUI client tool to interact with an API, and it just stands out and you don’t have to teach ’em how to set certain headers, certain ways and things, sometimes it just makes it a little bit easier.

That’s the version, and that enables us to manage expectations with that contract. As you said, to make sure that the consumer can know that when I wake up tomorrow, my applications, my integrations should still be running just fine, because if there’s some upgrade that happened on the server side, it shouldn’t be breaking. I have have seen organizations integrate the revision numbers, those minor upgrades from V1 to V2 into the URIs. And I definitely discourage that. What I like to do, and this came from Daryl Miller, from Microsoft who, he and others inside of Microsoft, really spent a lot of time thinking about how do we differentiate minor version changes from major version changes so that we don’t require someone to go make a code change every time we put a new operation in there, that it’s not breaking it, right?

So not ‘/V1.1’ in the URI, but just ‘/V1’. So he differentiated between the idea of “versions”, those major versions, and “revisions”, which are the minor versions. So if you’re into SemVar–semantic versioning–then that would be like a 1.1 to 1.2, where we’re saying we’re not making any significant breaking changes; this is a non-breaking enhancement. We’ve added something, we’ve improved something, we fixed a bug, some combination of things so that we can recognize that. So when you’re thinking about version numbers, you have to think about what your versioning strategy is.

I definitely recommend thinking about major versions as consumer opt-in, where you’re gonna have to convince someone to make a code change and invest the time, money, and energy, into switching from V1 to V2, versus a revision change, which shouldn’t be part of that header or that version number in the URI, but rather the server, the provider is opting into that. They’re making their backend changes to route from version 1.1 of the code base to version 1.2, but the consumer shouldn’t have to make any changes. It’s kind of transparent to them and to recognize that. So, from a consultant-side or a design coach-side and training-side, we see mostly version numbers, not revision, and those are usually in the URI as part of the path. And those are the most commonplace. So I’ve gotten to make peace with the fact that they’re in there, but if I had my preference, I’d probably shift over to the header. We see those occasionally, but not that often.

Cal: Very interesting. My personal preference is the header, but again, that’s just, having looked at different ways to do it, that seems to be my favorite. It’s always interesting to me. And we don’t see this much these days, but in the early days of APIs, you would see an API pop up, and I’m thinking about one specific company, and get really popular and everybody starts using it and then they realize they’ve gotta make changes. And so there’s all of a sudden a ‘/V2’ in there. The original API did not have a version number in it, but now V2 does because nobody thought that far ahead.

James: Yeah, that’s true. And I used to do that. I used to wake up every morning, and there would be a major social media website that we had to have an integration for, for the SaaS I was helping to get bootstrapped. And every morning I’d wake up and it was just a gamble. Is our site still working, or is something broken? Because they wouldn’t communicate that change. So not only did they not have a version in the URL, but they had no communication ahead of time to make us aware when it would happen. And all of a sudden we hopefully would figure it out ourselves.

Worst case is, we’d get a message from a user dropping us a note saying, ‘yeah, you know what, this thing isn’t working anymore, I’m trying to post something and it’s not working.’ So, you know, versioning is an important aspect of the API architecture when you’re thinking about it. Pre-planning for it, as you said, and also planning the communication mechanism and the processes by which you’re gonna make a decision of: am I just introducing a new revision, and have I done a good job of making sure I’m not breaking anything? Or introduce a new version?

Cal: Now I’m not one for throwing softballs here, but this next one is definitely a softball question. <laugh> When we are dealing with APIs, API security–security on the web–has become paramount in the past 10 years. You know, once we figured out, ‘Hey, people can do bad things and we gotta figure out how to stop them from doing that.’ So, with API security is that something that you work on at the architect level, or is this just something that we can slap on later and say, oh, we’re gonna use OAuth, or oh, we’re gonna use JWT or something like that?

James: Well, I’m personally in the camp that security doesn’t matter. No, I’m joking. <laugh> API security is a really interesting thing. By opening up an API, we’re opening up the surface area for vulnerabilities. Not only are we opening things up from the perspective of content that gets delivered [via] HTML and otherwise that we have to make sure we’re careful about, but we’re also encouraging people to automate with us to where we don’t own the user interface. They may be building their own, or just writing scripts and stitching things together and building their own automations we’ve never thought about. So from that perspective, API security is absolutely a must and OAuth, for the longest time, did a great job of telling us how to secure HTTP as a protocol, and websites, and some of those elements were very much applicable to APIs, but they didn’t directly call out API security, and over the last several years, they’ve done that. They’ve developed their top 10 for API security as well. So we can go look at those kinds of things and make sure that we’re protecting ourselves from them.

And there’s third party solutions that we can use to protect from that. So as an API architect, there’s a few things we have to consider. One is, it shouldn’t be at the last-minute. It shouldn’t be the last thing, maybe after the arduous task of documenting our API, do we go, oh yeah, maybe security. Maybe we should think about security. We need to have a plan. We need to have a plan at the organizational level. So for large organizations that have a chief security officer, or chief information security officer kind of role, that office will need to be involved in some of the design of an API.

Am I sharing PII or NPI data? Sensitive data that I shouldn’t be leaking out, or that I should be making sure that there’s an extra level of security for, maybe an extra authorization step or two factor authorization authentication to make sure that I know who I’m talking to should have access to this data. It might be a fine-grained, role-based access control, or what’s sometimes called data entitlements. You know, should I suppress some data because this person doesn’t have the right authorization, or completely eliminate the ability for them to call a particular operation within the API? There’s authorization schemes.

I see too many times people design and build APIs. They’re like, well, what’s the big deal? I mean, I’m just gonna grab my favorite web API framework or web framework that allows me to stand up an HTTP endpoint, throw some JSON, call it done. Oh yeah, you know, I guess we should authorize, well, let’s just use username and passwords. That’s not a good idea. Well, let’s just throw in some tokens, but, you know, we’ll think about how we generate these tokens and manage them and expire them and stuff later. We’re not gonna worry about that. And all those things lead to the problems you see in the news where an API was compromised because it was considered ‘security by obscurity.’ Well, no, one’s gonna pay attention to our API. No one’s gonna sniff our traffic. No, one’s gonna look at our mobile app and see what APIs it’s calling. So why do we need to secure this? Or, or, well, we’ll secure this API one way and this other API operation another way, and we’ll have all these mixtures of ways of doing things. I’m sure that’ll work out as well. And then we’ll just auto-increment our IDs from our database and expose those. And we won’t really do any authorization check. We’ll just give everybody a list of the resources by ID that they’re allowed access to. So they can see one, two, and four, but surely they’re not gonna try to access three. So we don’t need to worry about the fact that they’re not the owner of number three, you know, and we wanna use UIDs or some other kind of way to generate IDs that are non-sequential, which is an OAuth item that they’ve called out. I think it’s the first one, in fact.

So, it has to be thought of early and often, and that’s where the collaboration, we come back to that yet again, the API architect has to bring in the right people, make sure that all lines up, make sure that we understand: how are we gonna secure the API? What authorization rules do we have? Is it tied to the way that we manage users today? Or do we need something more by ingrained? You know, I’ve worked with universities, non-governmental agencies, organizations, and others, and they’ll sometimes take their login credentials in their active directory systems or their–let’s go back a little bit–their NEIS systems, or whatever else that manages their users, and that stuff’s still out there, you know, <inaudible> and other types of protocols are out there. They’ll just take that and tie it to an API, but then they get into trouble because a role for accessing applications is different than a role that accesses raw data and capabilities that your API exposes. So there’s, you know, a lot more considerations that we have to think about there. So security has to be factored in as part of that overall architecture from the design early on, before we get to the code or in tandem with our coding efforts.

Cal: And you are welcome to do the sequential UUIDs and stuff like that. I mean, if you don’t wanna learn the lesson of Panera Bread or, way back in the day, AT&T, you know–hey, knock yourself out, go do these things, you know. But you know, a well-placed one is gonna have to be–security’s gotta be upfront. And I totally agree with you: it’s gotta be thought of upfront and, like you said, you’ve gotta involve the security team, if your company has one If your company doesn’t have one, by all means, bring somebody in to help you think these things through. You know, I’ve seen so many people, you were talking about, you know, we’re gonna do this here, and this there, roll-your-own security. They’ve got specific patterns for this stuff for a reason, and we know what works best. So if you don’t have an information security team, bring a consultant like James in to talk and talk it through, and get somebody to help you figure these things out. Somebody that’s done this before.

Hey, last question: are there any considerations an API architect needs to take into account to make sure that the final product is scalable? I mean, I can’t tell you the number of APIs that I have whipped up, you know, and got it out there and then I’ve seen, well, then it gets a little bit of popularity, and I wake up in the morning, my server’s down ‘cause–great–it just got a little too popular and I didn’t bother to scale my server, or I did not bother to scale wide. So, what do architects need to do in advance so that they can scale up and that they can scale back down? Cause you know, you might need to scale up for Christmas, but in January you need to scale back down.

James: That, so…in the larger organizations, we oftentimes have teams–it’s labeled either DevOps, or infrastructure, or operations—that are responsible for handling either on-premises data center resources, cloud resources, or some combination. So, coordinating with them and making sure that all the proper nonfunctional requirements are factored in. So that’s a key element of it. Not only do we wanna have automated tests to test, to make sure that our code’s doing the right thing, and that at the contract level, our API’s doing the right thing, but we also need to kind of test the performance, the infrastructure, the scalability, the monitoring, the management elements of our APIs. So, all of those factors go into it.

One of the things that I teach on, and that’s covered in my recent book, is this idea of ADDR: align, define, design and refine. How do you approach your API design? And part of that is aligning on how the API needs to function. What does it need to do? What are the jobs to be done? And in addition, what are those non-functional requirements we need to think about? Are there scalability factors we need to consider? Are there other factors that we need to keep in mind the elasticity side, as you said, not only scaling up, but scaling back down for cost management. So all of those different elements and aligning on that and then defining what the API needs to do, what are the operations? Designing it, according to the API design style, you need to pick REST, GraphQL, GRPC and so on, and then refine that. So,continuing to seek feedback and incorporate that into the overall design. In that refined stage, that’s when we would start taking our code and start to really push it to the limits and see what its scalability factors are, and that may have an impact on your design.

So keep in mind that an API design is really forever. It is hard to move someone from a version one to a version two, and it’s terrible to experience as a consumer. What I had mentioned earlier with that story about waking up and stuff stops working because somebody made a breaking change because of some performance issue, let’s say, or some security issue that we didn’t think about. We need to break everybody’s integration so we can fix this, you know, critical thing. That’s painful and you can lose customers. You encounter customer churn really fast that way. So having a step in your design process where you’re able to factor those nonfunctional requirements in and reassess your design and say, you know, if we adjust this, we can actually bring this. We can improve our performance for consumers to their benefit and reduce our cost on the back end or, you know, all those other factors and taking that into consideration.

All those things have to be thought of, not just what does the JSON look like? What’s the URI look like? Which HTTP method am I using? Do I wanna use GraphQL? What does that resource graph look like? That all is important, but those non-functional requirements, like performance and pricing and scalability and everything else has to be factored in.

Cal: Very cool. Hey James, I wanna take this moment to say thank you for spending some time with us today. I have learned so much. I’m gonna throw a link to your book, your new book, in the show notes and a link to the one that you and Keith released. I assume that one’s still available?

James: Still available. Absolutely.

Cal: I’ll put links to both of those in there, because I know that they are both excellent tomes, but I wanna take, you know, you took some time outta your very busy schedule to be with us today, and I wanna say thank you. Audience, thank you for taking time out of your schedule, for being, to be with us here today. I hope you found this entertaining and informative and hey, do me a favor. If you like what you saw, go out to your favorite podcast catcher and leave us a rating, give us five thumbs, up five stars, whatever it is. If there’s some way that we can do this better, drop me an email, [email protected]. Let me know. I’d love to hear from you on how we can make this more usable for you. Thank you so much. And we’ll see you right next time, right here on Frontiers.