Navigating OpenAPI, TypeSpec, and API-Drift in the "Post-OpenAPI Era"

Net API Notes for 2024/09/12, Issue 244

Recently, there was a good, old-fashioned kerfuffle (or your online brouhaha) in the API space. In this edition of Net API Notes, I'll unpack Daniel Kocot's provocative positioning of API Design in a "post-OpenAPI era", contrast with the compelling response from Fabrizio Ferri-Benedetti, and end with how both don't name the much larger boogeyman haunting centralized API programs heavily invested in OpenAPI descriptions. That and more in this edition of Net API Notes!

Kocot's "Post-OpenAPI Era"

In late August, Nordic API's Bill Doerrfeld interviewed Daniel Kocot on API Design in the Post-OpenAPI Era. When Kocot was asked how he viewed the role of the OpenAPI specification in this era, Kocot responded:

"In this post-OpenAPI era, I see standards like OpenAPI as automatically generated artifacts. Nothing more. Let’s be honest: nobody should really create or edit OpenAPI, JSON schema, and other files directly. The reason is that the error rate is very high despite linting and validation in today’s IDEs. But there is no question in my mind that the standards need to evolve."

If your API life cycle is code first and OpenAPI descriptions are seen as primarily end-user documentation, deployed to a portal by the API developers and never touched again until the next release, then (perhaps) this attitude makes sense - why have fallible humans mucking up what should be an automatically generated abstraction of the codebase? It suggests a hyper-focus on rapid execution, where anything non-essential to code shipping is inefficient and ripe for automation. That attitude has been vogue in executive boardrooms for the past few years.

Further, Kocot championed TypeSpec as an advanced alternative with better support for subsequent automation, like code generation. (TypeSpec was previously known as both ADL and later Cadl [pronounced like 'cattle']. It is particularly capable at describing JSON data "shapes" for client libraries.)

Some organizations treat their OpenAPI descriptions as a post-code endeavor - something posted to the catalog for client discovery, kickstarting tests, and (maybe) initial client generation. This is especially pernicious among those organizations with a large number of existing APIs being documented and cataloged consistently for the first time. However, that process overlooks an essential aspect of the API-Design-First camp: stakeholder collaboration

In code-first development, OpenAPI descriptions are useful after development has occurred.

Ferri-Benedetti's Pro-Collaboration Rebuttal

Regarding Kocot's comment, Fabrizio Ferri-Benedetti wrote a post, TypeSpec Reminds Us Why OpenAPI Exists in the First Place. The entire piece is a thoughtful read, but this part, in particular, stood out to me:

"-while I sincerely hope that TypeSpec will triumph and extend to the point that it’ll ease the creation and editing of OAS while maintaining decent docs support, a part of me thinks it’s yet another attempt at circumventing the complexities of human behavior and the sociology of software development through a secret idiom that caters to developer sensibilities."

API-Design-First was promoted, often among API-as-product proponents, as a methodology for engaging stakeholders in meaningful conversations. For this group, APIs are not just technical byproducts but social and collaborative norm forming. Talking about an API required a medium of exchange - something technical enough to capture the major decisions around things like endpoints, payloads, and error conditions while still being approachable enough that non-technical stakeholders could still meaningfully engage with it. 

The point of the OpenAPI description in this phase of the life cycle is not to be error-free. Rather, it serves as a shared language for stakeholders—developers, product owners, and others—to discuss what an API could be before committing to implementation (and the messiness that might entail). This minimizes costly rework later on and ensures alignment early in the process. 

Following an API-design-first methodology, OpenAPI descriptions provide utility throughout much more of the API lifecycle.

American architect Frank Lloyd Wright said:

"You can use an eraser on the drafting table or a sledgehammer on the construction site."

OpenAPI descriptions enable "erasing" and redrawing of APIs during this collaboration step. Once stakeholders are in agreement, that same description then serves as a guiding reference for teams to build to. Finally, after launch, we can still enjoy all those OpenAPI benefits like catalog discovery, test generation, and client creation.

At their core, these opposing perspectives are not just about whether to use OpenAPI or TypeSpec. It’s a philosophical divide about the role that API descriptions play within the software development life cycle. Is the OpenAPI description merely documentation that should be auto-generated, or is it a design artifact that facilitates communication, iteration, and planning? The code-first crowd emphasizes speed, automation, and reducing human intervention, whereas the design-first camp prioritizes thoughtful planning and dialogue before a single line of code is written.

In the proper context, both camps could be correct. But neither addresses the long-term issue of API-Drift. 

But What About API-Drift? 

API-Drift refers to the gradual divergence between an API's documented behavior and its implementation. In either the code-first or API-design-first paradigms, there's a moment when the OpenAPI description is "done" and versioned independently from the referenced codebase. This is a good thing; sometimes, code changes will occur that won't impact the interface. At other times, like clarifying the summaries or examples, there may be a need to update the OpenAPI description independent from a code push. 

The problems occur when there are subtle changes to the code base that aren't subsequently reflected in the OpenAPI description. Over time, these can accumulate, leading to increased maintenance costs, unexpected integration failures, and declining developer trust. 

A disciplined API governance process that requires regular reviews on code deployments helps, but that is still reliant on comparing the OpenAPI description at discrete points in time. In a perfect world as an API program lead, I would like to see a run-time comparison, per request, between what the API is doing and what it purports to do. In the past, there was concern about introducing additional analysis overhead into the request path, which already had SLA expectations in the sub-200 millisecond range. Any out-of-band strategy, like reviewing and comparing logs post-request, ran afoul of sensitive information storage rules. 

Some products will sample an API, compare the results with a schema, and notify the appropriate folks when there's a deviation. But when it comes to unsafe operations - like transferring money across users or deleting accounts - I'm not sure how you could confidently do that sampling in a production environment. And if you aren't tracking drift in a production environment, are you really monitoring drift? 

It is a tricky problem for both code-first and API-design-first camps. Have you dealt with this in your API programs? If so, let me know how you approached it and what you discovered in the comments below - I'd love to get a better handle on what folks are doing about this growing issue.

Milestones

 Wrapping Up

Once again, I want to thank Emma Kriskinans and Indu Alagarsamy for their August guest contributions. Whether it was What Marketing Can Teach Devs About API Strategy or Surviving API Breakage, it is a privilege being able to host and promote some compelling, field-expanding thought. 

While they were putting in the hours on the words-side of things, I used my available hours to rebuild the gathering and analysis scripts for the API Jobs database, data that has contributed to posts on hiring trends and position norms. While I do much of that locally on my personal laptop, datasets and AI summarization still accrues costs. Paid subscribers to Net API Notes help cover these costs. In return, paid subscribers get access the entirety of ¡APIcryphal!, the true(ish) stories of APIs past. Those that have signed up for the email newsletter can upgrade to a paid subscription for as little as $8 a month. If you're in a position to do so, please consider supporting this endeavor and help me continue creating impactful and unique API writing. 

Thanks for considering it. Till next time,

Matthew [@matthew (Fediverse), matthewreinbold.com (Website)]

Subscribe to Net API Notes

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe