API Governance: It's Not About the Technology - Solving API's Collective Action Problem

Net API Notes for 2024/12/03, Issue 247

As a software consultant allowed to peek under the hood of large enterprises, I've noticed a pattern, regardless of industry. Technical leadership recognizes that there are APIs at every turn. The stilted efforts, incompatible security, and bespoke consumption patterns aren't delivering the agility and business innovation that they hoped for. Worse, rather than leveraging each other's work, teams keep creating variants (part 1, part 2) of something that already exists. They experience all the downsides of a complex, distributed system but enjoy fewer and fewer of the benefits.

Efforts to herd the cats are project-based and artifact-driven, like "Go create some standards", or "Graft linting onto the existing SDLC". And, almost universally, these efforts end up underwhelming and abandoned upon the next reorganization. 

IT shops continue to attempt to solve "people" with the only arrow in their quiver: technology. In other words, they are inadequately equipped to address a collective action problem

The Collective Action Problem: Yet Another Thing STEM Doesn't Teach

Collective Action refers to the efforts of a group of individuals or entities working together to achieve a common goal or address a shared issue. This is often in cases where a person working alone would be unable or less effective. 

Subsequently, a collection action problem is a situation in which multiple individuals or groups would all benefit from working together to take a certain action, but that work has an associated cost. This cost incentivizes these same people not to contribute despite leading to worse outcomes for everyone involved in the long term. 

Adopting API standards across a business unit or organization is a collective action problem. While everyone within the organization stands to benefit from the consistent application of API standards - positive outcomes such as simplified integrations, increased automation opportunities, and enhanced developer experience - adopting these standards comes with upfront costs. Teams may need to invest time and resources just to comprehend the new expectations, tools, and processes that may be part of a broad, "big-bang" standards initiative. And that is a separate body of work from updating their APIs, revising documentation, or changing workflows to comply with the new standards.

The challenge is that these costs are immediate and borne individually by teams, while the benefits, such as easier integration or long-term scalability, are distributed across the entire organization. This imbalance can lead to reluctance or outright resistance from some teams, especially those under time and performance pressure (which, in my experience, is practically all teams these days).

API design standards, documentation expectations, versioning and deprecation management, security compliance, monitoring and metrics, test coverage - the places where teams may take the expedient action rather than the right action are numerous

Top-Down, "Make Them" Approaches Rarely Work

Most people lack familiarity in approaching a collective action challenge. When technical leadership fully grasps the scale of their organization's API challenge, the instinctive reaction is to side-step the tricky, communal bit and find the single person or group with the hierarchical power to enforce compliance. The thinking goes: if only someone could "make them" (where "them" refers to the teams impacted by API governance) fall in line, the problem would be solved. This theoretical approach appears much more efficient - a top-down mandate eliminates debate and guarantees alignment. 

However, in practice, it rarely works.

The first step in this wishful thinking is identifying who should wield the proverbial stick. Is it the Office of the CTO? The Cyber Security group? Perhaps Enterprise Architecture? Each of these entities is perceived as having some measure of power over technical decision-making, making them seemingly ideal candidates to impose standards. But the reality is far more complex. These groups often lack the direct operational control needed to enforce mandates effectively and, more importantly, the incentive structures to make such efforts sustainable.

The allure of the "make them" approach lies in its simplicity. But simplicity is a deceptive siren call in the realm of distributed systems. Standards, once defined, do not implement or enforce themselves. The real work begins after the document is written: evangelizing its value, defending its relevance when challenged, and owning its evolution as organizational needs change. These are continuous, labor-intensive activities that require deep empathy for the teams affected, political acumen to navigate competing priorities, and a steadfast commitment to the larger organizational vision.

Let's be real: while plenty of individuals are happy to take on the initial research phase - drafting a standards document addressing their pet bugaboos, presenting it to stakeholders, and even getting it ceremonially approved - watch how quickly enthusiasm wanes when the conversation shifts to ownership. Who will champion this artifact long-term? Who will handle the pushback when teams inevitably say, "This doesn't work for us"? Who will make the judgment calls on ambiguous edge cases and adapt the standard as technology evolves? Silence usually follows these questions.

The reason for this hesitance is clear: owning API standards over time is not glamorous work. It is rarely the kind of achievement that leads to promotions or public recognition. It requires a patient, persistent effort to build trust and demonstrate value over months or years. 

The result is a vicious cycle. The mandate to create standards is issued, but the follow-through falls apart. Teams perceive the effort as another ivory tower initiative, disconnected from their day-to-day realities. Adoption lags without ongoing advocacy and support, and the standards document becomes just another entry on the internal CMS to succumb to bitrot.

Ultimately, the "make them" mindset ignores the root of the problem: a lack of alignment, not a lack of enforcement

Collaboration and Co-Ownership Is The Alternative

No company sets out to create a culture of resentment and resistance. However, to drive the kind of innovation and agility all companies say they want with their API investments, we have to start thinking about collaboration and co-ownership. 

It means providing API development teams with the tools, support, and incentives to align with organizational goals (Is it the number of integrations that is celebrated? Or an increasing average of clients-per-API, signaling reuse, and thus greater ROI?). It means creating feedback loops that ensure standards evolve based on practical realities, not just theoretical ideals. It also means acknowledging that governance is a long game, one that requires dedicated champions who are willing to roll up their sleeves and do the unglamorous work of making collective action successful.

Who Makes Collective Action Work? 

When solving the collective action problems inherent in API governance, the solution must begin with identifying the right individuals or roles to lead the charge. By their very nature, enterprise architects are uniquely positioned to take on this role. Their traditional responsibility has been to look across the entire organization, reconciling siloed practices and providing a unifying vision for technology strategy. However, successfully driving API governance is not merely extending their existing job; it requires enterprise architects to evolve.

The Traditional Role of Enterprise Architects

Enterprise architects have historically been tasked with defining the technical blueprints and roadmaps for an organization's technology landscape. They are often seen as "the planners", working at a higher altitude than the developers in the trenches. Their focus has traditionally been on long-term alignment between business objectives and technical infrastructure, balancing innovation with operational stability.

This role has come under scrutiny in recent years, as enterprise architects have been caricatured as "whiteboard astronauts" or "ivory tower" strategists—too detached from the realities of day-to-day development. While this critique is sometimes unfair, it underscores a challenge: enterprise architects are often more accustomed to creating frameworks than fostering the relationships needed to make those frameworks stick.

Why Enterprise Architects Are Uniquely Suited

Despite these challenges, enterprise architects possess several qualities that make them strong candidates for leading the charge on API governance:

  • Cross-Boundary Perspective: They have the broadest view of the organization's technical and business landscape, allowing them to identify dependencies, overlaps, and opportunities for alignment.
  • Strategic Influence: Their mandate often includes engaging with leadership, making them well-positioned to secure executive buy-in for governance initiatives.
  • Systems Thinking: They are trained to think about systems and long-term impacts with an explicit mindset for addressing collective action problems.

But these strengths alone are not enough. To succeed in this new role, enterprise architects must step out of their comfort zone and develop the interpersonal and facilitation skills to rally teams around a shared vision.

The Missing Skillset: People Skills

The most effective governance efforts are not dictated—they are nurtured. This is where traditional enterprise architecture falls short. Solving a collective action problem involves messy, human-centric work: bridging the gaps between teams, navigating political sensitivities, and finding ways to inspire hesitant or even distrustful leaders to participate. It requires the ability to listen empathetically, communicate persuasively, and build trust incrementally.

Training for enterprise architects in this area should include:

  • Facilitation Skills: Leading workshops to align teams on shared goals and priorities.
  • Conflict Resolution: Navigating disagreements about ownership, approach, or priorities without alienating stakeholders.
  • Change Management: Understanding how to guide organizations through the discomfort of adopting new practices.
  • Storytelling: Framing API governance in terms of tangible benefits to teams and the broader organization.

In other words, enterprise architects must complement their technical expertise with "soft" skills that turn abstract plans into concrete outcomes.

The Enterprise Architect as API Champion

To make collective action work, enterprise architects must reimagine their role. They need to become API champions, not just by defining standards but by owning the ongoing work of governance. This means:

  • Evangelizing the value of consistency and reuse.
  • Building relationships across organizational silos.
  • Acting as mediators when conflicts arise.
  • Leading by example, showing that collaboration leads to measurable results.

That job is no small undertaking. It will require patience, persistence, and a willingness to embrace the unglamorous work often not equated with a "technical" role. But with the right support, training, and mindset, enterprise architects can transform their traditional jobs into ones that reconcile siloed practices and turn the collective action problem into an opportunity for sustainable, long-term, API success.

Getting Started

Addressing the collective action problem will be a set of incremental experiments marked by the occasional setback. However, a journey of a thousand miles starts with a single step. If solving the collective action problem is something that you'd be interested in learning more about, here are a few additional links to get you on your way:

  • Seven Skills to Save Software, a presentation (and expanded transcript) I published two years ago summarizing my thoughts on the matter.
  • If you'd like a book, I'd recommend Leading Change by John P. Kotter; while numerous books on this subject, I found this one the most approachable and actionable.
  • Finally, the TED Radio Hour recently had a podcast about living longer and better, which is good on its own. However, in the final ten minutes (or so), subject matter expert Dan Buettner shares what techniques he's found helpful in rolling out health change initiatives in more than 70 U.S. cities. What is discussed there is also relevant to this conversation.

Milestones

Wrapping Up

It's December! Holiday heartburn, travel (planned or otherwise), and the rush to finish year-end work always jumble the editorial calendar. With #250 looming, it feels like something special is called for, but I'm not entirely sure what that might be. If you've got any suggestions, I'm all ears - ping me at hello at my name (.com). 

Till next time,

Matthew (@matthew in the fediverse and matthewreinbold.com on the web)

Subscribe to Net API Notes

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe