By Matthew Reinbold in Newsletter — Jan 23, 2020

Net API Notes for 2020/01/24 - Mailbag! - Issue 119

At my 'day job', I've been busy with year-end/year-beginning performance reporting. I thought that I'd take it easy by doing a mailbag roundup. However, that turned out to be more involved than trolling the internet!

As they say where I'm from, "Uff da!".

Let's get to some of your questions.

MAILBAG QUESTIONS

BEGINNING API DESIGN PRINCIPLES

Brandon Rohrer is a machine learning and robotics gent employed at iRobot. He recently tweeted a handful of questions beginners should consider for an API.

He listed:

Who will use this?
What will users know?
What will users' workflow be?
What are the major steps in each workflow?
What are the API functions or entry points?
How are they called, and what are their outputs?

Being highly technical but new(ish) to APIs, he was curious for additional thoughts. Vicki, a Twitter friend and sanity checker on my recent AI Governance Roadmap talk, invited me to comment.

Brandon's questions hint at a general design principle referred to as "outside-in". It reframes the perspective. It is not what data happens to be available, what the framework makes convenient, or what got the biggest applause at the last developer conference. Designs begin from the perspective of the consumer.

The characteristics of what a consumer deems "good" API design was brilliantly articulated by Joshua Bloch, circa 2007. In that video, which encompasses all APIs, not just the HTTP ones, Joshua states an API should be:

Easy to Learn
Easy to Use, Even w/o Documentation
Hard to Misuse (it's subtle, but this isn't the same as easy to learn)
Easy to Read and Maintain the Code that Uses It
Sufficiently Powerful to Satisfy Requirements
Easy to Extend (another subtlety often missed in v1)
Appropriate for the Audience

Achieving each of those things requires careful product management. I've written about some of those considerations when designing an API. I have a whole shelf full of books that go further into the nuance of API design. However, all of that detail is for naught if the design doesn't begin with the user's needs.

WHO CHOOSES API STYLES WITHIN A COMPANY?

A former co-worker reached out via LinkedIn. They had gotten a new job at Google and had a question:

"I continue to follow your LinkedIn posts and musings on various API-related topics. I've discovered that Google largely follows the RPC style HTTP based API pattern...

"It seems like most companies exclusively endorse one style over another. I'd be curious to hear your perspectives on why that is? Shouldn't developers be picking the right tool for the right job?"

Several years ago, I compared unmanaged API design across an internal development community to the Brazilian favelas.

"Picture a favela, or Brazilian shantytown. Each building successfully fills the role of a house. It provides shelter and security. Like an API, it is encapsulated and enjoys a certain level of cohesion. If all we were concerned about was the functional requirement to have lots of houses there we can look at the resulting hillside patchwork and call it a day."

"But expecting a new product to navigate this is unrealistic; those new to this environment will quickly be overwhelmed and lost. There has been no thought to things like accessibility or navigability because there was no coordinated vision for the holistic environment. There was no HOA or community planning. We lack a larger vision for the neighborhood."

In a favela, the ecosystem is unregulated. Yes, individuals may have a tremendous amount of unfettered freedom. However, this comes at a cost to the whole; this design pattern has significant stability, scaling, and sustainability problems. When a company like Google encourages (or even enforces) a particular API style, like gRPC, it is an attempt to avoid the integration overhead of inconsistent approaches.

The opposite of the favela, in this urban planning metaphor, is Paris. When you think of Paris, I'm sure there a handful of landmarks that come to mind, like the Eiffel Tower or Arc de Triomphe. However, there are also the wide, tree-lined boulevards and sky-scraper-free limestone blocks. The way Paris looks and the architectural harmony that many equate as picturesque is no accident. It was designed.

For example, residential buildings were limited to 4 or 5 stories early in the late 18th century. To squeak out some additional space, entrepreneurial owners began building the attics with walls sloped 45 degrees or more; shallower slopes robbed the owner of valuable floor space. These roof lines became known as "Mansard rooves".

(Photo taken by Thierry Bézecourt, CC BY-SA 3.0)

Some might say that Paris has draconian restrictions on what its building owners can and can't do. They might ask why an owner can't use whatever style is uniquely suited for their circumstances. However, the uniformity in construction is what many people, whether they know consciously or not, cite as being attractive about the city.

Maybe the things that Paris values aren't appropriate elsewhere. Rules and regulations should take into account the culture and context of where they are being applied. What works for Paris might not, for example, work for Hong Kong. For a city of APIs to be viable, however, someone should be thinking in terms of these meta-design patterns.

STRINGS, ARRAYS, AND GRAPHABLE JSON

In the last note, I discussed the GraphableJSON specification from Stephen Mizell.

Layering semantic meaning onto JSON increases predictability. Reducing coupling is always a good idea. One of the things I took issue with, however, was the recommendation to turn strings to arrays. On Twitter, Stephen asked me to elaborate.

The github documentation lists an email property as an example. To elaborate:

The first API version models a user or account structure with only a single email address per account
After being put into production, there's a need to support multiple email addresses ('work' v. 'primary', 'main' and any number of aliases, 'primary email' v. 'additional authorized user', etc.)

Changing a single value to a more complex structure to accommodate this new requirement is a breaking change. The graphablejson guidance, therefore, is to avoid declaring strings but an array or 'stream'. From the specification:

"Graphable JSON denotes a property as a relationship to a stream of values"

By defining the email property as an array rather than string upfront, any number of emails could be supported without a breaking change:

{
  "email": ["jdoe@example.com", "jane.doe@example.com"]
}

I do agree that using an array to allow for null-to-many values avoids a breaking change. However, we've lost meaning. If I need to email a user, is the first value as valid as the last? What if I need their work email?

When designing an API, my team and I spend a tremendous amount of time on naming things, often to the API's producers consternation. However, while words like "amount", "code", "status", and "type" might be evident to the people making that API, the consumer needs more. Which amount or what status are we talking about? Field names that appear in an API should be self-descriptive. Generic words like "type" don't carry much meaning on their own.

The "email" field could fall into that same context. Suppose, when we started, our data model assumed we only ever needed a single email. Modern life tells us otherwise. One solution, proposed by graphablejson, is to place all emails together into an array. This risks consumers developing assumptions as to the meaning or importance based on the order in which values appear (aka Hyrum's Law, another form of coupling).

Another approach is to use specific field naming. Additive changes to an array aren't breaking changes. Version 1 might start with:

{
  "email": "jdoe@example.com"
}

If we introduce an email alias and a work address we might model it like:

{
  "email": "jdoe@example.com",
  "email_alias":"jane.doe@example.com",
  "email_work":"jane.doe@acme.com"   
}

There's always the chance I misunderstood the purpose behind this specific graphablejson point. However, I tend to favor additional name specificity in the name of comprehension than arrays of similar, but not quite the same, things. Done correctly, adding additional meaning would not represent a breaking change.

MILESTONES

Service meshes were the buzzword of 2018. It is interesting to me, then, that buried at the bottom of a recap of ServiceMeshCon 2019 is the following tidbit: while 70% of the crowd admitted to experimenting with a service mesh, less than 10 attendees indicated they were using a service mesh in production.
It's official: OpenAPI's 3.1 version will support a JSON schema object.
Twitter has turned off SPDY support and changed how token and secret management works for its APIs.

WRAPPING UP

Looking for an in-person API event? Check out NetAPI.events. If you know of a meetup, hackathon, or conference that isn't there but should be, let me know by replying to this message.

Further, if you're in the DC area on February 4th, I'll be presenting "Why AI Success is Built on APIs". (This was the talk I was to give in January, but was postponed due to a snowstorm. I'll have slides an a transcript posted sometime later.)

Finally, thank you to the Patreons who support my writing. Watch my Patreon account for some upcoming laptop-sticker thank-you goodness.

Till next time,

Matthew @libel_vox and matthewreinbold.com