How Technical Theories Spread

Hey, hello, and howdy! It has been a minute.

I want to, again, thank everyone that provided the patronage, encouragement, and attention that helped this newsletter reach its 200th issue. That archive represents a tremendous, multi-year effort spent wringing sense from a slippery subject.

Some new tools, techniques, and trends do propel us forward. However, as my experience has borne out, not every "innovation" is beneficial. Ignoring new developments isn't the answer; nobody wakes up in the morning excited to march in place. But we do need to access what we're being pitched on its merits to ensure it's actually in our best interest. The other option is uncritically adopting whatever a vendor or tech influencer promotes. While "nobody ever got fired for buying IBM" may have once been true, the world has changed.

My guess, given you subscribed to this email, is that you are the type of person who cares about their craft. You want to work smarter, not harder. In that vein, how do we separate the wheat from the chaff? How do we recognize those impactful architectural ideas versus those that aren't worth our time?

THE ODDLY INFLUENCED PODCAST

One approach comes from Brian Marick's excellent podcast, Oddly Influenced. Brian is a co-author of the Agile Manifesto and an Agile consultant. The podcast explores how ideas from outside of software can be applied to software. Brian's first topic is why certain software practices become popular.

Brian begins the series by recalling how, around the turn of the century, there was a bandwagon around Test Driven Development (hereafter referred to as TDD). This groundswell struck Brian as odd; after all, he promoted the idea of writing tests a half decade earlier in his book The Craft of Software Testing (1994). Furthermore, there was plenty of other prior art; Dijkstra promoted a similar approach in his 1972 book, The Humble Programmer. So what was the confluence of events that launched a thousand conference talks and enterprise engagements in the aughts?

At the time of this writing, the following related episodes have been syndicated:

• Viruses, Cancer, TDD, and "Packages"
• jUnit and What Makes a Successful Tool
• Richard Rorty On Why Programmers Rejected TDD
• The Downsides of Packages have been syndicated.)

LESSONS ON THEORY GROWTH FROM CANCER RESEARCH

For answers, Brian compared and contrasted the acceptance of TDD with Joan Fujimura's ideas about technology and theory diffusion. Joan wrote Crafting Science: A Sociohistory of the Quest for the Genetics of Cancer (1997). (A chapter covering similar territory, also written by Joan and entitled "Standardizing packages, boundary objects, and 'Translation'", can be downloaded for free.

While Joan created her theories after observing the rapid rise of the "proto-oncogene theory of cancer" in the 1980s, the lessons are relevant to software trends. In short, Joan posited that theories spread better with tools, which must supplement mostly routine procedures.

That's a little too tidy, so let's unpack it.

THE THEORY MUST BE JUST VAGUE ENOUGH TO SUPPLEMENT AND NOT CONTRADICT PEOPLE'S ALREADY EXISTING THEORIES. IT MUST DO SO WHILE STILL PROVIDING ENOUGH UNIQUE BENEFITS.

I'm strongly reminded of RESTful APIs "descriptive" rather than "prescriptive" nature. Often criticized as a weakness, this bug is actually a feature.

RESTful APIs alternative approach rose precisely when SOAP/XML implementations were undergoing a thousand syntactic and semantic papercuts. Given its ubiquity, HTTP increasingly served as an integration "lingua franca". But developers had lots of pet theories of how that worked in practice. REST's constraints offered "just enough" benefits to be useful while complementing people's emerging notions around how "web services" should be architected.

Despite what purists might say, REST's tenants are flexible and open to interpretation. Do you like verbs in your resource names? There are no REST police to arrest you. Do you want to tunnel everything through POST? While I might cringe, you could do that too. There are plenty of reasons why these practices are problematic. However, REST's vagueness accommodated a diversity of thought (some good, others not so good) while still delivering a programmable interface. I can't overstate that point.

But that was not the only attractor.

THE TOOLS AND PROCEDURES PROMOTE A SIMILAR WAY OF WORKING EVEN IN DIFFERENT SITUATIONS, SO EVEN THE DAY-TO-DAY WORK IN DIFFERENT CONTEXTS BECOMES ALIGNED AROUND THE TOOLS.

Every IDE, programming language, and browser rapidly improved its HTTP support during REST's ascendance. The tools arms race started as an attempt to better support web page creation (and later Web 2.0 experiences. However, in the process, these same sets of tool improvements benefited RESTful developers - rather than the responses being in HTML, their calls came back as XML, CSV, and a whole lot of JSON.

These two different contexts - website and API development - were supported by the same tools. The complexity and number of today's experiences have meant the pendulum has swung back toward suites of specialized tools for particular jobs.

That leads us to a significant conclusion.

THEORIES PIGGYBACK ON TOOLS

Regardless of your feelings on query languages, I would guess you'd agree that GraphQL gives a great first impression. Query languages (the "theory") are not new. However, every installation of GraphQL includes GraphiQL, a powerful IDE that runs right in the browser. New users can begin rolling their own queries, testing ideas, getting feedback, and cementing understanding "batteries included".

Of course, GraphQL is not just a query language. It also comes with a runtime capable of parsing instructions and resolving where it needs to find data.

Content creators had a uniform platform to build training upon. The documentation uses the same graphical conventions. For someone learning, the comprehension gap was shortened between seeing somebody performing a task and replicating that task in one's own environment.

Subsequently, other tools have been developed, like those from Apollo. But GraphQL's success among developers (many of whom had no personal involvement in replacing prior distributed system attempts with REST) is largely due to that powerful initial tool experience.

SO WHERE DO WE GO FROM HERE?

Maybe it's obvious, but just because an idea goes "viral" does not mean it is without downsides. In the case of proto-oncogene theory, the sudden and total adoption means that too many people switched too soon. It attracted individuals new to the field (good!), but newer, less experienced scientists hewed toward shallower research (boo!). Even established scientists, needing to maintain funding, shied away from challenging problems in favor of "least publishable units"; anything that involved recombinant DNA was more likely to be published. Hearing these challenges discussed reminded me of software's "resume driven development" problem.

For uptake to happen, a theory shouldn't disrupt (too much) existing beliefs, and it shouldn't disrupt (too much) existing careers. Tools should progress not just the theory, but other contexts people already use to get work done. If you can identify those things, you may be on the precipice of something great.

MILESTONES

• The OWASP group is revising their API Security Top Ten list this year, and needs your help..
• ThreatX, a vendor selling API protection services to enterprise clients, announced that it raised $30 million in a Series B funding.
• Postman's 2022 State of the API report is now out. Interestingly, a new obstacle reported by respondents is "lack of API design skills".
• Gartner also has a new set of API survey results. They state:
• 98% of orgs use/plan to use internal APIs (88% in 2019)
• 94% of orgs use/plan to use public APIs (52% in 2019)
• 90% of orgs use/plan to use partner APIs (68% in 2019)
• 80% of orgs provide/plan to provide publicly exposed APIs (46% in 2019)

Oh, and microservices are leaving the trough of disillusionment!

WRAPPING UP

So lots of small, and small to others, changes these past few months. First, I've shuttered Net API Events; doing a quick search, all kinds of different groups are talking about APIs - pointing out a few here and there just wasn't representative of all the great community sources that are out there. So, it was a good resource in its time but that time has probably passed.

Heroku's shutdown of free accounts also was impactful. Over the years, I built multiple shims - helpful toys that did useful things but didn't warrant full-blown cloud infrastructure (and associated costs). I'm redoing a handful of the most useful and, if things go well, I'll share what I've found in a later newsletter.

Finally, if you haven't noticed, I migrated the newsletter from Tinyletter to Substack. I don't require a ton of bells and whistles. However, after kicking Substack's tires for the Better Bits book, it felt like it was time to be on a platform under active development.

Because of all the moving pieces, patronage remains on pause for September. However, if you're interested in signing up, head to my Patron page. In the future, these pledges will ensure the email remains free of ads, paywalls, or information selling.

Till next time,

Matthew @libel_vox and matthewreinbold.com

While I work at Concentrix Catalyst, hosts of enterprise pragmatism in a realm of one-size fits all dogma, the opinions presented above are mine.