"Going from Web Page to Web Platform is a big deal."

ProgrammableWeb's early reporting captured a period of explosive growth in the API industry. However, as approaches to API discovery matured, it did not translate to continued success for the site. In this piece, I'll explore the stories of why that may be.

Occasionally, moments emerge in daily business, changing the course of entire industries. The API landscape is a space as molded by legend as any other. While their accuracy to actual events remains debated, these tales remain the cornerstones of hallway tracks and executive talking points. These are API's apocryphal stories - the ¡APIcryphal!

Enjoy this public preview of ¡APIcryphal! for the next 30 days. After that, it will join the other API-industry-changing events available to paid subscribers. Sign up for a paid subscription to ensure continued access to this and future editions.

Also, this piece was only possible with the Internet Archive. I spent hours on its Wayback Machine for this piece. The Archive is more than a repository of old web pages; it is a guardian of online heritage. By preserving websites like ProgrammableWeb, the Internet Archive offers an irreplaceable tool to researchers, educators, and practitioners whose realm of inquiry exceeds today's business attention span. While the ¡APIcryphal! series is my attempt to capture and preserve stories like ProgrammableWeb, I am limited in what I can do. If you are able, please consider donating to the Internet Archive today.

Observance

The early 2000s marked a profound shift in how companies designed for and used the web. Unlike Web 1.0, which was characterized by static pages and a one-way flow of information, Web 2.0 emphasized user-generated content, usability, and interoperability. The term "Web 2.0" was coined by Darcy DiNucci and popularized by Tim O'Reilly, O'Reilly Media's namesake, during a 2004 conference. It was meant to serve as an easily expressible encapsulation for a slew of technology solutions, business models, network effects, and user interactions - all those things required to make dynamic web experiences.  

APIs played a crucial role in the Web 2.0 movement. APIs allowed developers to create "mashups", or information products that combine data and functionalities from multiple sources. An early 2005 example was when Paul Rademacher reverse-engineered the javascript for a new website called Google Maps. After scraping Craigslist housing listings, he overlaid the data on top of Google's map to create HousingMaps, the first-ever map mashup - something that blew even habitually online people's minds. 

In 2005, John Musser was working on a startup idea that involved integrating data from the web. Using APIs in a Web 2.0, mashup-like way seemed obvious, but finding the necessary APIs proved to be difficult. As Musser describes in a YouTube video interview:

"When I was researching, I thought there would be a place to go to find out about APIs, and it turned out there wasn't. So we registered the domain name and started."

That registered domain name was ProgrammableWeb.com. Explaining ProgrammableWeb's mission statement in an early post, Musser wrote:

"-the goal is to create a home page for Web 2.0 developers. Content to include news, reviews, comparisons, and examples. Formal APIs, unofficial APIs, and accidental APIs are all fair game." … "Why? Because going From Web Page to Web Platform is a big deal."

In addition to blogged coverage of the space, ProgrammableWeb also began compiling a directory of available APIs and the mashups that used them. In 2005, ProgrammableWeb launched with 40 APIs in its directory. The approach - combining breaking news and usable examples with a growing catalog of developer resources - was a hit. The directory provided a clear, up-to-date list of new web APIs that could be used in a consistent format. The mashup listings also served as inspiration for devs looking for ideas, as well as giving them a place to showcase their work. 

As Musser explained when I asked him:

"Lots of those early developers were really excited and grateful when they saw their creations showcased on PW. And as it turns-out, many API providers were also excited to see their APIs get greater visibility and awareness. So it was really a great opportunity to make all sides happy."

By 2010, the total APIs in the ProgrammableWeb directory had reached 2000 entries. The growing community attention led Alcatel-Lucent, a networking and communication company, to purchase ProgrammableWeb in June 2010. Alcatel-Lucent sought to strengthen its relationship with developers and promote its developer platform. ProgrammableWeb was promised support and resources to "promote the growth of the API, mashup and developer ecosystem". John Musser joined Alcatel-Lucent as part of the acquisition, and ProgrammableWeb was allowed to operate as a separate entity.  

However plausible that pairing sounded in principle, it unfortunately did not work in practice. Alcatel-Lucent ran into challenges shortly after the acquisition. Eventually they were forced to sell off non-core businesses and assets. In early 2013, that included selling ProgrammableWeb to Mulesoft. Mulesoft, in a significant growth phase and flush with new funding, saw ProgrammableWeb as essential to aiding it in becoming "the go-to destination for APIs and Integration" and "the GitHub for APIs". 

ProgrammableWeb was to continue its blog, API database, and stature as an independent entity. MuleSoft gained access to ProgrammableWeb's large and active API developers and consumers community. Combining ProgrammableWeb's API directory with MuleSoft's own API directory, APIhub, was to create "the world's largest and most complete repository of APIs".

John Musser left ProgrammableWeb shortly after the MuleSoft acquisition. To serve as editor-in-chief, MuleSoft appointed veteran technology editor David Berlind. Berlind's experience included promoting pro-online and blogging-friendly approaches to legacy technology brands like InformationWeek, BYTE, Network Computing, and Dr. Dobs. He had also served as the executive editor at CNET. MuleSoft's president and CEO Greg Schott touted Berlind's "years of experience and developer roots," calling them "the perfect combination for the continued success of the publication."

Upon accepting the role, Berlind commented on the decision to move from "traditional journalism" to being employed by a vendor:

"My paycheck comes from a vendor, but what I like is that I get to run it as a fully independent, objective news engine that's creating content. What we do every day is journalism."

Despite the subtle shift in public statements casting ProgrammableWeb primarily as a media property, the API directory continued to grow. In the first quarter of 2017, ProgrammableWeb's directory surpassed 17,000 APIs.

In 2018, ProgrammableWeb's parent company, Mulesoft, was acquired by Salesforce in a deal worth $6.5 billion. From various press releases at the time, it appeared that Salesforce aimed to leverage MuleSoft's capabilities to enhance its own API and integration offerings. 

ProgrammableWeb crossed 24,000 listed APIs in 2021. It continued to exist until it suddenly didn't. After nine years leading ProgrammableWeb, David Berlind announced in May 2022 that he "decided to move on to the next adventure". By October, ProgrammableWeb announced "a decision to shut down operations". 

Interpretation

Even when it closed, ProgrammableWeb remained one of the largest API directories. So what happened? 

The Alexa Rank attempted to identify the most popular million websites on the web and list them in order of popularity. Its methodology made it unreliable for deep or nuanced insights - a problem that contributed to its demise in 2022. However, as a coarse generalization, we can sense a site's relative stature over time. Starting with ProgrammableWeb's acquisition by MuleSoft, a trend line emerges. After a brief dalliance as a top 10,000 site, ProgrammableWeb's rank among sites gradually erodes, frequently landing outside the top 40,000 sites after 2018, when Salesforce purchased MuleSoft.

As Google Trends suggests, the trend occurred despite overall interest in various API styles continuing to increase during the same period.

The apparent contrast suggests a change in the maturity of the API industry and how developers sought information about it. 

Adam DuVander worked at ProgrammableWeb from 2009-2014, eventually becoming the site's executive editor. Responding to my emailed questions, he replied: 

"We had a lot of discussions at the time about how viable an API directory would continue to be.".."If that was a metaphor we could follow, then we were Yahoo! and Google was coming."

As API sophistication grew, organizations began offering their own carefully curated developer experiences. API programs drew investment as companies sought to replicate the success they saw in places like Twilio. The rise of APIs-as-a-Product meant that companies increasingly offered a polished, professional package - not the stub of coded output. Technical writers and developer advocates not only documented the API itself but published all manner of tutorials, trend pieces, and other content marketing to raise the SEO attractiveness of their offerings. 

These efforts were meant to attract developers whose API consumption habits had changed. At ProgrammableWeb's founding, carefully documented, easily accessible information was hard to come by. In addition, composability was still relatively new to many programmers working on the web - having a set of examples to draw upon for inspiration was handy. But in the aughts, that was no longer the case. Developers were discovering APIs via search engines (i.e., Google), eventually bypassing ProgrammableWeb's curated, yet destined to be incomplete, content. 

The effort required to maintain up-to-date and relevant listings makes any directory, like ProgrammableWeb, a demanding business model​. The high turnover of relevant content and the need for continuous innovation and adaptation to changing technologies to maintain growth (new formats, specifications, and user expectations) create a Red Queen effect just to maintain existing eyeballs. 

Those shifts are challenging enough on their own. Then online advertising collapsed due to the COVID-19 pandemic. Faced with uncertainty of the global pandemic, companies reduced their spending on sponsored content, banner ads, and enhanced directory listings - an important revenue stream for the site. With advertising unable to cover the cost of employees and freelancers, much less maintaining technical parity with increasingly sophisticated and SEO-optimized developer portals, MuleSoft sunsetted the site. 

I reached out to several individuals involved with ProgrammableWeb during this latter period to confirm these interpretations of events. They declined to answer my questions. 

Keys

Online directories are an incredibly difficult business model. Mashape started as an API marketplace, allowing developers to discover and use APIs listed in their directory easily. When that stalled, Mashape renamed itself to Kong Inc. and pivoted to API management and gateway solutions. Algorithmia began as a directory for AI algorithms via APIs. When this failed to gain critical mass, the site refocused on providing infrastructure for deploying and managing machine learning models. 

Directories are challenging for several reasons.

Success Is Reliant On Network Effects

A directory's value increases as more producers and consumers rally around it. However, achieving a critical mass of both users and content providers can be challenging. Without a robust user base, directories struggle to attract high-quality content, and vice versa.

DuVander again:

"One thing that was probably not obvious from the outside was how big of a job it was to maintain the directory."..."We hired a team of contractors, most of whom had degrees in library science. They helped us keep up with the growth while maintaining an accurate directory."

Musser:

"-there was a ton of manual effort. Always. Although it's natural to think there was a moment that it flipped to be less work for us, it never really did. Certainly over time there were more and more API providers who came to us, overall it was still primarily up to our team to go find and catalog them."

Monetizatizing Directories Is Hard

The most straightforward way of getting paid for a directory includes advertising, sponsored listings, and subscription fees. However, each of these methods has significant caveats:

• Advertising: Revenue from ads often requires substantial traffic, and the increased use of ad-blocking technologies reduces potential earnings.
• Sponsored Listings and Preferential Search Results: While effective, they can undermine the directory's credibility if users perceive it as biased.
• Subscriptions: Users may be unwilling to pay for access to a directory when Google search results are free.

Search Engines Provide Greater Breadth

Search engines like Google provide comprehensive search capabilities that exceed the reach of curated directories. Users are also habitually trained from years of searching for other things and may have diminished patience for navigating an unfamiliar or less-often used UI from a directory site. 

Software Continues to Evolve

Maintaining a directory requires continuous updates to ensure content accuracy and relevance; an ingestion process that accepts OpenAPI documents will need something new to support a GraphQL SDL file. Complex flows may require specialized documentation. This can be resource-intensive, especially if the directory spans multiple categories or industries. And just when one might have had a handle on that, there's now a V2. It all adds up to a significant amount of effort and sometimes manual verification.

Developers Support Behavior Has Changed 

Users' preferences and behaviors evolve. The rise of social media, forums, and community-driven platforms has shifted how users seek and share information. These platforms often provide more dynamic, interactive, and real-time content compared to static directories​.

Reversal

Despite the difficulty, several companies continue to pursue directory - or directory-like - businesses. The companies that express directory-like functionality go beyond simply compiling metadata and linking to original sources. 

Rapid (formerly known as RapidAPI), claims to be the current "World's Largest API Hub". More than just a directory, Rapid inserts itself between the API producer and consumer, providing a unified integration, access control, payment processing, and monitoring experience across numerous APIs, while charging a transaction fee per call for the convenience. Despite being valued at $1 billion in 2022, the company laid off 50% of its staff in 2023.

APILayer advertises itself as "a highly curated API marketplace" for "simplifying the process of finding an API". Like Rapid, they serve as a middleman between API producers and potential consumers, handling logistics like payment processing and user management. It appears to use the community driven APIList.fun to source new APIs for its catalog, among other means.

AnyAPI has a marketplace but charges for creating automated routine workflows between listed items. 

Postman claims to host "The largest collection of public APIs by developers around the world" but makes its money selling tooling. 

Berlind, while delivering testimony at a 2015 ONC's API Task Force hearing, described ProgrammableWeb's directory in the following way:

"It is important to note that when we make a record of an API in our directory, we do not offer anything beyond the metadata about that API. In other words, that API cannot be 'called' by a developer through ProgrammableWeb, nor does ProgrammableWeb host any of an API's associated assets (ie: documentation). ProgrammableWeb does however provide links that users can click in order to gain direct access to those assets."

Directories face numerous challenges that make them a difficult business model. Today's companies that tout API collections survive by offering more comprehensive service offerings. It remains to be seen if that effort will be enough to overcome the challenges defined above in the Keys section. 

Legacy

Each ProgrammableWeb acquisition brought a shift in strategic direction, reflecting the acquiring company's hopes to intertwine its business fortunes with the growing API industry. While ProgrammableWeb did evolve under each owner, the ultimate shutdown in 2023 was the culmination of a gradual but significant shift in how the industry supported API development and how developers acquired API information.

After ProgrammableWeb, John Musser founded API Science, an API monitoring company. He has since launched numerous other API ventures and is now the director of data and analytics for Ford's Autonomous Vehicles division. 

Adam DuVander worked in various developer relations roles before founding EveryDeveloper, a content strategy, developer engagement, and editorial coaching consultancy.

David Berlind is currently Editor-in-Chief of the Blockchain Journal. 

Shortly and sometime after MuleSoft announced it was shutting down ProgrammableWeb operations, the website - along with more than 17 years of content chronicling the API industry - was no longer available. At this time, a small percentage of ProgrammableWeb stories have been preserved on the parent MuleSoft blog. Additionally, some branded content can still be found on various platforms (YouTube, Apple Podcasts, Medium). The rest of the reporting, API directory, and mashup listings no longer exist outside of what can be found on the Internet Archive.