Wherefore Art Thou, Reddit?

Last week, rumblings of Reddit's new API pricing began making the rounds. In the time since the changes went from being a technology footnote to front-page news. As I write this, the site remains engulfed in protest, with over 7,000 subreddits representing a combined 2.7 billion subscribers, remaining private and unbrowsable.

In this edition of Net API Notes, I recap the source of initial Reddit API consternation, apply recent lessons we've had on platforms and developer ecosystem destruction, and end with what happens next. Are you currently attempting to scale a developer ecosystem, internally or externally? You will want to treat this as a case study of what can go wrong when shifting strategic priorities meets poor developer communication.

How To Burn Bridges (and Invent a Different Future) in Five Caustic Steps

Reddit Offloaded Much of Work To Its Developer Community

Every software development team has a finite time, energy, and budget. One of the advantages of an API architecture is that it can create an ecosystem of creativity, innovation, and potential solutions that the original development team may not have been able (or willing) to pursue themselves. Done correctly, an API catalyzes a community to support a myriad of edge cases. This strategic decision leverages external talent to expand the social network's reach, impact, and sustainability.

Reddit is a platform for hosting community forums called subreddits. When the platform algorithm notices popular new posts it puts them on the landing page and in several browsable feeds, like "Hot" or "Controversial". About 57 million people visit the site every day.

When community needs arose, Reddit often encouraged the community to solve its problems. Reddit didn't even have an "official" app on mobile devices until 2016, and even then, it was only because they bought the leading 3rd party solution, 'Alien Blue' (a solution that, upon a leadership change, was shut down). Support for many everyday moderation tasks remains complex, if even supported at all, in today's official app.

Moderation on Reddit is no trivial thing, either. Reddit has avoided much of the content moderation culture wars that have ensnared other social networks like Twitter and Facebook by delegating policy decisions to individual subreddit moderators. When problems arise, the individuals responsible for enforcement within a given subarea face questions (and abuse), not the platform. A pair of recent studies shows that this free labor equates to approximately $3.4 million a year, or 2.8% of Reddit's 2019 revenue.

To help them perform this critical (and unpaid) job, several tools utilizing the Reddit API have sprung up. While the free API strategy led to a rich developer ecosystem of tools, "gap filling" a variety of use cases that Reddit would never be able to prioritize or support. Providing a free API made strategic sense for Reddit until the online space changed - mainly, content strip mining in support of the current AI goldrush.

Today's LLMs Were Trained On Reddit's Data

In April, Steve Huffman, co-founder and chief executive of Reddit, announced there would be changes to the API pricing:

"The Reddit corpus of data is really valuable but we don't need to give all of that value to some of the largest companies in the world for free" - Steve Huffman, founder and chief executive of Reddit

Google's Bard and OpenAI's ChatGPT were only two of the most popular Large Language Models (or LLMs) trained on Reddit's data retrieved via API. And while OpenAI received a $10 billion investment from Microsoft, training sources like Reddit and Wikipedia didn't see a penny.

Reddit is not the only one. In my consultation, this has become an increasing concern over the first part of 2023. Companies that established harmonious API ecosystems are growing alarmed by the droning of vacuums, sucking out vast quantities of data for AI ends. They are looking to build toll booths across their landscape, and they want them yesterday.

With the desire to IPO, Reddit's API strategy was no longer a benefit but a bug. It was time to wrest back control, as is their prerogative. But here is where relatable intent and implementation diverge.

Keeping API Pricing Changes on the 'Down Low' Blew Up Instead

For several months, that's all Reddit API users had to go on - a promise that things would change. That changed two weeks ago when a phone call between the developer behind Apollo, a popular Reddit IOS and iPadOS client, and Reddit exploded across numerous subreddits. According to Apollo's author, Christian Selig, Reddit now expected him to pay $12,000 per 50 million requests.

Or Apollo's developer now needed to pay Reddit $20 million per year to keep running as is.

As if that wasn't enough of a shock, Christian needed to begin paying after June 30th or less than a month after being notified.

Understandably, this set off a chain of emotional outpouring from API consumers, moderators and communities reliant on customized, 3rd-party tools. Major Reddit communities, including some of the largest on the site, announced they would 'go dark' (or enter private mode) in protest from June 12th to June 14th.

To alleviate fears, Huffman held an 'Ask Me Anything' (or AMA) on Friday, June 9th. During that, he clarified that:

• Use of the Reddit API will cost money starting July 1st. Reddit will maintain a free data tier that allows 100 queries per minute per OAuth client (or ten queries if not using an OAuth client id).
• Reddit will charge $0.24 per thousand API calls for 'premium' API usage. (For comparison, that is 10-20x the cost of similar API access to sites like Imgur.)
• Mod tools like RES, ContextMod, and Toolbox will continue to have free access to the Data API. Further, free bots that help moderators and users (haikubot, setlistbot, etc.) should also have free access.

During this same AMA Huffman seemed to shut the door on making amends with the apps that are closing their doors.

"We acknowledge that the timeline we gave was tight; we are happy to engage with folks who want to work with us."

After Bridges Have Burned, There Is Lots of Ash To Shift Through

In addition to the previously mentioned Apollo app, makers of rif is fun for Reddit and ReddPlanet for Reddit have announced they will be shutting down their apps on June 30th. Several mobile clients like Sync, Boost, Relay, Infinity, BaconReader, and Joey, among others, have all said that the pricing doesn't work for them.

NetNewsWire announced they were ending Reddit API integration by the end of June. A newsreader app, they relied on Reddit's API for higher quality data instead of the available RSS feeds.

Extensions like toolbox are on thin ice, using the API but with the existing user's session. The developer doubts whether they will be able to continue. Smaller apps, like Nano (a Reddit app for the Apple Watch) are left in limbo, having yet to hear from Reddit.

PushShift is a popular mod tool that is currently working with Reddit to come to an agreement. At the time of this writing, there has not been an announcement.

The blackout, which moved most of Reddit's threads to 'private' mode, is wrecking several Google search results. As Ryan Broderick reported, many, many searches have Reddit threads as the topmost result. Those results, for the time being, are not accessible.

The similarities to Twitter's torching of its developer's ecosystem warrant examination. By Reddit's estimates, 90% of current API users would be able to continue using the newly announced free tier successfully. That said, recent events have shaken the community's trust in the platform. What would stop Reddit from announcing another change, with a similar, narrow window to respond, in the future? Even if served by the free API tier, how long would that last? And - in a cruel irony - why should another company enjoy the spoils of their effort?

The fear, uncertainty, and doubt raised by recent Reddit actions have permanently damaged the goodwill and future efforts among the 3rd parties. These 3rd parties were responsible for keeping Reddit communities vibrant and growing.

What's Next - Fediverse Alternatives?

What are some concrete lessons ecosystem developers can take from this situation? Never change your API? While Last.fm's API recently celebrated its 20th anniversary, it is highly unrealistic to expect an API to go decades without an update. Despite our best efforts, APIs must change to accommodate a changing world.

Provide more runway between announcing changes and enforcing them? Certainly - the 30 days notice is ridiculous, especially given the amount of money that 3rd parties were expected to pony up. But how much time is enough time? For major changes like pricing, it needs to be as much heads up as possible and then some. A minimum of 3-6 months seems necessary with many providers. When Google Maps announced significant new pricing in May 2018, the changes went into effect on July 16th (or a little more than two months - not great, but better). When Twitter retired v1.0 of their REST API in 2012, I recall there was something like a year to move to a new version - however, that history seems to be lost in Twitter's recent shenanigans.

For many Reddit users, this is another example of how centrally owned and operated platforms can't be trusted. They are quickly decamping for other alternatives. Two Reddit-like options are Lemmy and Kbin. Both sites utilize the ActivityPub protocol that I covered at the end of last year. This participation in the 'Fediverse' means the surrounding platform is decentralized. Furthermore, concepts like identity are portable across other Fediverse participants including Mastodon and Pixelfed.

Oh, and Jimmy Wales (the Wikipedia guy) announced he has a team also working on a Reddit-like service. It is 'not quite ready yet'. However, he has publicly pledged that after they finish their 'trust' algorithm, they will seek to add ActivityPub support. They even have a draft API.

On Wednesday, the 14th, many currently protesting subreddits will return to public availability. Some tools will close up shop by the end of the month, but others will remain. Communities will persist because the switching costs are so incredibly high. Six months from now, we'll look back on the events that transpired as just another in a long line of online growing pains.

All of this would be relegated to a footnote, except for the vocal, talented minority that emerges from this seeking something better. Their strong conviction, combined with this unique inflection point in social networks, suggests some exciting and more equitable futures - not just for APIs but for communities of all types. Reddit's recent actions soured more than they might think. But tomorrow's lemonade promises to be that much sweeter.

Milestones

• OWASP has updated its API Security Top 10 for 2023. A new YouTube channel, SisiNerdTV, provides an overview in video form.
• Lorna Jane Mitchell has published a new blog post clarifying OpenAPI's different usage of the 'Summary' and 'Description' fields.
• OpenAI announced a slew of new API updates. Simon Willison has a nice roundup of the changes, including functions?!
• SmartBear announced OpenAPI 3.1 support in their open-source tooling.

Wrapping Up

Thanks, as always, to my paid subscribers. Researching, writing, and rewriting takes a fair amount of time. And one of the "fun" aspects of online publishing is that effort doesn't necessarily translate to success. Support from the subscribers means I can apply the time and care necessary to tell a story correctly, rather than worrying about having "hits". That means a lot to me. Thank you.

That's all for now. Till next time,

Matthew (@matthew and matthewreinbold.com)

Update 2023/06/16

“Well, that escalated quickly.”

In the last couple days, things have gotten ugly:

• In a leaked internal email, Reddit CEO Steve Huffman says the protest over API changes “will pass”.
• That pours further outrage fuel on the fire, with several subreddit moderators promising to continue the protest “indefinitely”.
• In reaction, Reddit changed their moderation policy to allow users to vote out moderators, with the intention of replacing moderators continuing to protest.
• Many Reddit users, upset by the hostile actions of the platform, began deleting their posts and comments. However, multiple users have reported that Reddit undid those actions), making those posts and comments visible again. The action would seem to be a GDPR violation for European users.

Update 2024/02/21

Today it was reported that Reddit signed an exclusive AI content licensing deal with Google. Google gets a significant body of content to continue training its LLMs, and Reddit receives approximately $60 million per year.