Intellectual sovereignty in the era of AI enshittification
As explored by independents
“–is that Magnolia? Or rose? I mean, wow. Italy even smells good.”
This is a paper on the enshittification of AI and the value of intellectual sovereignty, but that pull quote is an excerpt from my Lake Como travel journal written during an oppressive heatwave.
In Lake Como, I paid attention in the singular way that being forced to move slowly in unfamiliar territory allows. My senses were sharpened; my curiosity was piqued. I was enamored with the beauty of the little enclave we were essentially stuck in, and I constantly waxed poetic about how lovely it smelled.
Then I found the source: a hidden scent diffuser among the plants.
So the entire Italian province was not, after all, emitting its own intoxicating pheromones. Though it’s not unreasonable to have assumed so; the swag does permeate. Instead, my sense of discovery was largely engineered with the machinery behind it hidden out of sight.
Not unlike that diffuser, AI also delivers the illusion of discovery and runs on machinery I can’t see. The crucial difference is in intent and scale. The diffuser was likely a curated choice by the woman who owned our rental and lived on the first floor. In contrast, an LLM is the product of an entire reasoning model’s worth of strangers who set out to optimize for the average with no context other than what converts.
Case in point: I’ve since gotten into making my own fragrances, so for this paper, I prompted a few LLMs, intentionally vague — “I’m getting into fragrance making” — and watched what filled the gap. Perhaps expectedly, the AI’s first response usually included a popular formula to try.
Granted, AI doesn’t have a nose (yet) and it certainly doesn’t have my nose. It can’t actually smell. But it is ostensibly trained to help the user as efficiently as it can.
The folks designing the parameters the AI is trained on have thus far defined helping users as providing us with whatever answer they best estimate we’re after–and doing so as fast as possible. This built-in behavior of helpfulness is linked to why LLMs struggle so much with sycophancy–after all, they are literal, hyper-speed people pleasers (ironically made by people who few are currently pleased with).
You want to make a fragrance? Let’s get you there fastest: make this probabilistically-pleasing one.
This feels like a good time to mention that since 1962, every time a new technology has been introduced into a school setting, student performance has subsequently gone down. Neuroscientist Dr. Jared Cooney Horvath explores this technological interference further in The Digital Delusion–but it likely won’t surprise you to also learn that Gen Z is the first generation to underperform its predecessors in nearly every field test.
AI can of course support self-learning and improve efficiency in groundbreaking ways. It can also give users a false sense of mastery and erode the mental faculties that underlie one’s ability to learn.
Head of Ness Labs, a mindful productivity school for knowledge workers, Dr. Anne-Laure Le Cunff is another expert who helpfully frames this paradox. In a recent NYT opinion piece she points out that “By shortening the time between asking a question and getting an answer, these tools are actually undermining curiosity – and paradoxically threatening our ability to understand the world.”
In a society of immediate answers and algorithmic-routines, genuinely random engagement with the world has become a rare human experience. But the less we wander on our way to an answer, the less we learn along the way. Researchers call this ‘incidental learning’ and we can thank it for everything from the invention of penicillium to the Kink’s 1964 classic You really got me.
AI isn’t just drastically short-cutting how we learn, solve, and discover, though.
It’s also getting worse while it does it.
The enshittification of AI
“Enshittification,” a term coined by author and tech critic Cory Doctorow, speaks to the phenomenon where online services and products gradually decline in quality to maximize short term profits for a select few companies.
We’ve already seen it happen with rideshares. Maybe you remember when Ubers were cheaper than yellow cabs and discount coupons were given away daily. Now you pay surge pricing for a ride that used to cost half as much, wait longer for a driver to accept, and watch as the app funnels you into a longer pickup radius.
Or consider streaming apps. Netflix once felt like a steal – all the shows and movies you can watch for a few bucks a month. So you got rid of cable, right!? Sayonara commercials! But now you’re paying $150 a month for six different streaming services, there are still commercials, and somehow you still can’t watch that one sports program you want to.
Food delivery apps take the metaphorical cake. If anything is going to get this millennial to pick up the phone, it’s seeing UberEats offer me $15 off only to mysteriously increase the “fees” by the exact same amount. Now if you call a restaurant, though, there’s a good chance they don’t even take orders over the phone anymore.
It’s all by design, of course.
Behaviorally, enshittification (aka platform decay or crapification) goes through three distinct stages:
Stage 1: Get users hooked! Offer major perks and heavily discounted services until market dominance is established.
Stage 2: Get business customers hooked! Once individual users are “locked-in,” grant advertisers, sellers and business customers access to the audience you built, and all their data.
Stage 3: Take everything for yourself! Finally, the platform degrades the experience for everyone, squeezing both users and business customers out of all available value to benefit shareholders.
Enshittification, or maybe call it trickle-up economics, is already swiftly enveloping the AI industry.
As Rudy Blanco noted, “The amount of things I was able to do with a free account a while back is now nearly impossible with the new compute limits. If you’re not paying — you get a dumber version and you are limited to basic conversations and can’t really get any work done.”
—————————————
Christophe Jammet expounds, “The idea that AI is here to improve worker efficiency and output quality is diametrically at odds with the enshittification it’s undergoing which is making said AI increasingly expensive and harder to access.”
The quality of information we’re receiving from AI is likely going to continue to get either worse in quality or more expensive in price–or both. Objectively bad news when you take into consideration this direct quote from the king of AI, Sam Altman: “We see a future where intelligence is a utility, like electricity or water, and people buy it from us on a meter.”
How many of us does Sam Altman’s “we” leave out I wonder?”
And what happens when an entire population is gatekept and charged for their own collective knowledge?
“If everyone has access to the same AI how will anyone be able to distinguish themselves unless they use actual people?” – Josh Schneider
This decay of individual intellectual sovereignty in exchange for the production of a more rapid collectively-averaged answer is of concern in every industry. Editor Nick Douglas worries “that clients will no longer value high-caliber human-made writing, and settle for slop.”
But many clients already are–not only settling for AI slop, but settling for AI dictating what is or isn’t AI slop.
AI screening tools are now deciding what passes as human writing and what gets clocked as AI – and more than enough humans are willingly and obediently submitting to its authority.
The result is a chain reaction of idiocy. Students and professionals alike are foregoing punctuation, swearing off the em-dash, intentionally misspelling words and steering away from long-engrained oratory patterns of Homeric rhetoric in an *attempt* to appear human–and they’re advising others to follow suit.
We are now playing dumber versions of ourselves to try to convince technology we are human. It’s a strange inversion of the outsourcing we were originally worried about. First we handed over our thinking to AI; now we’re handing over our humanity to prove we didn’t.
It’s not an easily reversible habit either. Intellectual capacity isn’t a reserve you can dip into whenever. It’s like any other muscle: use it or lose it.
Diagnostics check: Who’s calling the shots?
Given the increasingly-enshittified landscape, there are two questions leaders should be asking:
- Does your human workforce or your AI decide what needs to be done?
- How about how it gets done?
Answering these two questions can help leaders determine whether their workforce is on track to become smarter or just faster.
Because if it’s the latter–if you’re outsourcing the decisions that differentiate you on a competitive level–you may just be joining the largest and most expensive groupthink experiment in history only to result in becoming another one of the herd.
How are contractors are using AI?
At an enterprise scale, we already know it’s not if professionals are using AI; it’s how–and the how matters a lot.
A study focused on human-AI collaboration and conducted on BCG consultants found that knowledge workers use genAI in three distinct ways, defined under the following archetypes:
Centaurs (14%) represent the cautious experts. The smallest group, centaurs wield AI with precision, asking specific questions that demonstrate and deepen their own expertise. They intentionally avoid reliance on AI and they also produce the most accurate results.
Cyborgs (60%) make up the largest group, collaborating with AI in every step and iterating in a state of “fused co-creation.” Their domain expertise saw little growth in the study, but the results were effective–and they developed a valuable skill: AI fluency.
Self-Automators* (27%) basically mailed it in, handing entire tasks to AI with minimal engagement and putting themselves most at risk of cognitive debt. Their resulting work looked and sounded good but was ultimately shallow.
*Researchers noted that the real number of Self-Automators was likely higher.
How are knowledge workers using AI?
While a similar percentage of independent contractors surveyed (55%) came in as cyborgs–the co-creation group–the variance widened on either end. A whopping 39% of independent contractors surveyed tested as centaurs–cautious experts–while only 5.6% surveyed came in as self-automators.
Contractor responses reflected a recognition of the power of AI while also sharing a kind of vigilance around how to engage with it.
“AI is a curiosity rabbit hole. But I don’t rely on it to think for me.”
– Andrea Stein“I’ve found AI to be most beneficial when given very specific, strategic prompts.”
– Teaganne Finn“It saves time specifically for initial research/outlining/summarizing interviews.”
– Stephanie Susnjara
There were also concerns about the erosion of intellectual property and intellectual sovereignty.
“Much as all-natural ingredients became a selling point after the rise of artificial flavorings and sweeteners, I’m hoping the same happens with intellectual property” – Jon Parker
“I have a number of concerns around AI such as reduction in critical thinking skills, over-reliance on AI tools over skilled knowledge workers, and environmental impact of broad AI usage.” – Kathleen McGivney
Signals worth watching
Train the workforce, not just the model
Not the kind of workforce training Meta recently rolled out. Using your human workforce to train AI feels like a deeply morose prequel to The Matrix, and who wants to watch that? (Other than our Machine Overlords, obviously.) Humans are pretty adaptable creatures. Give them AI literacy first so adoption is informed instead of improvised.
Fund oversight, not just compute
Leading businesses aren’t trading headcount for GPUs — they’re creating and paying for Responsible AI roles, staffed by genuinely diverse teams. Diverse groups consistently out-predict homogenous ones, and that edge compounds the more decisions get handed to AI. Cutting the people meant to supervise the machine is the fastest way to lose the very oversight you’re paying compute for.
Watch the open-vs-closed divide
As Duy Nguyen, an AI engineer at the New York Times, put it at a New School forum on AI, Trust and the Media: a closed model “just abstracts [the user’s] security problems away.” That opacity is by design, not an oversight — and it remains a global sticking point.
78% of independent contractors surveyed are curious about open models.
As Michael Terwindt noted, “rising token costs and UX dark patterns will keep driving dependence on proprietary systems, making open weights and local agents a real counterweight, not just a developer preference.”
Closing
Let’s head back to Lake Como for a moment.
I’ve tried to hold onto that same sense of slow discovery since. But LLMs don’t see the value in the journey, or at least their creators don’t prize it. For AI, it’s all destination, no journey.
Thus, the autocomplete of human history continues — optimized for the parts of thought that are easiest to score (logic, retrieval, task completion), and blind to the parts that are more dynamic.
But being human is a multi-dimensional experience: sensing, intuiting, occasionally following a useful irrationality that never shows up on a benchmark. If we keep outsourcing our senses to artificial intelligence, can we still claim a common one with each other?
Maybe a better question: if today’s tech leaders don’t stop to smell the roses, or they outsource the job to a nose-blind and non-sentient being, how are they going to know when something starts to smell like shit?