The Retention Economy: Kept at All Costs

Years ago I started noticing a pattern in my Instagram Reels. In between the clips I enjoyed watching, the app kept feeding me clips that I had never asked for and did not want: suggestive, borderline stuff, from accounts I did not follow and had no interest in. So I did what Instagram tells you to do when a recommendation misses. I marked them not interested. Once, then again, and then so many times I lost count. It changed almost nothing. The feed had formed a view of me, and no amount of me saying otherwise seemed to shift it.

It was only much later, when Instagram added a proper set of controls for tuning what it recommends, that I managed to get things back to something I recognised, but still not entirely. It should not take that much effort, kept up over years, to convince a service that I do not want what it keeps putting in front of me. Somewhere along the way, an ordinary app for sharing photos and videos with friends had decided that the way to hold my attention was to show me things I found neither interesting nor welcome. And for a while it was right, in a sense, because as long as I kept tapping not interested, I was still there.

I want to be careful here, because it is easy to reach for the wrong explanation. I do not think anyone at Instagram decided to automatically send suggestive content to men on purpose. Instead, I think it was built to do whatever keeps people watching, and it worked this out on its own. That is worth keeping in mind for what follows: no one designed it to target anyone specifically. It optimises for attention, and our weaker moments are where the attention tends to be.

This is the thought I keep returning to. The feed was not trying to entertain me. It was trying to keep me. Those are not the same goal, and almost everything that troubles me about modern social media lives in the gap between them.

Keeping You, Not Pleasing You

We usually call this the attention economy, as though the platforms were competing to be the most interesting thing on your phone. I think that name flatters them. If the goal were really to interest you, a bored user would count as a failure. But a bored user who keeps scrolling is not a failure at all. What these platforms optimise for is not your interest but your time, and the two come apart more often than you would think. It is closer to a retention economy, and the only number that truly matters to it is whether you are still here.

Delight is one way to keep someone watching, but it is not the cheapest or the most reliable one. Provocation works too, and so do unease, outrage, and the quiet pull of not wanting to miss whatever comes next. A feed built to make you happy and a feed built so you cannot put it down would look quite different from one another, and what most of us are carrying around is the second kind.

Meta’s own leadership described this mechanism back in 2018, when explaining why Facebook would start demoting what it called borderline content. Mark Zuckerberg noted that as a post gets closer to the line of what is allowed, growing more suggestive or more provocative, people engage with it more, on average, even when they say afterwards that they did not like it. It is worth reading that back slowly, because it is a striking thing to admit: engagement goes up precisely where satisfaction goes down.

Researchers have since measured the same gap directly. One team compared an engagement-ranked version of Twitter against a plain chronological feed. The engagement algorithm surfaced more of the emotionally charged, politically hostile posts, which is roughly what you would expect. The part worth reflecting on is what people said when the researchers asked them: for those divisive political posts, they preferred the ones the algorithm had pushed down. On the content that sets people most against each other, what the feed served up and what users said they wanted came apart.

My own experience was a trivial version of this, but the same pattern shows up when researchers look at the platform as a whole. An investigation into how Instagram ranks the ordinary photo feed found that images showing more skin were significantly more likely to be pushed in front of people than other posts, and made up a noticeably larger share of what users were shown than of what was actually being posted. The researchers were careful about the limits of their work, since they could not see inside Meta’s servers, and Instagram rejected the findings. However, it lines up with what the company had already conceded about how engagement behaves near the line.

The Content Is Interchangeable

If this was only about my cluttered Reels feed, it would hardly be worth an essay. What makes it worth writing about is that the mechanism does not care what the content is. It has no particular interest in skin. It has an interest in your attention, and it will reach for whatever holds yours. Point the same algorithm at a different person and it finds a different lever.

Point it at a teenage boy, and it finds the manosphere. When researchers in Dublin built fresh accounts modelled on young men and simply let the recommendations run, the feeds served up content promoting rigid and hostile ideas of masculinity within minutes. On YouTube Shorts it took two. By the end of the experiment, toxic or problematic content made up around three-quarters of everything the accounts were shown, most of it from the manosphere. Nobody had gone looking for it. The accounts had only signalled, in small ways, who they might be, and the algorithm did the rest.

Point it at someone who tends to dwell on sadness, and it finds despair. Amnesty International registered TikTok accounts as thirteen-year-olds in Kenya, the USA, and the Philippines, and had researchers watch by hand what each was served after pausing on videos touching on depression, self-harm, or suicide. All three tipped into a rabbit hole of that content within twenty minutes, the slowest of them taking the full twenty to get there. In a separate, longer-running study using automated accounts in Kenya and the USA, close to one in every two videos served after five to six hours was of the same kind, roughly ten times the rate shown to accounts with no interest in mental health at all. The algorithm did not create the low mood it found. It noticed it, and then, chasing watch time, it fed it.

Point it at a whole population, and it finds division. This one is not a critic’s guess; it is what Facebook found when it studied itself. One internal presentation said, in plain words, that the company’s algorithms exploit the human brain’s attraction to divisiveness. Another concluded that 64 percent of the people joining extremist groups on the platform were doing so because Facebook’s own recommendation tools had pointed them there.

Three independent research teams and one internal audit have now documented four different harms with one cause in common. The subject is interchangeable. What does not change is the direction of travel, which tends towards something a little more provocative, a little further past the line, than whatever you are looking at now. A feed pulled a little further each time holds you a little more reliably, and reliably holding you is the whole point of the business.

What the Algorithm Really Does

Here the platforms have a defence, and I think it is a fair one, so it is worth taking seriously. They will tell you they do not push anything on anyone. They only show you more of what you have already shown an interest in. And that is undeniably true. Every one of those unwanted clips, every manosphere video, every bleak late-night post, began with a real signal from a real person. But that does not clear them. Looked at properly, it points straight at what is wrong.

Consider what showing an interest actually amounts to, at the scale of a single person. It is not a decision, and it is not a value you hold. It is a glance that lingered a second too long, a thumb that slowed without being told to, a flicker of attention at one in the morning. We are all made of these flickers. And a good deal of what makes us who we are, rather than the sum of our worst passing impulses, is that most of them are meant to fade. A healthy inner life is mostly made of thoughts we notice and let go of: a flash of anger or envy or want, often gone by the time we wake up the next day. That fading is not a fault in the mind. It is the mind doing its job. Forgetting, most of the time, is a kindness.

We know this well enough that we have built whole practices around relearning it when it slips. The core of mindfulness and meditation, which by now has a reasonable clinical evidence base, is training in exactly this: learning to watch a thought as a passing event rather than a fact about yourself or an instruction to be obeyed. Therapists have people picture each thought as a leaf on a stream, seen and then let go. This is not soft advice with nothing behind it. In clinical work on mindfulness-based therapy for anxiety, that specific capacity, which clinicians call decentering, has been shown to be one of the active reasons people get better.

Now hold that up against the feed. If health is partly the ability to let a thought pass, then the algorithm is close to the first thing we have built that reliably works against it. It catches the flicker in the moment it appears, in the tired second or the low scroll before sleep, and instead of letting it fade the way it would offline, it hands the thought back to you and watches to see whether you reach for it again. Where mindfulness trains you to notice a thought and let it go, the algorithm does the opposite: it notices which thoughts you dwell on, and shows you more of them. A mood that would have lasted a minute and a half becomes a session. A run of sessions becomes a habit. A habit, kept up long enough, starts to feel like something true about who you are.

This is where the platforms’ defence stops working, to my mind. Yes, the interest was yours. Yes, the thought was really there. Nobody is disputing that. The algorithm did not put the darkness in you. But it treats a dark thought no differently from a hobby: it notices that you lingered, and it offers up more of the same. With a recipe or a football clip, that is harmless. With your lowest moment, the kind of thought a better day would have let drift off, it is not. Some of us spend a long time learning to let those moments pass, and we have built an industry, running at enormous scale, designed to reach out and catch them before they can.

The Cost Is Not Shared Evenly

There is one more thing this algorithm does, and it is easy to miss because it has nothing to do with the content. It does not press on everyone equally. The same nudge reaches all of us, but it does not land the same way, and I want to go carefully here, because it is easy to put wrongly.

Screen time follows what researchers call a social gradient. Across a range of studies, and in Denmark in particular, children and young people from less well-off homes tend to spend more time on screens, not less. A Danish review by VIVE, the national centre for welfare research, found this pattern across national survey data on children and young people rather than in a single snapshot. The authors are careful to say the picture is not uniform across every measure, and that it is often hard to know whether heavy screen use is dragging well-being down or low well-being is driving people towards their screens. But the skew towards heavier use among the less well-off shows up consistently enough to take seriously.

The reason is not that poorer people are somehow weaker. It is structural, and fairly ordinary once you say it out loud. Reaching for a screen when life bears down on you is a small controllable world when the real one feels like anything but. What differs between people is the other escapes they can afford. If you have money, a hard week has other exits: a gym, a therapist, a weekend away, a restaurant visit, time bought back with help you pay for. If you do not, the free app is the cheapest comfort available, and it is free precisely because you are the thing being sold. So the people with the least room to pull away tend to be the ones who spend the most time inside it.

And they know. This is not a story about people who cannot see what is happening to them. In a study of Danish university-college students, nearly a quarter scored high-risk on a standard smartphone addiction scale, a group that in practice was averaging more than six and a half hours a day on their phones. Of that group, almost three-quarters called their own use a problem, and 91 percent had already thought about cutting down. Recognising the problem, and wanting to pull back from it, has plainly not been enough on its own to change how the hours add up.

The stakes are not marginal, either. The country’s own evaluation institute found that young people’s well-being begins to slip once daily leisure screen time climbs above two to three hours, and the students already flagged as high-risk above are logging far more than that. The national health authority has gone as far as to tell the country outright that screens should take up less of everyday life. When a public-health body feels the need to say that in plain language, the default has already drifted a long way from anywhere anyone would have chosen to put it.

The Work Is Left to the User

Let me be clear about what I am saying and what I am not saying, because this is easy to overstate. I am not saying social media is useless. I use these services too, and some of them have given me a great deal, particularly when I moved across the country and when I lived abroad. What I am saying is more limited, and I think harder to shrug off: a system built to maximise the time you spend will, left to its own devices, drift towards whichever version of you is easiest to keep hold of. Nobody chose that drift on your behalf, and it does not weigh on everyone the same.

The tools to push back on it do exist, and I am glad they do. It was Instagram’s own controls, in the end, that quietened my feed. But look closely at what that fix really is. It is something I had to know about, go looking for, and then keep on top of, one setting at a time, on one app, for myself. And the controls force me to tell it, topic by topic, exactly what I do not want to see. The effort sits entirely with the user. The default underneath it, which is to optimise for retention whatever that takes, is left exactly where it was, running the same way for everyone who never finds the settings, or never has the time to go hunting for them.

This is the part I cannot quite make peace with. We have built something very good at holding human attention, aimed it at whichever version of us is cheapest to keep, and then handed each person the job of fighting it off alone. An algorithm tuned to draw out the worst version of us was switched on for everyone by default, and the work of switching it back off was left to whoever happens to have the time, the knowledge, and the room in their life to try. It is the wrong way round.

The part that unsettles me most sits underneath all of it. Look across these studies and audits and the same detail keeps returning: so many of them are about the young. They are at once the least able to pull away and the ones whose sense of the world is still forming around what the algorithm shows them. The teenagers whose feeds fill with the manosphere, or tip towards self-harm, are the coming generation of voters, parents, and lawmakers. The default nobody chose is helping to shape the people who will one day choose the defaults for everyone else, and it is not the kind of thing that fades on its own.