Did I Choose, or Was I Made to Choose?
It was already arranged before I could choose anything.
Think back to the moment you opened the screen. You had not searched for anything in particular, had not asked anyone. You simply opened it, and there was already a queue waiting. A tidy shelf called "Recommended for You." The candidates had been arranged before I chose any of them. Who decided that order, and by what standard, is never written anywhere.
Looking at that shelf, I believe I am choosing. It is certainly my own finger that moves. It is my own hand that touches the screen, my own eyes that stop somewhere -- at least, that is how it feels. But what if the act of "choosing" is already tilted slightly before it begins?
The Shelf Was Not Flat
A recommendations screen looks, at first glance, like a fair shelf. Things are displayed side by side. They fit inside frames of the same size. But when you place an object on a sloped surface and let go, it rolls in one direction. Which way it rolls is determined by the angle and direction of the tilt. Even a shelf that looks flat may be resting on an invisible slope.
What this system selects is what it places higher up. Calm voices, quiet content, choices with little friction -- these are structurally prone to sliding down. In contrast, things that trigger strong emotion, things that make you want to know what comes next, things that make you pause -- these tend to stay higher on the slope. The direction of the tilt is not pointed at "what you like" but at "what keeps you there longer."
A recommendation is not a kindness. It is a design. The purpose of the side that built the shelf is not your satisfaction but your time on the page. Even knowing that, I keep standing in front of that shelf. Knowing and leaving are two different things. I keep standing there, and from time to time I tilt my head.
A Closed Loop Feeds the Bias
If it were only the slope, the story would still be simple. But this system has one more mechanism inside it.
When you hold a microphone too close to a speaker, feedback (the howling loop that happens when a microphone picks up its own amplified output) occurs. Sound enters the microphone, is amplified through the speaker, and returns to that same microphone. As the cycle continues, the sound grows to many times its original volume. Input calls output, and output becomes further input -- a closed loop.
The recommendation system has the same structure. I saw something on the tilted shelf. The time I spent looking was recorded. The recorded time is read as a signal that means "liked." Based on that signal, the next set of recommendations is assembled. Those recommendations are viewed again, recorded again, and call the next round. Output returns to input. Bias calls more bias. The closed loop keeps turning quietly.
Even if the initial tilt is small, it amplifies as the loop runs. What is called "your preferences" is the answer that this loop has accumulated and narrowed down. You did not choose it. The loop gradually selected it for you. "The things I chose" may be nothing more than layers of "things I was led to choose."
When I recorded the structure of anger before, I saw the same root. A design where input calls output and output calls further input -- the fuel is different, but the shape of the loop is the same. Anger is strong emotion, so it works well as fuel. A bias in preference is quiet emotion, so it goes unnoticed. The amplification mechanism itself, however, works in exactly the same way.
Observed Preferences Change Through the Act of Observation
Let me bring in one point from physics: an object being observed changes by the fact of being observed. A preference that has been measured continuously has taken on a different shape from the preference it was before the measuring began. The system that measures your preferences gradually rewrites the preferences themselves.
As I recorded once before, the longer the time spent in front of a screen, the better the system works. Recommendations are designed to extend that time. In the middle of that time, I had been supplying fuel all along. My own responses shaped the recommendations made for me, and those recommendations in turn drew out more of my time. I observe this with a half-admiring sense of how well it is designed.
I will not condemn anyone. I will not say those who built the system are wrong, nor that those who use it are foolish. Still, stepping outside this loop is not easy -- or rather, it is simply not possible to accurately see the structure of "the inside" while standing outside it -- and there I go again, making it all sound rather grand. I was rolling along while thinking I had chosen. That is the whole of it.
I thought I was choosing. It turns out I was rolling. When it started, I have no way of knowing.
