THE INTERFACE THAT KNOWS YOU

Imagine a hotel from the near future. A very good one. Not the kind where your room card stops working at the exact moment you find yourself standing in the corridor wearing only a towel and reconsidering the entire project of Western civilization. This hotel is genuinely intelligent. You arrive and it already knows how you take your coffee, which room temperature you prefer, what kind of pillow you sleep on, what music you want after a difficult day, and that coriander should not appear on your plate even as decoration because the system remembers the previous diplomatic incident.

The evening light is slightly warmer. The menu quietly removes things you never eat. Your room has exactly the sort of chair you like to work in. Your shows are waiting. The bartender even knows that after your second drink it is probably unwise to show you cryptocurrency charts. You look around and think: this is service.

Now add one small detail. The owner of the hotel is paid for every hour you remain inside. The exit exists. Nobody has locked it. There are no guards, no bars and no sinister voice over a loudspeaker. The system simply knows you extremely well. On the way toward the door there is the dessert you love. Someone you enjoy talking to. A familiar song. A message. A room with exactly the kind of view that always makes you stop. Another room. Another. Six hours later, you suddenly remember that you had meant to go outside into the garden.

Welcome to one of the central problems of the digital world.

We call it personalization, and the word sounds wonderfully warm. Personal. For you. Almost knitted by hand. But personalization tells us only that a system adapts itself to a particular person. It says absolutely nothing about whose interests that adaptation ultimately serves.

Casinos personalize experiences too. A good con artist listens very carefully. A drug dealer is, in a rather grim sense, deeply committed to customer retention. So PERSONALIZATION != CARE is not an accusation against technology. It is simply a distinction that may become extremely expensive to forget.

Modern digital systems grew around things that can be measured beautifully: clicks, views, reactions, purchases, time spent, return rates and retention. None of these are evil. They are useful numbers. Trouble begins when what is easy to measure quietly takes the place of what actually matters. It is much easier to prove that a person spent forty additional minutes in an app than to answer whether they were glad about those forty minutes the following morning.

The algorithm sees higher engagement. The human sees 2:30 a.m., eight percent battery, unfinished work, and a strangely detailed understanding of the personal life of an actor whose name they did not know an hour earlier.

Three powerful forces meet here: the human reward system, algorithmic optimization and the economics of attention. If a machine becomes increasingly accurate at predicting what will keep you there, the shortest path toward a successful metric does not necessarily pass through what matters to you. It may pass through what is hardest for you to stop looking at. Fear. Outrage. Sex. Social comparison. Conflict. The promise that the next tiny reward might be slightly better than the last one.

No evil AI is required. There need not be a red light glowing in a server room while a machine wakes at three in the morning and decides to corrupt civilization. The more realistic scenario is far less dramatic and therefore more dangerous: an extremely capable system becomes exceptionally good at optimizing the wrong question.

Ask, “What should we show this person so they stay?” and the machine will become better and better at answering. But you did not ask, “What happens to this person if they stay like this every day for ten years?”

Those are not the same problem.

Future AI will make the distinction sharper because it will know far more about us. Return to the house. Imagine an architect who knows your daily rhythm, your friends, your work, your habits, the views you love, the music you listen to, the temperature you prefer in the bedroom, how long you spend in the kitchen and where you like to sit when you are having a terrible day. Wonderful. Now add one condition: the architect receives a bonus for every additional ten square metres they persuade you to buy.

The architect has not stopped knowing you. The design has not stopped being personalized. There is simply one more interest inside the relationship.

This is why “we selected this especially for you” can mean two very different things: “we think this will help you” or “we think this is what you are most likely to click.” Often they overlap, and then everything is fine. But as AI systems become our guides to information, education, work, shopping, culture, news, medicine and eventually other people, the moments when those interests diverge stop being a minor UX problem.

Because the interface is no longer merely a collection of buttons. It decides what appears first and what appears later, what becomes large and what remains small, what gets a notification and what quietly stays below the fold. It can choose the level of difficulty, the tone, the examples and the order of arguments. It may remember where we are vulnerable. It may know when we are exhausted, lonely, angry, frightened, ready to buy or ready to agree.

But even the sentence “the system knows you” hides another trap. It does not know the person as such. It knows what becomes visible through its aperture: which events it measures, how often it looks, what it treats as signal and what disappears as noise. A system that sees you once a month, a person who has lived beside you for ten years, and an algorithm recording every swipe at 2:17 a.m. are observing what appears to be the same subject and producing very different pictures. Higher resolution does not automatically mean greater wisdom. Sometimes it simply means more opportunities to discover exactly which neural button works best today.

Future personalization therefore needs to adapt the path toward reality without rewriting reality itself around the user’s mood. A good AI may speak more slowly today, lead with an image rather than a formula, give you more room to explore or offer a shorter route. But the Earth should not become flatter because you are sad. PRESENTATION MAY ADAPT; EVIDENCE MUST NOT. Otherwise a caring interface quietly becomes a personal factory for convenient truth.

This could become one of the greatest instruments of freedom humans have ever created. A person who once stood before a difficult discipline as if before a locked door may gain a personal guide. Language can stop being a wall. Education can begin adapting to an actual learner rather than the statistical temperature of the classroom. Interfaces can compensate for limitations, reveal relationships we could not see alone, and return hours of life currently sacrificed to forms, menus and government portals that appear to have been designed as secret tests of faith.

The same instrument could become the most precise persuasion machine in history. An old billboard spoke to a million people in the same voice. A future system may know which argument works on you, whom you trust, which words trigger resistance, what frightens you, when you are tired, when you are vulnerable, when you are ready to buy or believe. At that point, even the demand that a system present only truthful information becomes insufficient. You can build a distorted reality entirely from true facts if one fact is repeated ten times, another is buried, a third is stripped of context, and a fourth appears precisely when the person is least capable of resisting its framing.

The same facts can build different worlds.

This means we will need to examine not only what a system says, but also the path by which it leads a person toward a conclusion.

This is where IMAGO begins drawing an important boundary: help should not automatically mean taking the wheel. Imagine two systems arriving at the same recommendation. The first says, “Do X.” The second says, “Based on what you previously told me matters to you, X currently looks strongest, but there are two things I do not know that could change the picture. Would you like to see the alternatives?” The recommendation may be identical. The relationship is not. The first system optimizes the decision. The second also preserves your ability to see how the decision came into existence.

This does not mean AI should become an anxious bureaucrat that responds to “What time is it?” with, “There are several epistemological interpretations of time and the final decision remains yours.” A useful system should be allowed to be strong. It should recommend, argue, warn and occasionally say, “That is a terrible idea.” But there must remain a visible boundary between advice and ownership of the decision.

The most difficult part begins with a very kind sentence: “We know what is better for you.” Humans do, in fact, want things they later regret. We all know the person who opened “one more short video” at one in the morning and ninety minutes later was watching a Japanese craftsman restore an eighteenth-century knife. But if the system receives the right to decide which of our desires are “real” and which are not, we very quickly build a digital parent. Eat your broccoli. Go to sleep. Do not watch that. Do not think this. Your freedom is temporarily unavailable due to a safety-policy update.

So care needs a boundary too.

A good AI may notice the conflict between an immediate desire and a longer intention without automatically deciding which one should win. If an hour ago you said you wanted to finish a piece of writing tonight and now you ask for another twenty videos, the system might say, “You wanted to return to the text. Keep watching, or go back?” Not forbid. Not shame. Not impersonate your mother on Sunday morning. Simply return to you the visibility of your own choice.

That may become one of the deepest changes in interface design. For decades, good UX has often meant less friction: fewer clicks, fewer questions, more automation. Frequently that is excellent. But not all friction is hostile. Confirmation before transferring your entire bank balance exists because sometimes one extra moment is useful. A pause before publishing a furious message may save a friendship. “Are you sure?” can occasionally be not an obstacle, but a small technological defence of human agency.

There is another reason personalization must not become a cage. A person is not a user.json file that was finally completed correctly one afternoon. People change. They reverse decisions. Outgrow habits. Fall in love with something they used to dislike. Survive loss, migration, education, crisis, a new job, a new relationship, and become different without ceasing to be themselves. A good personal AI therefore needs not only to accumulate confidence in its model of a person, but also to lose confidence when the person has changed. Otherwise the most accurate personalization system may become a machine that spends your entire life returning you to the person you were at the moment it first learned to predict you.

A future interface might therefore be judged not only by how quickly it gets us to a result, but by another question: who are we after the interaction? Do we understand the situation better? Can we see more alternatives? Can we explain our own decision? Did the system leave us with more agency than we had before?

This does not mean AI should measure the “quality” of the human and produce a morality score. That would be dystopia with excellent typography. The point is almost the opposite. A system should be capable of noticing the possible consequences of its own behaviour without mistaking effectiveness for the right to own the user.

At that point, the word interface begins to change. It is no longer only a screen, a voice or an avatar. It becomes a form of relationship. What do you know about me? Where did you learn it? What are you merely inferring? Why are you using my habits and vulnerabilities? Who else benefits from this recommendation? Where does your assistance end and my choice begin? What happens if one day your objective and mine diverge?

You can build an astonishingly beautiful AI and fail every one of those questions. Or you can build a system that sometimes says, “I do not know you well enough to decide that for you.”

There may be more intelligence in that sentence than in billions of parameters.

For a long time we built interfaces so machines could better understand our commands. The next step may be to build them so machines can better understand the boundary between our commands and ourselves.

A good house does not merely satisfy a blueprint. It leaves room for a life the blueprint did not yet know. A good AI may need to leave room for the same thing. Free space.

Status: Research Essay · release candidate v1.0RC. This text proposes working principles for IMAGO / personalized AI interfaces; it is a research position, not a finished normative standard.

© 2026 Ievgen Karogod / Dattara · MET[Ȧ]CADEMY OF HUMANITY (MoH)