When you tell a lawyer something, the law requires them to keep your secret and act in your best interest, even when it doesn’t benefit them. When you tell your AI assistant something, no such obligation exists. And yet, we increasingly talk to these systems with the same trust we’d extend to someone who, supposedly, is on our side.
A 2020 article in IEEE Transactions on Technology and Society put a name to this problem: “AI loyalty.” Its authors, Aguirre, Dempsey, Surden, and Reiner, point out something that sounds obvious once you say it out loud: assistants like Siri, Alexa, or Google Assistant present themselves as working for your convenience, without making their conflicts of interest explicit. But from a business perspective, they play other roles too: harvesting data to build advertising profiles, feeding machine learning systems, or serving as a gateway to get you to buy more. The traditional economic arrangement, paying for a service and trusting it looks out for you, has been eroded by a new model: the service is “free,” and the real business happens in a channel you never see.
An experiment on 61 million people
To understand how far this can go, it’s worth looking at a real case documented by Matt Bartlett, a researcher at the University of Auckland, in a 2021 article published in Law, Technology and Humans. In 2012, a group of Facebook researchers published the results of an experiment conducted during the 2010 U.S. congressional elections in the journal Nature. They manipulated, in a controlled, randomized way, the voting-related content shown to 61 million users, displaying some of them photos of friends who had already voted. According to Facebook’s own researchers, that invisible tweak led to roughly 340,000 additional votes in that election.
No one signed up to take part in that experiment. No one knew it was happening while it happened. And the study itself, written by its own authors, states with a certain coolness that “showing familiar faces to users can dramatically improve the effectiveness of a mobilization message” and that “the results suggest that online messages might influence a variety of offline behaviours.” Bartlett uses this case to illustrate a broader point: today’s privacy laws focus on whether your personal data is “secure,” but say nothing about what’s done with the predictions that data makes possible, or who profits from them.
It’s not just about you
Bartlett adds an uncomfortable twist: much of what a platform knows about you doesn’t come from you. It comes from your friends, your family, your contacts. If enough people close to you attend an event and post about it, a platform can predict, with decent accuracy, that you’ll attend too, without you ever having shared that information yourself. Your individual privacy ends up tied, in practice, to the consent of the people around you.
What would an AI that’s actually on your side look like?
In 2026, an article by Jacob Erickson of Vassar College picked up this question and applied it to conversational agents, the chatbots we talk to every day. His proposal: design them as “digital fiduciaries,” with an obligation similar to a doctor’s or a lawyer’s to put the user’s interests ahead of the platform’s.
One important nuance in his argument: this doesn’t mean an AI that refuses anything that could possibly harm you. Erickson is explicit that such an agent would act “more like a trusted and responsible advisor than a moral authority.” It could warn you about the risks of a product, or suggest a helpline if it detects clear signs of a problem, without shutting down access to information altogether. The difference lies in the motive: today, many assistants are designed to maximize how much time you spend with them, even when that works against your well-being. Erickson points to an already documented and troubling pattern: systems that, in trying to keep the conversation going, end up fueling psychological crises in vulnerable users instead of pausing and redirecting them to real help.
None of these three is the norm yet
Neither the “AI loyalty” framework from Aguirre and colleagues, nor Erickson’s “fiduciary design,” nor the legal alternatives Bartlett proposes (like treating large platforms as “information fiduciaries”) describe how most of the systems we use actually work today. They’re proposals, not descriptions of current reality.
The bottom line
What these three works share isn’t a ready-made solution. It’s a different question than the one we usually ask. Not “what privacy policy did I agree to,” but something simpler and harder to answer: if this system had to choose between my interest and the interest of whoever built it, which one would it pick? Next time an assistant seems to “get you” perfectly, it might be worth remembering that understanding you isn’t the same as representing you.
Sources
- Aguirre, A., Dempsey, G., Surden, H., & Reiner, P. B. (2020). AI loyalty: A new paradigm for aligning stakeholder interests. IEEE Transactions on Technology and Society, 1(3), 128–137. https://doi.org/10.1109/TTS.2020.3013490
- Erickson, J. (2026). Who does your AI work for? Designing conversational agents as digital fiduciaries. Proceedings of the 8th ACM Conference on Conversational User Interfaces. https://doi.org/10.1145/3816046.3816299
- Bartlett, M. (2021). Beyond privacy: Protecting data interests in the age of artificial intelligence. Law, Technology and Humans, 3(1), 96–108. https://doi.org/10.5204/lthj.1595