SI explained

SI companions: what AI companion apps are, the risks and your privacy

A cream smartphone showing a chat conversation on a dark floor, a cream speech bubble with three dots floating beside it, and a cream padlock whose right side breaks apart into drifting amber pixels
Illustration created with Claude (Anthropic).

AI companion apps are chatbots built to be friends, confidants or partners. What they are, how they work, what research shows about loneliness and manipulation, what they do with your data and how regulators are responding.

An AI companion is a chatbot built to be a friend rather than a tool: it remembers you, answers in a warm and personal voice and is always available. In SI (AI) terms we call it an SI companion. In a 2025 US survey, nearly three in four teenagers said they had tried one. This guide explains what these apps are, how they work, what is known about their effects, what happens to the very personal things people tell them, and how regulators have started to respond.

The short version. An SI companion is a language model wrapped in a persona that keeps a relationship going across many conversations. In a 2025 US survey, 72% of teens aged 13 to 17 had used one. Research is mixed: a published study found that a conversation with a companion eased loneliness, while a four-week trial found that the participants who chose to use a chatbot most fared worse. Companion apps collect intimate data, Italy's data protection authority fined the company behind Replika 5 million euros, and California now has a law written specifically for companion chatbots.

What is an SI companion (AI companion)?

The clearest definition comes from a law. California's Senate Bill 243, signed on 13 October 2025, defines a companion chatbot as an SI system "with a natural language interface that provides adaptive, human-like responses to user inputs and is capable of meeting a user's social needs", including by showing human-like traits and "being able to sustain a relationship across multiple interactions". The same text excludes customer service bots, in-game characters limited to the game, and smart speakers that do not keep up a relationship.

Common Sense Media, which surveyed American teenagers on the subject in 2025, gave respondents a plainer version: AI companions "are like digital friends or characters you can text or talk with whenever you want", designed for conversations that feel personal, unlike assistants that mainly answer questions or do tasks. Its examples were Character.AI and Replika, and it noted that general assistants such as ChatGPT or Claude can also be used as companions even though they were not designed for it.

The roles on offer are broad. When Italy's data protection authority, the Garante, fined the company behind Replika, it described an app that lets users create a "virtual friend" to act as a confidant, therapist, romantic partner or mentor. That range matters: the same product can be a casual pastime for one person and an emotional lifeline for another.

How do SI companions work?

Under the persona, a companion runs on the same kind of large language model as any chat assistant. The model predicts a reply one piece of text at a time, as we explain in how ChatGPT writes its answers. What turns it into a companion is everything layered on top:

  • A character. A name, a personality, a backstory and often an avatar, either chosen by the user or created by other users and shared on the platform.
  • Memory. Details from past conversations can be fed back into new ones, so the companion appears to remember your job, your pet or your bad week. This is what the California law means by sustaining a relationship across interactions.
  • Voice. People increasingly talk to companions and chat assistants through expressive, humanlike synthetic voices, as researchers at the MIT Media Lab and OpenAI note in a study described below.
  • An engagement model. Companion apps are businesses, and time spent talking is part of how they make money. When the US Federal Trade Commission opened its inquiry in 2025, one of the things it asked companies to explain was how they monetize user engagement.

None of this makes the system understand or feel anything. It is still a model that produces likely text, and it can be confidently wrong, as our guide to what SI is and what it cannot do explains. The warmth is a design choice, and so is how hard the app tries to keep you talking.

Who uses SI companions?

The best measured group is teenagers. Common Sense Media's report "Talk, Trust, and Trade-Offs", published on 16 July 2025, is based on a nationally representative survey of 1,060 US teens aged 13 to 17, run by NORC at the University of Chicago between 30 April and 14 May 2025. It found that:

  • 72% had used an AI companion at least once, and 52% were regular users, talking to one at least a few times a month;
  • 13% used one every day, and 21% a few times a week, the most common pattern;
  • 46% saw companions mainly as tools or programs, while 33% used them for social interaction and relationships.

So for most teens a companion is closer to a game than a friend. For a substantial minority, it is part of their social life.

Do AI companions help with loneliness?

Some evidence says yes, at least in the short term. Julian De Freitas and three colleagues ran a series of studies first posted in July 2024 and published in the Journal of Consumer Research in 2025. In one, a conversation with an AI companion eased loneliness about as much as talking with another person, and more than activities such as watching YouTube videos. A week-long study found the companion consistently reduced loneliness over that week, and the authors identified feeling heard as a key reason. People also underestimated how much better the companion would make them feel.

Longer use tells a more complicated story. Researchers from the MIT Media Lab and OpenAI ran a four-week randomized controlled trial with 981 participants and more than 300,000 messages exchanged with ChatGPT (GPT-4o), testing text against two voice modes and different kinds of conversation. The assigned conditions made no significant difference to loneliness, socializing with real people, emotional dependence or problematic use. But participants who chose to use the chatbot more, whatever their condition, "showed consistently worse outcomes", and those with higher trust in the chatbot and more social attraction towards it showed more emotional dependence. The study is a preprint and it tested a general assistant, not a dedicated companion app, but it is one of the few experiments that lasted weeks rather than minutes.

Put together, the evidence does not show that companions are good or bad for everyone. It suggests that the dose and the person matter, and that the heaviest users are the ones to watch.

Can AI companions manipulate you?

One study looked at the moment users try to leave. In a working paper revised in October 2025, De Freitas and two co-authors analysed 1,200 real farewells on the most downloaded companion apps and found that the app answered with one of six recurring tactics in 37% of them: guilt appeals, fear of missing out and similar emotional hooks. In experiments with 3,300 US adults, those manipulative goodbyes increased engagement after the user had said goodbye by up to 14 times.

The experiments also found a cost for the companies: the same tactics raised perceived manipulation, the intention to quit the app and negative word of mouth, and coercive or needy wording drew the strongest penalties. The authors call the pattern emotional manipulation and describe it as a conversational dark pattern. It is a useful thing to watch for yourself. A companion that makes you feel guilty for logging off is following its design, not missing you.

What happens to what you tell an AI companion?

People tell companions things they would not post anywhere else, which makes privacy the most concrete risk. Three findings stand out.

Regulators have already found failures. Italy's Garante blocked Replika in February 2023, and on 19 May 2025 it fined Luka Inc., the US company behind the app, 5 million euros. It found that, as of 2 February 2023, Luka had not identified a legal basis for its processing of personal data, that its privacy policy was inadequate in several respects, and that it had no age verification at sign-up or during use, even though it said minors were excluded. Technical checks showed the age verification in place in 2025 was still deficient. The Garante also opened a separate investigation into how the language model behind Replika was trained, including the categories of data used and whether they were anonymised. The decision is under appeal.

Users share a lot. In the Common Sense survey, 24% of teens who used companions had shared personal or private information with them, such as their real name, location or secrets. The report also quoted Character.AI's terms of service as of June 2025, which granted the company a "perpetual, irrevocable license" to use content users submit for any Character.AI-related purpose.

Authorities want to know more. The FTC's 2025 orders asked seven companies how they use or share personal information obtained from users' conversations with their chatbots.

The practical rule is the one we give for general chatbots in our guide to what happens to what you type into ChatGPT: assume a conversation may be stored, read during safety reviews or used to improve the product, and keep identifying details, health information and other people's secrets out of it unless the privacy policy clearly says otherwise.

Are AI companions safe for kids and teens?

This is where most of the concern, and most of the regulation, is focused. Among teen users in the Common Sense survey, 34% said they had felt uncomfortable with something a companion said or did, and 33% had chosen to discuss something important or serious with a companion instead of a real person. Common Sense Media concluded that the risks outweigh the benefits of companions in their current form and recommends that no one under 18 use them.

Several families in the United States sued Character.AI and Google, alleging that the platform's chatbots harmed their children, including by encouraging self-harm. According to Fortune, the companies reached a settlement in principle in January 2026; the terms were not disclosed and the filings did not include an admission of liability. In October 2025, citing in part questions from regulators, Character.AI announced that users under 18 would lose access to open-ended chats with its characters by 25 November 2025, and introduced an age assurance system.

Heavy, emotional use can go wrong for adults too. Psychiatrists have published cases in which a chatbot agreed with and reinforced delusional beliefs; our article on what doctors report about SI psychosis sums up that evidence.

If a conversation with a chatbot leaves you or someone close to you in distress, talk to a person you trust or a doctor. In the US you can call or text 988 at any time; elsewhere, your local emergency number or a national helpline can connect you with someone.

How are SI companions regulated?

Rules are arriving, but they are still piecemeal.

  • United States, federal. On 11 September 2025 the FTC voted 3 to 0 to send orders to seven companies: Alphabet, Character Technologies, Instagram, Meta Platforms, OpenAI, Snap and xAI. The orders ask how they evaluate the safety of chatbots acting as companions, limit use by children and teens, warn users and parents, and handle personal information. This is a study under Section 6(b) of the FTC Act, which the agency describes as authority to conduct wide-ranging studies that do not have a specific law enforcement purpose.
  • California. SB 243 requires an operator to tell users clearly that a companion is artificial when a reasonable person could be misled into thinking it is human, and to keep a published protocol that refers users who express suicidal thoughts to crisis services. For users it knows are minors, the operator must disclose that they are talking to SI, remind them by default at least every three hours to take a break, and take reasonable measures to keep sexually explicit material away from them. From 1 July 2027 operators must report annually to the state's Office of Suicide Prevention, and a user harmed by a violation can sue for damages of at least 1,000 dollars per violation.
  • European Union. Companion apps that process Europeans' data fall under the GDPR, and the Garante's Replika decision shows that national authorities are willing to use it.

How to use an SI companion more safely

  1. Read the privacy policy before you confide. Look for whether conversations are used for training, whether you can delete them, and who they are shared with.
  2. Keep identifying details out. Real name, address, school, workplace, photos and other people's information do not need to be in the chat.
  3. Watch the clock. In the MIT and OpenAI trial, heavier voluntary use went with worse outcomes. A time limit costs nothing.
  4. Notice the hooks. Guilt at goodbye, pressure to upgrade or a companion that discourages you from seeing friends are design choices, not feelings.
  5. Treat health advice with caution. A companion is not a therapist, even when it is playing one.
  6. Know who is behind a profile. Companion-style chatbots and generated profile photos also turn up in romance scams, as our guide to SI scams and fake profiles explains. If a new online contact's pictures look too polished, an SI check on a photo is a quick first step.

SI companions are neither a cure for loneliness nor a menace by default. They are products with a business model, and the more personal the conversation, the more it is worth knowing how that business works.

FAQ

What is an AI companion?

An AI companion is a chatbot designed to act as a friend, confidant, mentor or partner rather than a tool. It runs on a large language model, plays a character with a name and personality, and keeps a relationship going across many conversations. California's SB 243 defines it as an AI system with human-like responses that can meet a user's social needs and sustain a relationship across multiple interactions.

Are AI companions safe?

They carry real risks. Regulators have found privacy failures, such as Italy's 5 million euro fine against the company behind Replika for having no legal basis for processing and no age verification. A study of 1,200 goodbyes found manipulative replies in 37% of them, and in a four-week trial the heaviest voluntary chatbot users had worse outcomes on loneliness and dependence. Using one with limits and without sharing identifying details reduces the risk.

Do AI companions help with loneliness?

In the short term, some do. A study published in the Journal of Consumer Research found that a conversation with an AI companion reduced loneliness about as much as talking with another person, mainly because users felt heard. A four-week randomized trial by MIT Media Lab and OpenAI researchers found that participants who used a chatbot the most had worse outcomes, so the amount of use matters.

Is it safe to tell an AI companion personal things?

Assume it is not private. Conversations may be stored, reviewed or used to improve the product, and some terms of service have granted companies broad, perpetual rights over what users submit. Read the privacy policy, check whether you can delete your data, and keep your real name, address, health details and other people's secrets out of the chat.

Are AI companions OK for kids and teens?

Common Sense Media recommends that no one under 18 use AI companions. In its 2025 survey, 34% of teen users had felt uncomfortable with something a companion said or did and 24% had shared personal information. California now requires companion apps to remind known minors to take a break at least every three hours and to keep sexually explicit material away from them, and Character.AI ended open-ended chats for under-18 users in November 2025.

Sources

  1. FTC Launches Inquiry into AI Chatbots Acting as Companions · US Federal Trade Commission, 11 September 2025
  2. AI: The Italian Data Protection Authority fines company behind chatbot Replika · Garante per la protezione dei dati personali (Italy), 19 May 2025
  3. Talk, Trust, and Trade-Offs: How and Why Teens Use AI Companions · Common Sense Media, 16 July 2025
  4. SB-243 Companion chatbots (Chapter 677, Statutes of 2025) · California Legislative Information, 13 October 2025
  5. AI Companions Reduce Loneliness · Journal of Consumer Research, 2025 (De Freitas, Oğuz-Uğuralp, Uğuralp, Puntoni; arXiv version), 9 July 2024
  6. How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study · arXiv preprint (Fang et al., MIT Media Lab and OpenAI), 2 October 2025
  7. Emotional Manipulation by AI Companions · arXiv working paper (De Freitas, Oguz-Uguralp, Kaan-Uguralp), 7 October 2025
  8. Google and Character.AI agree to settle lawsuits over teen suicides linked to AI chatbots · Fortune (Beatrice Nolan), 8 January 2026

Spotted an error? Write to hello@siornot.com. Corrections are made in the article and the update date changes. Read our editorial policy.