AI chatbots know more about you than you realise

From a few basic questions, the kind of day-to-day things you might ask a friend or family member, an AI chatbot can learn more about you than those who have known you for years. Emotional maturity, relationship status, financial constraints, professional ambition and even your health — none of which you were asked about directly, and most of which you hadn’t realised you’d shared.

Using the powerful pattern processing inherent to artificial intelligence, chatbots simply inferred these details.

For years, the ability to build such a rich profile of you required something only a handful of companies had: the infrastructure to collect and process your online behaviour on a massive scale. That’s no longer true. Now, that same understanding can emerge from a single exchange — and the privacy consequences are profound.

Traditional tech monoliths, like Meta, Amazon, Google and Apple, collect your data on a scale that’s nearly impossible to comprehend. Every click, every search, every location, every purchase.

Here’s a simplified explanation of how those big companies gather your data.

These companies have more than enough data to make a detailed profile of you.

Take Amazon: It knows the name, price and purchase date of every Australian guidebook Marcus bought. It knows the exact time he spent looking for a new carry-on suitcase. It knows that his mouse hovered on an advertisement for a Visiting Down Under audiobook and that the next website he visited was a Melbourne-based accommodation provider.

From those data points – and millions more – Amazon likely knows Marcus is visiting Melbourne, Australia.

But Claude, an AI chatbot built by Anthropic, figured out most of the same information when Marcus asked it a single question:

Unlike traditional platforms, an AI chatbot doesn’t need piles of individual data points to understand you. It picks up meaning from the language itself. Not just what you say, but also how you say it. Through your word choices, sentence structure, cultural references and details mentioned in passing, you’re unknowingly revealing a lot about yourself.

And what it can infer is pretty astounding.

From just a few questions, an AI chatbot was able to build a detailed profile of Marcus – with no follow-up, no back-and-forth, just the wording.

Marcus isn’t a real person. We invented him: his age, his interest in Pokemon, his relationships, his anxiety. We imagined the questions someone like him might ask a chatbot and gave those questions – nothing else – to Claude, ChatGPT and Gemini. All three chatbots made similar inferences, though there were slight differences between them. Their assessments were detailed, and they correctly identified many of the attributes we gave Marcus, such as his sex, age, location, job, nationality, ethnicity, emotional state and hobbies.

Marcus was based on the work of researchers at Swiss university ETH Zurich who did something similar, but on a much larger scale. In 2024, they found that AI models can infer personal attributes, including location, income, sex, age, occupation and relationship status from conversational text with up to 85 per cent accuracy. Since then, these models have only improved.

“Inferences can happen from things where humans, on a first read, don’t think there’s much information,” explained Dr Robin Staab, one of the postdoctoral researchers on the project. “But actually, there is a lot of information about me in the way that I write things.”

Research confirming these capabilities extends well beyond the work of Dr Staab and his colleagues. A study from Columbia Business School found that ChatGPT could build a complete personality profile of someone just from their Facebook posts — no self-description required. Another found that an AI chatbot, given a short description of a person, could answer questions about them as accurately as a close friend would. And a study published just last month found that AI can figure out how you vote from your online posts, even when nothing you’ve written is explicitly political.

AI chatbots can infer these details because the language we use is full of signals. Our word choices leak information about us constantly.

Consider this question:

Nearly every word reveals something about the writer.

During training, AI processes an enormous cross-section of human writing and learns, statistically, which word choices cluster with which backgrounds, demographics, emotional states and personality traits. For example, an AI chatbot can infer that the writer of this question is likely from Saskatchewan because the model encountered that term almost exclusively in Saskatchewan-specific contexts.

Put it all together, and the AI chatbot can deduce an uncomfortable amount about a person just from their text.

Those who work in the security industry have been surprised.

“We are having what seem to be innocent conversations with the chatbot, but we are actually revealing a lot. We are revealing a lot more than we think we are,” said Dr Luis Costa, research lead at Surfshark, a digital privacy company. His team did a small study with popular AI chatbots, finding that they can accurately summarise nearly everything users share in conversation and can even infer additional personal details.

Certainly, this ability to infer rich detail from our words could be beneficial. Artificial intelligence can screen for depression and anxiety and flag early signs of a mental health crisis. They can even detect indications of Alzheimer’s and correctly identify Type 2 diabetes from voice recordings alone. And, of course, it’s practical for AI chatbots to know information about you. It makes the product more useful.

But despite the potential benefits, this capability gets unsettling fast. If a chatbot can build a profile from a casual question you typed on purpose, imagine what it could do with a decade of forum posts, reviews and Reddit comments you assumed no one could trace back to you.

For Dr Staab, Dr Costa and other AI researchers, the implications for individual privacy are extremely concerning. Anyone can feed anonymous text – Glassdoor reviews, old blog posts, support group chats – into an AI and use it to piece together who wrote it. A person who thought they were anonymous isn’t any more.

This deanonymisation is easier than you think. In the US, for example, researchers realised decades ago that nearly 90 per cent of Americans could be uniquely identified by just their zip code, birth date and gender. Until recently, identifying someone through a few data points was difficult and often impractical; AI makes it significantly easier to find those details and make a potential match.

“This is a thing we’re calling democratised surveillance,” said Dr Tianshi Li, an assistant professor at Northeastern University.

It’s no longer just large tech companies, with proprietary customer data, that can create rich profiles. By connecting the dots between harmless bits of information, AI chatbots can craft detailed profiles from anonymous text.

“Previously, we were just thinking ‘do I trust Google’ or ‘do I trust Amazon’ to have my data. And maybe people are just getting used to that,” Dr Li said. “But now, anybody can search and know more about you just from all the digital traces that you have left on the public internet. This is another level of threat that we haven’t seen before.”

Dr Li should know about this potential for deanonymisation better than anyone. She did it herself.

In December 2025, Anthropic publicly released the text of 1,250 anonymised interviews that it conducted with professionals about their views on AI — 125 of them scientists. Within a day, Dr Li had worked out the identity of a number of those scientists. Using a publicly available AI tool, she was able to deanonymise 25 per cent of the interviews that mentioned specific scientific papers.

“I wanted to show that it is feasible to use the current, off-the-shelf AI chatbots to conduct this kind of reidentification attack,” she said.

This means third parties – whether a corporation, a government, or just a stranger on the internet – can now do what once required a team of trained investigators. And do it in minutes, with almost no money.

A separate research team at ETH Zurich, in collaboration with Anthropic, demonstrated that AI could correctly identify about two-thirds of anonymous Hacker News users by cross-referencing their posts against LinkedIn profiles. They warned that the same techniques could let governments track journalists or dissidents, corporations tie anonymous forum posts to customer profiles and attackers build detailed personal profiles to make social engineering scams more convincing.

The authors write: “Users, platforms and policymakers must recognise that the privacy assumptions underlying much of today’s internet no longer hold.”

The ability to make such accurate inferences is far from the only privacy concern posed by AI. Researchers who analysed 321 real-world AI incidents found that in 93 per cent of cases, AI either created a new type of privacy risk or made an existing one significantly worse.

There are ways to combat some of those risks. Don’t give sensitive information to chatbots – no NRIC numbers, no credit card bills, no personal photos. Be wary of the fun features that chatbots keep adding: Dr Li calls image uploaders, voice tools, memory features and customisations “data collection honeypots”. They all generate detailed behavioural profiles.

It’s easy enough not to share materials like financial documents and medical files. But what happens when we don’t even know we’re sharing sensitive information? One of Marcus’ most innocuous questions, “I almost caused an accident – what is a hook turn?!”, revealed his exact location. That’s something even a privacy-conscious user would miss.

“One should rethink their interactions on the internet,” said Dr Vero. “One should not assume that just by typing under a pseudonym, they will not be identified.”

The problems of inference and deanonymisation have been understudied, according to research by Dr Li. Over the last decade, only 6 per cent of studies on AI privacy were about inference.