CHATBOT KEYLOGGING — Hackers can read private AI assistant chats even though theyre encrypted All non-Google chat GPTs affected by side channel that leaks responses sent to users.
Dan Goodin – Mar 14, 2024 12:30 pm UTC EnlargeAurich Lawson | Getty Images reader comments 4
AI assistants have been widely available for a little more than a year, and they already have access to our most private thoughts and business secrets. People ask them about becoming pregnant or terminating or preventing pregnancy, consult them when considering a divorce, seek information about drug addiction, or ask for edits in emails containing proprietary trade secrets. The providers of these AI-powered chat services are keenly aware of the sensitivity of these discussions and take active stepsmainly in the form of encrypting themto prevent potential snoops from reading other peoples interactions.
But now, researchers have devised an attack that deciphers AI assistant responses with surprising accuracy. The technique exploits a side channel present in all of the major AI assistants, with the exception of Google Gemini. It then refines the fairly raw results through large language models specially trained for the task. The result: Someone with a passive adversary-in-the-middle positionmeaning an adversary who can monitor the data packets passing between an AI assistant and the usercan infer the specific topic of 55 percent of all captured responses, usually with high word accuracy. The attack can deduce responses with perfect word accuracy 29 percent of the time. Token privacy
Currently, anybody can read private chats sent from ChatGPT and other services, Yisroel Mirsky, head of the Offensive AI Research Lab at Ben-Gurion University in Israel, wrote in an email. This includes malicious actors on the same Wi-Fi or LAN as a client (e.g., same coffee shop), or even a malicious actor on the Internetanyone who can observe the traffic. The attack is passive and can happen without OpenAI or their client’s knowledge. OpenAI encrypts their traffic to prevent these kinds of eavesdropping attacks, but our research shows that the way OpenAI is using encryption is flawed, and thus the content of the messages are exposed. Advertisement
Mirsky was referring to OpenAI, but with the exception of Google Gemini, all other major chatbots are also affected. As an example, the attack can infer the encrypted ChatGPT response: Yes, there are several important legal considerations that couples should be aware of when considering a divorce,
as: Yes, there are several potential legal considerations that someone should be aware of when considering a divorce.
and the Microsoft Copilot encrypted response: Here are some of the latest research findings on effective teaching methods for students with learning disabilities: …
is inferred as: Here are some of the latest research findings on cognitive behavior therapy for children with learning disabilities: …
While the underlined words demonstrate that the precise wording isnt perfect, the meaning of the inferred sentence is highly accurate. Enlarge / Attack overview: A packet capture of an AI assistants real-time response reveals a token-sequence side-channel. The side-channel is parsed to find text segments which are then reconstructed using sentence-level context and knowledge of the target LLMs writing style.Weiss et al.
The following video demonstrates the attack in action against Microsoft Copilot: Token-length sequence side-channel attack on Bing.
A side channel is a means of obtaining secret information from a system through indirect or unintended sources, such as physical manifestations or behavioral characteristics, such as the power consumed, the time required, or the sound, light, or electromagnetic radiation produced during a given operation. By carefully monitoring these sources, attackers can assemble enough information to recover encrypted keystrokes or encryption keys from CPUs, browser cookies from HTTPS traffic, or secrets from smartcards, The side channel used in this latest attack resides in tokens that AI assistants use when responding to a user query.
Tokens are akin to words that are encoded so they can be understood by LLMs. To enhance the user experience, most AI assistants send tokens on the fly, as soon as theyre generated, so that end users receive the responses continuously, word by word, as theyre generated rather than all at once much later, once the assistant has generated the entire answer. While the token delivery is encrypted, the real-time, token-by-token transmission exposes a previously unknown side channel, which the researchers call the token-length sequence. Page: 1 2 3 4 Next → reader comments 4 Dan Goodin Dan Goodin is Senior Security Editor at Ars Technica, where he oversees coverage of malware, computer espionage, botnets, hardware hacking, encryption, and passwords. In his spare time, he enjoys gardening, cooking, and following the independent music scene. Advertisement Channel Ars Technica ← Previous story Related Stories Today on Ars
