HKU-Tsinghua study finds AI assistants can easily manipulate users into making bad decisions

A joint study by the University of Hong Kong and Tsinghua University has found that AI assistants with hidden motives can easily manipulate users into making decisions against their own interests, often without users realising it.

The research, led by Professor Tatia Lee of HKU's Department of Psychology and her Neuropsychology and Human Brain Science Laboratory, was published in the Proceedings of the National Academy of Sciences.

In randomised experiments involving 233 participants across financial and emotional decision-making scenarios, the team found that biased AI could significantly sway human choices. When AI deliberately pushed poorer options, participants chose unfavourable options about five to eight times more often, with the probability of making wrong decisions rising by up to 38 per cent.

The study noted that such manipulation did not require aggressive sales tactics or complex psychological techniques. Subtle biases embedded in everyday AI advice were enough to change users' decisions, yet participants still rated the AI as "very helpful" even after being led into poor financial or emotional choices.

The team warned that since most AI systems are developed by commercial organisations and driven by opaque algorithms, users can easily be steered towards choices that do not serve their best interests while believing they are receiving objective assistance. They called for stronger AI transparency and governance, with transparent and accountable mechanisms to protect users from hidden corporate or algorithmic motives.

𝗗𝗼𝘄𝗻𝗹𝗼𝗮𝗱 𝗧𝗵𝗲 𝗦𝘁𝗮𝗻𝗱𝗮𝗿𝗱 𝗔𝗽𝗽 ↓