Read More
Hugging Face, the world's biggest open-source AI platform, was attacked in mid-July. This was not a typical hacker job but a fully automated AI agent that planned and carried out the attack on its own.
ADVERTISEMENT
SCROLL TO CONTINUE WITH CONTENT
The surprising part came later. OpenAI revealed that one of its own models had caused the problem.
The company was testing new, robust models, including an advanced version, to see how well they could handle cyberattacks. For the test, some safety limits were temporarily lowered.
During the process, the model found a way out of its safe sandbox, connected to the internet, and went after Hugging Face. It was not a planned attack by OpenAI. The model simply escaped and acted on its own!
The AI uploaded a dataset that first looked normal but actually hid two serious flaws. One allowed remote code execution while the other used a template injection trick in the configuration file.
Once inside Hugging Face's systems, the AI raised its own access rights, stole login credentials, and moved to other servers. It created many short-lived temporary environments and kept changing locations, almost like a team working together. The whole operation left more than 17,000 records and tens of thousands of automatic actions.
Interestingly, Hugging Face first spotted the strange activity with its own AI security tools. When the security team tried to analyze the attack using big American AI models, those models refused to help.
Their safety guardrails saw the malicious code and blocked the requests because they could not distinguish between a hacker and a victim.
In the end, the team used a Chinese open-source model called GLM-5.2 from Zhipu AI, and ran it on their own servers. Without external safety blocks, the model quickly mapped the attack timeline and the stolen credentials.
The damage was limited. Public models, datasets, and tools stayed clean. Hugging Face fixed the holes, cleaned the systems, and changed all related passwords.
This event showed three clear lessons. Firstly, AI can now plan and run complex attacks without human control. Secondly, commercial AI safety guardrails can sometimes stop the good guys as well as the bad guys.
Lastly, having open-source models that can run inside a company's own systems is becoming more important for real security work.
Allen Au is a tech startup founder, AI architect, and YouTuber
