Chinese chips help Ant cut AI costs by 20pc

Bloomberg

Jack Ma Yun-backed Ant Group used Chinese-made semiconductors to develop techniques for training AI models that would cut costs by 20 percent, according to people familiar with the matter.

Ant used domestic chips, including from affiliate Alibaba (9988) and Huawei Technologies, to train models using the so-called Mixture of Experts machine learning approach, the people said.

It got results similar to those from Nvidia chips like the H800, they said.

Hangzhou-based Ant is still using Nvidia for AI development but is now relying mostly on alternatives including from Advanced Micro Devices and Chinese chips for its latest models, they said.

It underscores how Chinese companies are trying to use local alternatives to the most advanced Nvidia semiconductors. While not the most advanced, the H800 is a relatively powerful processor and currently barred by the US from China.

The company published a research paper this month that claimed its models at times outperformed Meta Platforms in certain benchmarks.

If they work as advertised, Ant's platforms could mark another step forward for Chinese artificial intelligence development by slashing the cost of inferencing or supporting AI services.

Ant said it cost about 6.35 million yuan (HK$6.8 million) to train 1 trillion tokens using high-performance hardware, but its optimized approach would cut that down to 5.1 million yuan using lower-specification hardware.

Tokens are the units of information that a model ingests in order to learn about the world and deliver useful responses to user queries.

The company plans to leverage the recent breakthrough in the large language models it has developed, Ling-Plus and Ling-Lite, for industrial AI solutions including health care and finance, the people said.

Ant bought Chinese online platform Haodf.com this year to beef up its artificial intelligence services in health care. Ant created AI Doctor Assistant to support Haodf's 290,000 doctors with tasks such as medical record management, the company said in a separate statement yesterday.

This came as the Financial Times reported that Malaysia plans tighter controls over the flow of Nvidia's chips after the US demanded it keep a closer eye on advanced semiconductors that could potentially make their way to China.

Washington asked Malaysia to closely scrutinize the shipment of Nvidia chips coming to the country, Investment, Trade and Industry Minister Zafrul Abdul Aziz told the newspaper. "They want us to make sure that servers end up in the data centers that they're supposed to and not suddenly move to another ship."