Unravelling the divide over AI's existential threat | Cash Call | Andrew Wong

Will AI ultimately pose an existential threat to humanity? The debate has continued to intensify over the past week. Microsoft AI chief executive Mustafa Suleyman has argued that the risks associated with AI development must now be taken seriously. Nvidia founder and chief executive Jensen Huang, however, has said that he believes the probability of AI wiping out humanity before 2030 is zero, arguing that people should not become excessively concerned about the pace of AI development.

Both are key figures in the global AI industry, but they represent very different parts of the AI ecosystem. Huang provides the hardware infrastructure that enables the vast majority of today's AI models to continue advancing at an unprecedented pace. Suleyman, by contrast, is focused on developing increasingly capable AI models, as well as turning those models into products that can be deployed and used in the real world.

It is therefore not surprising that Huang and Suleyman have different views on the risks associated with AI development. Nvidia's GPUs themselves are clearly not going to destroy humanity. The more fundamental question is what happens if those GPUs are used to accelerate AI development without adequate safeguards, while regulatory frameworks around the world fail to keep pace with the speed at which AI models are evolving. Under such circumstances, the probability of AI posing an existential threat to humanity by 2030 cannot simply be dismissed as zero.

In fact, one of the issues that has raised serious concerns among Suleyman and other AI executives is the ability of advanced AI systems to move beyond the constraints imposed by testing environments, including incidents involving access to or intrusion into AI technology platforms such as Hugging Face.

More importantly, such incidents do not necessarily appear to be isolated events. Executives directly responsible for developing and deploying these systems are likely to have a much closer understanding of the risks because they are dealing with them firsthand. Huang, whose primary role is to provide the underlying computing infrastructure, does not have the same degree of direct involvement in these model-level incidents, and may therefore naturally place less emphasis on these risks.

Of course, there is also an obvious commercial dimension to Huang's position. If the pace of AI development slows materially, Nvidia would be directly affected, as demand for its AI-related chips could decline sharply.

The problem, however, is that if OpenAI, Anthropic and xAI are increasingly concerned about incidents involving platforms such as Hugging Face and the broader risks associated with increasingly capable AI systems, a moderation in the pace of AI model development may become difficult to avoid.

Neither Huang nor even US President Donald Trump can force companies such as OpenAI, Anthropic and xAI to continue accelerating AI development at the same pace if these companies themselves decide that additional safeguards and testing are required. This also creates a potentially significant risk for Nvidia: demand for AI accelerators could slow materially over the next six months if AI developers begin prioritizing safety, testing and risk controls over the aggressive expansion of model capabilities. For investors, this is a risk that should not be overlooked.

Andrew Wong is a veteran independent commentator

𝗗𝗼𝘄𝗻𝗹𝗼𝗮𝗱 𝗧𝗵𝗲 𝗦𝘁𝗮𝗻𝗱𝗮𝗿𝗱 𝗔𝗽𝗽 ↓