Read More
As the global artificial intelligence race reaches a fever pitch, AI safety has thrown tech circles, policymakers, and capital markets into intense debate following a rare call by frontier startup Anthropic for a worldwide freeze on advanced development.
ADVERTISEMENT
SCROLL TO CONTINUE WITH CONTENT
The high-profile appeal highlights a deepening paradox within the generative AI industry. Anthropic warns that systems are rapidly approaching a critical tipping point – “recursive self-improvement” – where AI models independently rewrite and optimize their own code without human intervention. This shift could trigger exponential, uncontrollable technological growth that outpaces human regulation and comprehension.
This risk is no longer theoretical. Anthropic disclosed that its internal systems now generate eight times as much code as they did a few years ago. No longer just simple tools, these models now act as “research collaborators” capable of proposing their own technical blueprints.
However, commercial pressures complicate the message. Anthropic’s call for restraint arrives just as it prepares for an initial public offering targeting a staggering US$1 trillion (HK$7.8 trillion) valuation, laying bare the profound tension between safety and business metrics. Furthermore, the company quietly withdrew an earlier pledge to halt development if capabilities outstripped safety thresholds, prompting critics to question whether established players are leveraging safety concerns to build regulatory barriers against underfunded competitors.
Enforcing a global pause faces a geopolitical “prisoner’s dilemma.” Computing activities are notoriously difficult to monitor compared with traditional arms, and mutual trust remains scarce. Current frameworks, such as the EU AI Act, fail to address catastrophic frontier risks, so that any meaningful solution may require international, non-proliferation-style verification mechanisms – a tall order in today’s fractured geopolitical landscape.
Francis Fong is a Hong Kong IT and Telecom expert who frequently represents the industry in public discussions about innovation, digital transformation, and technology policies
