Tech giant Google has begun capping access to its Gemini artificial intelligence models, the clearest signal yet that the global compute squeeze is killing off the βtokenmaxxingβ era and forcing the cost discipline Australian boardrooms have so far failed to impose on their own AI rollouts.
Tokens are the small chunks of work β tiny, standardised units of data β that AI models process for every task, leading to a fad in the tech world last year of βtokenmaxxingβ as companies that treated token use as a proxy for productivity pushed staff to consume as much as possible.
But companies such as Uber and Meta have now stepped away from the practice as the cost of token usage has soared to astronomical levels, and capacity constraints have forced companies such as Google to curb sales.
The Financial Times reported this week that Google told Meta around March it could not provide all the Gemini capacity the social media giant wanted to buy, with the cap still in place and other Google customers also affected. Meta is one of the worldβs largest enterprise AI customers; that even it cannot get all the compute it wants signals how severe the global shortage has become.
New Australian research from search firm Elastic shows the squeeze is already changing how local businesses think about AI spend. One in three Australian organisations exceeded their AI budget last financial year and 32 per cent have paused, cancelled or wound back deployments because the cost could not be justified.
Nvidia chief executive Jensen Huang said in March he would be βdeeply alarmedβ if a $US500,000 ($724,000) developer spent less than $US250,000 on tokens. Meta engineers reportedly consumed more than 60 trillion tokens in 30 days, an outlay one estimate put at roughly $US900 million, and Uberβs chief technology officer said in April the ride-share company had burned through its full-year AI budget in four months.
Since then, Amazon and Meta have deleted internal leaderboards for token use. βPlease donβt use AI just for the sake of using AI,β senior vice president David Treadwell told staff in May. That month, Uberβs chief operating officer told a podcast that the βlink is not there yetβ between token use and genuine productivity.
The hardware to sustain that pace does not exist. Memory chipmakers SK Hynix, Samsung and Micron have sold out most of their supply of the high-bandwidth memory AI models depend on, while rental prices for Nvidiaβs older H100 graphics processing units are up about 30 per cent since November.
To survive the squeeze, engineering teams are increasingly abandoning massive foundational models in favour of specialised Small Language Models and alternatives that can be hosted locally at a fraction of the cost.
The scramble to build physical infrastructure has triggered an unprecedented land grab. Australian-founded Firmus, planning an ASX listing valuing it at up to $12 billion, said on Monday it had signed a deal with Nvidia to build AI data centre capacity in Indonesia that it forecasts will generate $US25 billion to $US30 billion in revenue over its first six years.
David Alonso, the national AI market lead at Deloitte Australia, said model providers had shifted from licence and subscription pricing to pricing based on consumption, removing the implicit subsidy that had made AI feel cheap. βItβs the end of the era of AI subsidy,β Alonso told this masthead. βTokenmaxxing β¦ becomes now more of a problem in itself.β
Elastic country manager Jeremy Pell said the squeeze would force companies to control their AI costs. βBecause demand is outstripping the physical infrastructure the basic laws of economics will take over, and token costs are inevitably going to rise,β he said.
Alonso said the shift did not mean Australian companies would spend less on AI overall, however. βIf anything, they will β¦ keep growing and β¦ your cost line is highly likely to still go up,β he said. βBut itβs now that clear need to link this to value.β
The Elastic survey, conducted by Pure Profile and commissioned by Elastic, suggests local businesses are not yet measuring whether their AI spend delivers. Only 8 per cent of decision-makers track AIβs contribution to revenue or cost savings. Yet half of them plan to increase AI spend over the next 12 months, with 32 per cent saying they will only do so with clearer proof of value.
βOver the next 12 months we predict a massive shift from AI usage to strict AI accountability,β Pell said. βThe era of evaluating success by how busy your usage dashboards look is officially coming to a close.β
Alonso said Australia had a βwindow of opportunityβ of about two years to attract investment in domestic data centre capacity that would give Australian businesses local compute and more control over their token costs.
The Business Briefing newsletter delivers major stories, exclusive coverage and expert opinion. Sign up to get it every weekday morning.
David Swan is the technology editor for The Age and The Sydney Morning Herald. He was previously technology editor for The Australian newspaper.Connect via X or email.