The cost of AI compute is set to explode: implications and opportunities

The thesis of exploding AI compute costs

Dwarkesh Patel predicts a potential increase of more than 10x in the cost of compute in the coming years. This rise is driven by the increasing ability of AI models to monetize compute, making investment in GPUs increasingly profitable. If a human-level software engineer could run on an H100, the rental value of that H100 would exceed $250,000 per year, or 15x the current spot price.

Market dynamics and consequences

AI labs like Anthropic see their revenues multiply tenfold while their compute capacity only triples each year. To sustain this growth, either margins increase drastically, or the price of compute skyrockets, or a growing share of compute is dedicated to inference. Labs prefer to avoid this last option, as it would mean a slowdown in training progress. The author believes that growth in margins alone is insufficient, leaving the increase in compute cost as the dominant factor. This situation will favor the most performant and efficient AI models, as the high cost of compute will make the use of less optimized models prohibitive. This could also exclude certain current AI applications deemed less profitable.