Need to look at the hardware cycle as nVidia is still making more efficient hardware to bring the $/token down. BoA dude probably got burned on some loser AI companies that don’t have enough compute to deliver before the bills are due and is extrapolating it to the whole industry.
First, Nvidia isn’t on a trajectory to bring cost per token down, that’s not how the finances of a bubble like this works. nVidia is enjoying near monopolistic status, as your choice to ignore options from competitors in your comment illustrates. Even if they had credible evidence that Rubin could reduce the overhead of a hypothetical datacenter by 10 million, the business will probably price it at 12 million more (the “value” of the savings plus the value of being able to brag about offering the latest generation). The cost optimization will only come after the bubble pops.
Incidentally, a significant leap in efficiency could also pop the bubble. The bubble is predicated on all this eventually being very expensive and high margin. If it becomes more accessible without gigantic investment, well the walled garden business won’t stand up. An abundance of supply can pop it just as much as a failure of expected demand. We are talking about the economics of the bubble, and a hypothetical improvement in the merits of the tech does not necessarily map to economic results and in fact commonly is opposed to it.
Need to look at the hardware cycle as nVidia is still making more efficient hardware to bring the $/token down. BoA dude probably got burned on some loser AI companies that don’t have enough compute to deliver before the bills are due and is extrapolating it to the whole industry.
First, Nvidia isn’t on a trajectory to bring cost per token down, that’s not how the finances of a bubble like this works. nVidia is enjoying near monopolistic status, as your choice to ignore options from competitors in your comment illustrates. Even if they had credible evidence that Rubin could reduce the overhead of a hypothetical datacenter by 10 million, the business will probably price it at 12 million more (the “value” of the savings plus the value of being able to brag about offering the latest generation). The cost optimization will only come after the bubble pops.
Incidentally, a significant leap in efficiency could also pop the bubble. The bubble is predicated on all this eventually being very expensive and high margin. If it becomes more accessible without gigantic investment, well the walled garden business won’t stand up. An abundance of supply can pop it just as much as a failure of expected demand. We are talking about the economics of the bubble, and a hypothetical improvement in the merits of the tech does not necessarily map to economic results and in fact commonly is opposed to it.
I heard google is shifting to per-token subscription models that are much more expensive, and other AI are changing their pricing models too?