• 0 Posts
  • 60 Comments
Joined 3 years ago
cake
Cake day: June 10th, 2023

help-circle



  • Ugh. Even if I wanted this product, their track record on long term product support has erased my trust. They never fixed connectivity for the Chromecast and now they’ve discontinued it. Why am I gonna buy something that breaks down or gets an expensive subscription in a couple of years.

    Plus, if I post a video their stakeholders don’t like on YouTube I risk getting locked out of every Google service including Gemini, all by automated processes and with no way to even talk to a human to explain my case. They are judge, jury and executioner within their ecosystem.












  • Let’s do some estimates:

    • An 8x H100 machine costs about $20 / hr to rent.
    • With a 70B model with 4K context, a H100 node can do about 300 requests in parallel.
    • A single response takes around 30 seconds to generate.
    • An average user sends about 300 messages / month.

    The throughput of a node is

    300 concurrent * (3600 / 30) = 36 000 messages / hour.

    The cost per message, then, is $20 / 36 000 = $.00055…

    With 300 messages per month, the compute cost for the AI vendor is 300*$20/36000 = $0.16 / month per user. By contrast, a subscription costs $20.

    So given these assumptions, it’s other things (like R&D, safety research, training runs, free accounts, etc) that represent the bulk of the cost and those could be scaled down to turn a profit. What will they do? Give how hyped AI is currently and the competitive landscape, I don’t think they’ll increase prices that much. We have products like DeepSeek on the horizon which are much cheaper, so it’s more likely that they squeeze money out of it by becoming more efficient.