How much does AI cost? It depends on a a few different dimensions, but for a typical use case where a user enters a query & expects about a 200 word response, the cost varies from $0.03 to $3.60.
Different models cost different amounts. Also, the size of the context window is an important factor.
The context window is a fancy way of saying the quantity of background information the user can provide to the AI.
When asking an AI to write a two-paragraph essay in the style of Hemingway, having a few sentences of his writing won’t help much. But perhaps uploading one-third of the Old Man & the Sea imbues Ernest’s style within the prose – no adverbs, simple direct sentences.
Longer tracts cost more. A 500 word essay totals $8.40 with the fanciest version of GPT-4 & $0.07 with Llama2, Facebook’s open-source model.
There is a 120x difference in cost between the top of the line AI & the free version.
Which is better? It depends on the use case & the amount of optimization one is willing to commit to the model.
These prices aren’t fixed. Earlier this week, OpenAI announced a 3x reduction in costs – a trend that will continue as engineers optimize the chips, the algorithms, & the systems for AI.
Relative to other infrastructure costs, these fees can be significant & understanding the gross-margin impact of this new layer of software will be essential, & will likely push startups from the more expensive larger models to more capital-efficient AI over time.
