EasyDeepLearn

How do you estimate LLM API cost for a workload?

medium

Answer

  • Cost  =  inputtokensper\mathrm{Cost}\; = \;\mathrm{input}_{\mathrm{tokens}}\mathrm{per}_request × inputprice  +  outputtokensper\mathrm{input}_{\mathrm{price}}\; + \;\mathrm{output}_{\mathrm{tokens}}\mathrm{per}_request × outputprice\mathrm{output}_{\mathrm{price}} × QPS × seconds.
  • Input tokens are usually 3-5× cheaper than output.
  • Watch out for: (1) system prompt + few-shot bloating input tokens; (2) unbounded maxtokens\operatorname{max}_{\mathrm{tokens}} on user side; (3) failed requests still charged in some APIs; (4) function-call round trips (each call is a request).
  • Optimization levers: prompt caching (cache system prompt → discount on input), tighter maxtokens\operatorname{max}_{\mathrm{tokens}}, batching for embeddings.
Check yourself — multiple choice
  • Free service
  • Cost  =  inputtokens\mathrm{Cost}\; = \;\mathrm{input}_{\mathrm{tokens}} × inputprice  +  outputtokens\mathrm{input}_{\mathrm{price}}\; + \;\mathrm{output}_{\mathrm{tokens}} × outputprice\mathrm{output}_{\mathrm{price}} × QPS; input often 3-5× cheaper; watch out for system prompt  /  maxtokens\mathrm{prompt}\; / \;\operatorname{max}_{\mathrm{tokens}}
  • Only per request
  • Same as GPU cost

LLM cost: input + output tokens × price × QPS; optimize via caching / tight maxtokens\operatorname{max}_{\mathrm{tokens}}.

#cost#production

Practise LLMs & GenAI

214 interview questions in this topic.

Related questions