Why are multimodal LLMs so expensive to run?
mediumAnswer
- Images tokenize into many tokens: a 1024×1024 image on GPT-4V ≈ 1500-2500 input tokens per image, on Claude 3 similar, on Gemini fewer.
- Each image is like a paragraph of text in cost.
- Complex documents (multi-page PDFs) can be 20-100k image-tokens.
- Also, vision encoders are compute-heavy (ViT-H).
- Optimizations: (1) resize images before uploading; (2) crop to region-of-interest; (3) use vision-language models with dynamic resolution (Qwen-VL, InternVL); (4) OCR text and send text only for text-heavy documents.
Check yourself — multiple choice
- Free to process images
- One image ≈ 1500-2500 tokens; optimize via resize, crop, dynamic resolution, or OCR-then-text for text-heavy docs
- Same as text cost
- Only inference
VLM cost: images = many tokens; resize/crop/dynamic-resolution/OCR-first optimizations.
#multimodal#cost#production
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- How much does it cost to pretrain a modern LLM?
- How should max_tokens be set in production?
- What is prompt caching and when does it help?
- How do you use one prompt to extract multiple fields at once vs many separate prompts?
- What can you cache in a RAG pipeline?
- How do you pick a GPU for serving a 70B LLM?