What is LiveBench and why does it exist?

medium

Answer

$LiveBench (Abacus AI, 2024): monthly-refreshed benchmark with contamination-free questions across math, coding, reasoning, data analysis, and language.$
$Prevents benchmark contamination by rotating questions and drawing from recent sources (arXiv preprints, current news, live coding contests).$
$Complements Arena / MMLU by giving a fresh capability read after each model release.$
$Standard tool to check 'is this new frontier claim real, or a contamination artifact?'.$

Check yourself — multiple choice

Same as MMLU
Monthly-refreshed contamination-free benchmark drawing from recent sources → fresh capability read after each release
Only for chat
Fixed benchmark

LiveBench: rolling contamination-free benchmark to test real capability.

#evaluation#benchmarks

214 interview questions in this topic.