LiveBench - A benchmark testing platform for large language models (LLMs) designed to provide uncontaminated test data and objective scoring.

## Primary Purpose of LiveBench The primary purpose of LiveBench is to evaluate the capabilities of large language models (LLMs) across various tasks, providing researchers and developers with a fair and reliable assessment of model performance. ## Ensuring Uncontaminated Test Data in LiveBench LiveBench ensures uncontaminated test data by generating new questions monthly based on recent datasets, arXiv papers, news articles, and IMDb movie summaries. This prevents models from having been trained on the test data. ## Key Features of LiveBench The key features of LiveBench include: - Uncontaminated test data: Monthly updates based on recent resources. - Objective scoring: Each question has a verifiable objective answer for automated scoring. - Diverse and challenging tasks: Includes 18 tasks across 6 categories: mathematics, coding, reasoning, language, instruction following, and data analysis. ## Submitting Models for Evaluation on LiveBench Researchers can submit their models for evaluation on LiveBench by sending an email to [email protected] or opening an issue on the GitHub repository. The team will test the model and provide performance feedback. ## Accessing the LiveBench Dataset Users can access the LiveBench dataset on the Hugging Face platform, which is available for research purposes. ## Academic Recognition of LiveBench LiveBench has been recognized academically with a paper published on arXiv in June 2024 and is planned to be featured as a highlight paper at the ICLR 2025 conference. ## Task Categories in LiveBench LiveBench includes tasks in the following categories: - Mathematics: Based on math competitions and recent datasets. - Coding: Evaluates programming tasks like code generation and debugging. - Reasoning: Tests logical reasoning and complex problem-solving. - Language: Assesses language understanding and generation. - Instruction Following: Evaluates the ability to follow complex instructions. - Data Analysis: Involves tasks related to data processing and analysis. ## Significance of Objective Scoring in LiveBench The objective scoring mechanism in LiveBench ensures fairness and consistency by using verifiable objective answers for automated scoring, eliminating subjective biases that can arise from human or LLM-based evaluations. ### Citation sources: - [LiveBench](https://livebench.ai) - Official URL Updated: 2025-03-31