Artificial Analysis

The Artificial Analysis Intelligence Index is our synthesis metric for assessing model intelligence and tracking AI progress. Today we announced v4.1 with changes listed below. Links: Artificial Analysis Intelligence Index - https://artificialanalysis.ai/ Artificial Analysis Intelligence Index Methodology - https://artificialanalysis.ai/methodology/intelligence-benchmarking Intelligence Index v4.1 brings the following updates: 1. We upgraded three evaluations, removed one, and reweighted the Intelligence Index: ➤ Upgraded Terminal-Bench Hard to Terminal-Bench 2.1 and τ²-Bench Telecom to τ³-Bench Banking. Both move to newer, more robust task sets with harder, more realistic agentic scenarios that better separate frontier models ➤ Upgraded GDPval-AA to GDPval-AA v2. The upgrade re-baselines Elo to human performance at 1000, introduces a rotating panel of frontier-model judges, and raises the turn limit from 100 to 250 for longer-horizon agent trajectories ➤ Removed IFBench due to saturation. The benchmark no longer distinguishes frontier models sufficiently, so we have removed it from the Intelligence Index. We will continue to run it and publish results on new model releases 2. Cost per Task, Time per Task, and Tokens per Task: Three new per-task metrics, reported for every model and based on the Intelligence Index. We take the total cost, total time, and total output tokens for a model to run the Intelligence Index and divide by the number of tasks across its evaluations, giving the average cost, time, and output tokens to complete a single Intelligence Index task 3. Cached input token reporting: We now report cached input tokens and their impact on cost, including the cost to run the Intelligence Index, to better reflect the real cost of running each model Timestamps: 0:00 - Intro 0:26 - Evaluation and Benchmark Changes 1:00 - Cost per Task, Time per Task, Tokens per Task additional metrics 1:39 - Cache pricing breakdown 1:54 - Leading overall models 2:05 - Leading open-source models 2:19 - Methodology and outro

GDPval-AA v2 is Artificial Analysis' evaluation framework for OpenAI's GDPval dataset. It assesses language models' capabilities on 220 economically valuable tasks, covering 44 occupations across 9 sectors contributing to GDP in the United States. **Update:** We've just released GDPval-AA-v2 with upgraded evaluations including a panel of judges, the main methodologies in this video still hold but they've been improved with v2. All links below lead to the updated results. Links: • GDPval-AA v2 leaderboard - https://artificialanalysis.ai/evaluations/gdpval-aa • GDPval-AA v2 methodology - https://artificialanalysis.ai/methodology/intelligence-benchmarking#gdpval-aa • GDPval Public Dataset - https://huggingface.co/datasets/openai/gdpval • GDPval blog post by OpenAI - https://openai.com/index/gdpval/ • GDPval paper by OpenAI - https://arxiv.org/abs/2510.04374 Timestamps: 0:00 - Intro 0:04 - Task Breakdown 0:24 - GDPval-AA mentions in the wild 0:31 - GDPval-AA Leaderboard (as of June 12 2026) 0:53 - GDPval-AA Example Tasks 1:18 - GDPval-AA Implementation 2:06 - Comparing Elo Scores 2:40 - Cost and Token Usage 3:22 - Extra Resources