EdgeBench Analyzes AI Agents, Leaderboards, and Scaling Laws

EdgeBench has been presented as a practical benchmark for evaluating advanced AI agents, offering analysis across diverse task categories, runtime environments, and interaction-time budgets. The framework allows for the examination of benchmark taxonomy, execution settings, internet requirements, judging logic, and scoring metadata. Users can download a dataset snapshot from Hugging Face to begin their analysis.

The benchmark's leaderboard data is extracted directly from the repository README. This data is then standardized for model names, and task-level results are reshaped into an analysis-ready format. Performance comparisons are made across multiple time budgets, enabling researchers to understand how different AI agents perform under varying constraints. The analysis includes fitting log-sigmoid scaling curves to identify performance trends.

EdgeBench measures category-level score improvements and inspects tasks that show the largest gains. It also studies how SForge rescale functions transform raw evaluation outputs into normalized benchmark scores. The tutorial outlines the process of installing required Python libraries such as huggingface_hub, pandas, numpy, and scipy, along with configuring display settings for analysis. Specific models analyzed include Claude Opus 4.8, GPT-5.5, GPT-5.4, GLM-5.1, and DS-V4-Pro, with time budgets ranging from 2 to 12 units.