AI Agent Harness Architecture Dictates Performance Over Model Choice

The architecture of an AI agent's "harness"—the system that orchestrates the interaction between a large language model (LLM), tools, and the context window—is a more critical determinant of performance than the specific LLM used, according to insights from harness engineering literature and an open-source course by Paul Iusztin published through Decoding AI. An experiment conducted by LangChain's Terminal-Bench demonstrated that by solely modifying the harness while keeping the LLM constant, a coding agent's ranking improved dramatically, moving from approximately 30th place to the top 5. This finding reframes the primary decision in developing AI agents from selecting a model to architecting how the agent loop operates.

Iusztin's course, "Building a Coding Agent From Scratch," introduces a Python agent named Decode, which delineates three distinct "run modes" for the agent loop. Each mode is characterized by a unique latency profile, necessitating different inference providers. The core of the system is a "headless harness," meaning it lacks its own user interface. Within this harness, the agent loop executes its shared cycle: the LLM selects an action, a tool performs that action, and the resulting observation is fed back into the system, all interacting with the context window. The agent itself is designed to be minimal, with Decode's agent definition comprising approximately 20 lines of Pydantic code that compose the model, tools, and output type. For comparison, the leaked source code for Claude Code reveals a core loop of roughly 150 lines, with all other functionalities such as memory management, skill integration, sandboxing, permission handling, Language Server Protocol (LSP) feedback, and context compaction being part of the harness.

The three run modes emerge from how interfaces are plugged into this core harness. Mode 1, "Interactive, online," connects a terminal user interface to a single, live session running in memory within the same process. Events stream back as tokens arrive via asynchronous generators. A key challenge in this mode is "steering" user input, preventing corruption when a tool call is in progress. Decode addresses this through a steering queue and a priority gate, buffering input until a safe point in the loop. The loop exposes two critical points for interaction: MODEL_REQUEST, occurring before the next model call, and WOULD_STOP, signaling the potential end of a turn. Three input methods are mapped to these points: a standard 'Enter' key press steers within the current turn, while 'Alt+Enter' queues a follow-up instruction to be executed after the current turn concludes, and the 'Esc' key can be used for other functions, such as aborting an action or exiting a mode.