OpenAI, Anthropic Add Skill Recording to AI Assistants

This summer, both OpenAI and Anthropic launched new features that enable their AI systems to learn tasks by watching a user perform them, a significant evolution from relying solely on textual prompts. OpenAI introduced Record & Replay in June, a capability integrated into ChatGPT and Codex that allows users to demonstrate a workflow, which the AI then converts into a reusable skill. Shortly after, Anthropic unveiled Record a Skill within its Claude Cowork platform, enabling users to record their screen while performing a task and narrate their reasoning, allowing Claude to generate a skill it can execute independently. The simultaneous emergence of these similar functionalities from two leading AI companies suggests an acknowledgment that prompt-based instruction alone is insufficient for achieving advanced AI capabilities.

This development echoes challenges faced in earlier AI advancements, such as the creation of voice assistants. The author, drawing from experience building the Chinese version of Siri at Apple over 12 years, highlights the transition from natural language interaction to the more complex problem of context awareness. Early voice assistants could process initial requests but struggled to retain user preferences or habits across multiple interactions. A common ambiguity, like setting an alarm for "6," illustrates this gap: users assume their personal schedules are understood, but the AI lacks this implicit context. The disconnect between spoken intent and actual meaning is the fundamental issue that OpenAI and Anthropic are now addressing with their new skill-recording features.

The "show, don't tell" approach recognizes that human workflows are highly individualized and often rely on unspoken context, or "tacit knowledge." Even individuals in identical roles may utilize different tools, follow distinct sequences of actions, and possess a wealth of unarticulated understanding about how to perform their tasks. Traditional AI models, often trained on generalized data, struggle to capture this nuanced, person-specific operational knowledge. By allowing AI systems to observe and record these unique demonstrations, companies are attempting to bridge the gap between explicit instructions and the implicit, often idiosyncratic, methods that define effective task execution. This move signifies a deeper integration of AI into practical workflows, moving beyond simple command execution to a more adaptive and personalized form of assistance.

These new features represent a strategic pivot for AI development, acknowledging the limitations of purely linguistic or rule-based training. The ability for AI to learn by observation, akin to human apprenticeships, allows for the capture of complex, multi-step processes that are difficult to articulate comprehensively through text prompts. This approach is particularly valuable for tasks involving specific software interfaces, intricate sequences of operations, or domain-specific tacit knowledge. By enabling AI to internalize these demonstrated skills, developers aim to create more capable and intuitive AI assistants that can better adapt to individual user needs and operational environments, moving closer to a truly context-aware and personalized AI experience.