Cactus Compute Releases Needle 2 Tool-Calling AI Model

Cactus Compute has released Needle 2, an open 45-million-parameter model designed for tool calling, device interaction, and structured data extraction. This model distinguishes itself by shipping as a single, compact 14MB binary file. It requires approximately 28MB of RAM to execute a full session, making it exceptionally suitable for resource-constrained environments. The model's weights are trained and deployed using Cactus Quants at CQ2-bit precision. It is integrated directly into Cactus Compute's proprietary C++ engine, eliminating the need for any separate runtime installation or downloads during inference. This integrated approach streamlines deployment and reduces overhead for developers. Reported decode throughput for Needle 2 demonstrates its efficiency across various hardware platforms. On a Raspberry Pi 5, it achieves 500 tokens per second. For more advanced consumer devices like the Meta Quest 3S and Apple Vision Pro, throughput ranges from 400 to 1,500 tokens per second. Even on sub-$200 mobile phones, the model can process between 300 and 700 tokens per second. The core design philosophy behind Needle 2, as stated by the Cactus Compute team, is to efficiently map unstructured text, such as a "messy sentence," directly to a predefined typed function signature. This process, they argue, does not necessitate extensive world knowledge or complex open-ended natural language generation. This focused approach is why a 45-million-parameter count is deemed sufficient for its intended tasks, and why the model is optimized for hardware lacking dedicated GPUs or Neural Processing Units (NPUs). Needle 2 is engineered for broad deployability. It is available as prebuilt binaries and a static library for a wide array of operating systems and architectures, including macOS, Linux (supporting x86-64, ARM64, ARMv7, RISC-V, and MIPS32el), Windows, Android, iOS, watchOS, tvOS, and WebAssembly. Cactus Compute has confirmed that the Pebble smartwatch is already utilizing Needle 2 locally within its Index 01 application to perform offline voice actions. The target audience for Needle 2 includes any development team building firmware or applications for hardware with limited computational resources. This encompasses seed-stage wearable and Internet of Things (IoT) startups, mid-market consumer electronics original equipment manufacturers (OEMs), robotics engineering teams, and large device manufacturers seeking a reliable offline fallback mechanism for their products. Cloud-first Software-as-a-Service (SaaS) teams are noted as benefiting less from this specific model due to its edge-centric design. Industries that can leverage Needle 2 include smart home devices, wearables, low-end mobile devices, automotive in-cabin control systems, service robotics, retail kiosks and point-of-sale (POS) terminals, routers and IP cameras, and highly regulated environments where data privacy mandates that audio processing must remain on the device. Potential applications are diverse, ranging from enabling voice-to-action capabilities on screenless devices and facilitating offline control of appliances to performing field extraction from receipts and invoices, tagging enumerated values, and implementing local routing logic that only escalates to the cloud when confidence levels are low. Architecturally, Needle 2 employs what the Cactus Compute team refers to as a Simple Attention Network. This architecture is characterized by its specific recipe: it replaces the standard Feed-Forward Network (FFN) with a Hadamard MLP, retains Grouped-Query Attention (GQA) for efficient attention mechanisms, and incorporates an "engram key-" component, though the description of this latter element is incomplete in the provided text.