Summary
Sam Witteveen reviews Cactus Needle, an open-source 26-million-parameter function calling model from YC-backed startup Cactus, notable for being orders of magnitude smaller than comparable models while matching or beating them on single-shot function calling benchmarks. Needle was distilled from Gemini 3.1 Flash Lite and uses an all-attention architecture with no feedforward or MLP layers anywhere in the network — just attention and gating — which eliminates the dense knowledge-storage layers that dominate standard transformer parameter counts.
The model was pre-trained on 200 billion tokens using 16 TPUs over 27 hours, then post-trained on 2 billion tokens of synthetic function calling data spanning 15 tool categories. On single-shot benchmarks it outperforms Function Gemma (10x larger), IBM Granite, LFM, and Qwen 600M (20x larger). At inference time it reaches up to 6,000 tokens per second prefill and 1,200 tokens per second on consumer devices, and can be fine-tuned on CPU alone. Witteveen demonstrates it live with set-timer, send-email, create-note, and a 20-function smart-home tool set, noting accurate argument extraction even from free-form natural language.
Witteveen is candid about limitations: the model is not suited for multi-step reasoning, parallel tool calls, or skill trees requiring progressive disclosure. He proposes cascade architectures as a practical workaround — using Needle for tool routing while larger models handle complex reasoning. The HuggingFace repo and GitHub fine-tuning code are publicly available.
📺 Source: Sam Witteveen · Published July 12, 2026
🏷️ Format: Review
