Kan AI styre en robotarm gennem en madopskrift i et kontrolleret køkken ?

DeepMind's RT-2 og efterfølgere viste, at end-to-end vision-sprog-handlingsmodeller kunne følge en opskrift med fejl hovedsageligt inden for menneskelig rækkevidde.

Background

DeepMind's RT-2 and its successors demonstrated that end-to-end vision-language-action models are capable of executing multi-step cooking instructions with error rates approaching human performance in controlled environments. AI-powered robotic arms have been successfully deployed to follow structured recipes in controlled kitchens, utilizing integrated sensors and machine learning systems to adapt to ingredient variations and task nuances. Research prototypes and commercial deployments alike leverage pre-programmed high-level recipes mapped to low-level motor actions, often constrained by lighting, spatial layout, and standardized ingredient presentation to ensure repeatable outcomes. Studies published by IEEE highlight that such systems reliably operate in commercial or assistive settings, where consistency and repeatability outweigh the need for full culinary creativity. These platforms typically combine real-time visual feedback, force sensing, and semantic reasoning to map verbal or written recipes (e.g., "chop onion," "whisk egg") into executable arm trajectories. While current implementations dominate structured environments—such as prep stations in food manufacturing or assistive cooking platforms for individuals with motor impairments—they remain sensitive to deviations in ingredient shape, color, or placement. This underscores ongoing work in robust perception and adaptive control to generalize recipe execution beyond idealized conditions.