{"id":50832,"date":"2026-05-20T18:31:22","date_gmt":"2026-05-20T18:31:22","guid":{"rendered":"https:\/\/agooka.com\/news\/business\/i-gave-my-openclaw-agent-a-physical-body\/"},"modified":"2026-05-20T18:31:22","modified_gmt":"2026-05-20T18:31:22","slug":"i-gave-my-openclaw-agent-a-physical-body","status":"publish","type":"post","link":"https:\/\/agooka.com\/news\/business\/i-gave-my-openclaw-agent-a-physical-body\/","title":{"rendered":"I Gave My OpenClaw Agent a Physical Body"},"content":{"rendered":"

Save StorySave this storySave StorySave this story<\/p>\n

I recently gave my OpenClaw a real robot arm to play with. The results just about blew my own neural network.<\/p>\n

The AI agent was able to configure the arm, use it to see and slowly grab things, and even train another AI model to pick up and place specific objects. And they say AGI is still a few years away! (I\u2019m joking, it probably is).<\/p>\n

The results have me convinced that we may be on the brink of a robotics breakthrough. Training and controlling robots used to require considerable skill. Today\u2019s AI models can make it almost easy.<\/p>\n

\u201cAI-powered coding is super exciting because it has the potential to bridge the gap between conventional engineering methods, which are reliable but don't generalize, and contemporary vision-language-action models, which generalize but are not yet reliable,\u201d says Ken Goldberg, a roboticist at UC Berkeley who is exploring the approach.<\/p>\n

\"I<\/p>\n

I told OpenClaw to try moving its new arm and it came up with this little wave.<\/p>\n<\/figure>\n

I bought a prebuilt arm called a LeRobot 101. It\u2019s part of an open-source project from HuggingFace that makes it relatively cheap to start building and experimenting with robotics.<\/p>\n

The LeRobot comes with two arms: a controller arm that a person operates using a handle and a trigger, and a follower arm with a camera that replicates those movements. You can train an AI model by teleoperating the controller arm and having the model learn how to move the follower in response to what it sees on the camera.<\/p>\n

Building With OpenClaw<\/h2>\n

Before using OpenClaw, I spent several hours trying to connect and calibrate the robot, at one point nearly breaking the motors by applying the wrong settings, which caused them to overheat.<\/p>\n

Then, with help from OpenClaw and Codex, I was able to vibe code a simple program that closed the claw\u2019s gripper when it spotted a red ball. In the terminal, Codex went through the tricky work of configuring the connections to the robot. Then, with my help, it calibrated the positions of its joints. It also wrote a Python script that used several libraries to identify and grip the ball in question. Vibe-coding isn't perfect of course, and hallucinations can introduce bugs especially when working with different hardware, but the results were impressive.<\/p>\n

\"Then<\/p>\n

Then, with my help, the robot-agent figured out how to identify and grip a red ball.<\/p>\n<\/figure>\n

A neat result, yes, but not exactly Terminator. Next I tried having OpenClaw help me train a model to control the arm. We experimented with a few different approaches, and OpenClaw was adept at guiding me through the process and checking the error rate of the model after each training run.<\/p>\n

\"Finally<\/p>\n

Finally, the robot arm was able to pick up objects.<\/p>\n<\/figure>\n

Code as Policy<\/h2>\n

The idea that AI-powered coding could offer a powerful new way to build robots was first highlighted in a research paper from 2022 that dubbed the approach \u201ccode as policy.\u201d Since then, AI\u2019s coding skills have advanced at a dizzying pace, and the code-as-policy method has gained traction in many labs.<\/p>\n

Goldberg\u2019s research group, together with researchers from Nvidia, Carnegie Mellon University, and Stanford, recently developed a new benchmark called CaP-X to measure the robot capabilities of coding models. Interestingly, CaP-X shows that the best model for programming robots isn\u2019t Claude or ChatGPT but Gemini\u2014perhaps because Google DeepMind has focused on training its models to be multimodal and make sense of the physical world. Along with the benchmark, the researchers created CaP-Gym, an environment that lets coding agents control both simulated and real robots. They also developed CaP-Agent0, an agentic framework that boosts the performance of coding models so much that they beat models trained to control a robot\u2019s movements directly on some manipulation tasks.<\/p>\n

Goldberg\u2019s team is working with Nvidia to explore the potential of the code-as-policy approach. I spoke to Spencer Huang (none other than Jensen Huang\u2019s son), who has been involved in organizing hackathons inside the company to let people try their hand at vibe coding robots. Huang is currently working on a research project with Goldberg that should make the code-as-policy approach compatible with more robot software tools.<\/p>\n

\u201cNearly anyone can get into robotics, which is the true holy grail,\u201d Huang tells me. Making it possible for people to control robots with spoken or typed commands, or by demonstrating an action, is the \u201ccritical unlock for robots in society,\u201d he adds.<\/p>\n","protected":false},"excerpt":{"rendered":"

Save StorySave this storySave StorySave this story I recently gave my OpenClaw a real robot arm to play with. The results just about blew my own neural network. The AI agent was able to configure the arm, use it to see and slowly grab things, and even train another AI model to pick up and […]<\/p>\n","protected":false},"author":1,"featured_media":50833,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[36],"tags":[],"class_list":{"0":"post-50832","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-business"},"_links":{"self":[{"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/posts\/50832","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/comments?post=50832"}],"version-history":[{"count":0,"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/posts\/50832\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/media\/50833"}],"wp:attachment":[{"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/media?parent=50832"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/categories?post=50832"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/agooka.com\/news\/wp-json\/wp\/v2\/tags?post=50832"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}