AffordSim Uses Language to Automate Robot Grasp Data, Transferring Weakly to Hardware

A recent preprint, AffordSim, proposes generating simulated robot manipulation training data by letting a task described in plain language determine where a gripper should contact an object. The system builds a matching scene, grounds the relevant contact region on the object's surface, and samples grips within that region that a motion generator can execute. Policies trained on this generated data reportedly reached 24 percent average success on a real robot arm without real-world tuning. The authors argue this cheaper automated approach can approximate hand-marked contact regions, which have historically limited how far simulated data can scale. The commentary notes that the approach still depends on human-labeled data and vocabulary, so human judgment moves earlier in the pipeline rather than disappearing.