Responsive Teleoperation

At NVIDIA GEAR, I made a new teleop stack with improved latency and accuracy. Hardware included a redesigned finger tracker glove and Rob Knight’s V2.3 Dexhand.

The better kinematics and motors of the hand, calibration per user, and improved retargeting made in-hand manipulation possible. I was able to reorient an apple, rotate a side of a Rubik’s cube, and do a charge pen spin.

I have recordings of me pen spinning down below. It’s a good benchmark for reactive performance of a hand, as the pen is essentially falling through my fingers, with just the right impulses at the right times to keep the motion going. If you can have a robot pen spin, you can likely have it do any dexterous task, but no hand can do it at human level yet…