Baxter Inserts USB Cables With GelSight Sensing

From sdk-wiki
Jump to: navigation, search


Robust manipulation and insertion of small parts can be challenging because of the small tolerances typically involved. This project demonstrates the USB insertion task by Baxter with the use of GelSight tactile sensor that has much higher resolution than competing methods. Using a feature-based matching technique on the high-resolution tactile maps generated, we are able to localize small parts held in a robot hand very accurately and achieve high success rate for the insertion task.

USB Insertion and Tactile Map
(a) The Baxter robot inserts a USB connector into a mating hole. The robot relies on tactile sensing in order to localize the connector in the gripper and therefore to calculate how to move the hand. (b) Illustration of the tactile map that describes what the connector “feels like” to the robot. The brightness corresponds to the protrusion of the object in contact. The tactile map is used to localize the connector relative to the gripper.



GelSight is a recently developed tactile sensing technology that can measure the geometry of a contacted surface at a resolution as fine as a few microns. The GelSight principle of operation is as follows. A piece of clear elastomer gel is coated with a reflective membrane. When an object is pressed against the membrane, it deforms to take the shape of the object’s surface. The membrane is illuminated by LEDs that project from different directions so that multiple images of the same membrane deformation are captured. A 3D height map of the surface can then be reconstructed using photometric stereo algorithms.

GelSight Principle
Illustration of GelSight operating concept. (a) An elastomer gel is coated with a reflective membrane; (b) the membrane deforms when the gel makes contact with an object; (c) with proper illumination, it is possible to calculate a 3D height map of the object surface using photometric stereo methods.

Fingertip GelSight Sensor

We build a fingertip GelSight device for use on Baxter, as illustrated below.

GelSight Fingertip
GelSight sensor design suitable for mounting on a robot fingertip. (a) A fingertip GelSight sensor, and (b) its schematic diagram (side view).
GelSight on Baxter
Integration of the GelSight sensor on the Baxter robot hand. (a) the CAD model. (b) One finger is equipped with a fingertip GelSight sensor.

Tactile Map Registration and Mapping

To be continued...