Vision Assisted Control for Manipulation Using Virtual Fixtures

Allison M. Okamura

The Johns Hopkins University
Department of Mechanical Engineering
Engineering Research Center for Computer Integrated Surgical Systems and Technology

 The “steady hand” concept is a way of providing assistance for direct manipulation by applying constraints on the motion of a tool shared by a user and a robot. In this presentation, we explore in detail one family of constraints: virtual fixtures for use in path following tasks. Vision is used to sense the desired path, and then the robot encourages motion toward and along the path through a direction-based control law. A “soft” virtual fixture allows the user to move in other, non-preferred directions, maintaining the user's sense of autonomy and control. Experimental results show that user performance in assisted path following is enhanced with virtual fixture augmentation, and differs with varying fixture compliance. We demonstrate improvements in path following and positioning on structures at both macro and micro scales. In addition, we present preliminary results on recognizing human intent during task execution, using Hidden Markov Models to segment force and position data. This work has applications in robotic assistance for procedures such as microsurgical retinal vein cannulation and suturing during robot-assisted minimally invasive surgery.