Vision servoing in unstructured environments

Vision is one of the most important sensing abilities when it comes to human hand manipulation. The coarse arm position comes from the nerve system controlling muscles while the contact sense is achieved through natural tactile nerves in the human skin. The relation with respect to other objects is accomplished through vision. When it comes to robots, a lot of work has been done in the area of position sensing and force feedback. It usually comes in a form of an encoder or resolver. The sense of touch/force is partially accommodated through use of force/torque sensors and/or whiskers. However, in many teleoperation systems the visual feedback is provided and analyzed by human operators not by the machine. Certain tasks could be performed with the use of automatic or assistive machine vision. In order to do that the machine vision must be an additional system closing the position loop in the control system. This paper will address how current teleoperation systems could benefit from the use of machine vision trying to perform the tasks in an environment that resembles a real environment. Many publications report progress in the field of vision servoing but very few attempts to apply the results in an environment that is closely related to remote teleoperation. The paper will also attempt to analyze the use of multi camera system depending on the task, object and available hardware. It will describe the necessary experiments to test the validity of the presented analysis and conclusions.