Enhancement of 3D Stereo Vision System for Assembly Purposes

Abstract

The research effort expended upon the problem of performing industrial application using 3D machine vision, it focuses basically on the mechanical assembly as industrial application. As part of the mechanical assembly, a pick and place process has been developed to achieve the thesis objectives.

Performing industrial applications by normal labors or using CNC machines seems to be direct approach to achieve the required task, where these direct approaches cannot be used in some applications like performing processes in small places or dangerous environments to human. Robotics vision based systemshave been used for these difficult applications. 3D Vision is used in robotics as an advanced technique to perform these industrial tasks. Gestures recognition and human Skeleton tracking are the most focused topics in 3D vision in the new researches.

There are problems and challenges in the 3D vision based systems capable of performing industrial tasks. The problems and challenges rise from the non-ideal behavior of sensors and actuators, also the complexity of linking and controlling the different parts of the system.

The vision system acts as a visual sensor for the robot used in mapping the environment and localization of objects. 3D vision system seems to be a great solution applied in industrial robotics. Microsoft Kinect is one of the best 3D vision sensors been used recently. It is used as a perfect interface between humans and computers.

With the release of Microsoft Kinect in 2011, several researches were concerned developing systems equipped with 3D camera sensor to perform some industrial processes. However most of them are targeting depth map for navigation and localization of objects in surrounding environment or gesture recognition for a user in front of the 3D camera sensor to control a robot or machine. Meanwhile using the 3D joint coordinates from human skeleton tracking to perform 2D industrial task based on position control is a new trend in vision track.

The proposed thesis offers a novel approach in acquiring the 3D joint coordinates from the human skeleton tracking by Microsoft Kinect, calibrating and filtering these data, analyzing and calculating the required distance and angle for the robot to perform a pick and place task.

Microsoft Kinect acts as the 3D camera sensor for the proposed system and an input source for the controller before sending motion command to the actuators. Acquiring the 3D Joint coordinates from the human skeleton tracking is discussed in details with calibrating the data and filtering the different noises from this data.

A finite state machine algorithm has been developed acting as the controller for the system. It determines the required state (process)to be done by the robot based on the input data from the Kinect. Another algorithm is discussed which calculates the required distance and angle for the robot.

The robot used in the system is assembled based on Lego Mindstorms EV3. It basically contains two front wheels with two servomotors, two back wheels and a front gripper.

A practical experiment has been developed by a user performing a pick and place task in an environment and a robot emulate this task in another equivalent environment with the same positions. The evaluation of the system showed promising results in terms of accuracy of the robot reaching the required object and placing it in the required position.

Another simulation experiment has been developed to approve the proposed technique in the thesis.