Pick and Place Robotic Arm

OpenManipulatorX, USB Fisheye camera and MATLAB (CV Toolbox)

Goals

Project goal was to use an outside-in USB Camera, OpenManipulator X, and the MATLAB Computer Vision toolbox to autonomously sort balls of four different colors into respective bins.

Code Architecture

The arm's functionality is split into a robot class for planning and control, and an image processor class for object detection and localization.

A snapshot is first taken of balls on a checkerboard plate. The image processor processes the image and extracts the ball color and centroid coordinates with color masking and blob detection (more info below.)

The robot then plans a multi-step trajectory to quickly move above the ball, open the end effector, carefully decend and grasp the ball, then move above correct bin and drop it safely.

Computer Vision

Overcoming lens distortion

Masking and detecting centroids

Control

DH Forward Kinematics

Geometric Inverse Kinematics

Jacobian Inverse Kinematics

Experimental Netwon-Raphson Method

Trajectory Generation

Quintic Trajectory Generator

Code

Unfortunately because of tight integration with WPI's RBE3001 course material (arm drivers and solutions to course labs) I have been told not to post code on Github.

I can share on a request basis thought.