The purpose of this project was to develop a graphical user interface which would allow a user to give a
high level point and click command to control a robot. The action being controlled is the stacking of boxes by using
Camera Space Manipulation, an approach to robot control
using machine vision developed at Notre Dame with NSF funding under grant number 9700702. An explanation of the components of the system is given below or you can
try a simulation of the system to see how it works.
The CSM box stacking system is composed of:
- three CCD cameras mounted on pan/tilt units from the ceiling
- a single spot laser pointer and a multiple spot laser pointer mounted together on a single pan/tilt unit from the ceiling
- a Kawasaki Js-5 industrial robot with controller
- a box handling end effector (the first-ever AME476, Robotic Senior Design, project, Spring 2001)
- a PC with a frame grabber and TTL I/O board running MS Windows 98
The three cameras have a grayscale resolution of 640x480 and are mounted on pan/tilt units which allow for repositioning
to refine the best view of the workspace. A single framegrabber board handles images from all three cameras.
|
The lasers are
mounted together to ensure alignment and to reduce the need for another pan/tilt unit. Each laser is controlled by a single
TTL signal from the I/O board on the PC. The four pan/tilt units are networked together and are controlled through a single
serial port on the PC.
|
The laser pan/tilt unit is the only one which used within the stacking program. All camera
alignment is handled manually.
The Kawasaki Js-5 is a 6 degree of freedom, 6-axis industrial robot often used for material handling. The Js-5 controller,
like almost all comercial robot controllers, is designed for teach and repeat operation.
|
In order to use the robot for CSM, the controller's serial interface was taken advantage of. Usually used for
connecting to a terminal for programming, the serial
link allows for sending the controller joint position data as well as for prompting for the current robot configuration.
|
|
This accounts for the stops which the robot must make during the engagement procedure since the joint rotations cannot be
sampled on the fly with the controller's serial interface.
The box handling end effector was conceived, designed, and built by a group of five seniors enrolled in the first-ever session of AME476, Spring 2001. The new course, AME476, fulfills the senior design course requirement and serves as the fourth and final class in the robotics sequence of electives.
The end effector's dual purpose is to display a number of "cues" to be used by the positioning routine,
|
and then to secure a box once positioned by the robot.
The black circles in the image to the left are the visual cues which can be easily located within a camera image
by taking advantage of the properties of a projected elipse. More information on the visual cues can be found
here.
|
The position of the center of the cues in all three cameras is used to determine the position of the end effector relative
to the box being engaged.
Once the positioning routine is complete, a TTL signal from the I/O board is sent to gripper to engage the box.
|
The gripping mechanism is based on a single servo actuator which draws the two gripper faces together and locks in place.
When the positioning routine locates the end effector above the receiving box, the TTL signal is cut and the gripper
disengages the box.
|
|
To the Simulation
|
