The Webcam will locate an object of the target color. It will then use the servos to look directly at the target, meaning the target will be in the center of the camera's point of view.

This is based on how a person would identify an object of interest on a table. They would notice a color out of their peripheral vision and move their eyes to look directly at it.

Knowing the height of the camera off the table and the fact that it is looking directly at the target, the location of the target can be calculated. The tilt of the camera is used to determine the distance from the base of the camera to the target. The pan of the camera is then used with a rotation formula to determine where the target is on the table.

While this math may not seem to map well to human behavior in a way it does. We do not use trigonometry to calculate where to put our feet when walking or grabbing objects however we can estimate distane well due to our depth perception. Since the webcam used in this project had no easy way of dealing with depth perception the trigonometric calculations were an acceptible replacement.

The arm will be moved to align vertically with the target. By vertical I mean in the y-axis in the coordinate system used by the program. If the camera is looking straght ahead it is looking along the x axis. The arm aligns with the target in this manner to make the next steps simpler.

This step is not well mapped to human behavior but is instead a circumstance of the libraries used in developing this system. A human would not need to first align their arm before moving towards the target, they would simply move towards the target. This is explained in further detail in the Future Work section.

The arm will then move directly towards the target's location. As it moves the gripper will be rotated to continuously point at the target. This avoids any errors which may occur as the arm moves.

This step is designed to resemble hand-eye coordination feedback in humans. As we try to grab an object or catch something in the air we will identify where the object is and move our hand in that direction. We may not know exactly where the object is but so long as our hand stays in the correct orientation we should be able to grasp the target. This technique is modeled in this program through the use of a feedback loop which makes sure the gripper is pointing at the target while the arm moves towards the target.

The gripper has been partially colored so the camera can recognize it. Both halves of the gripper have been colored blue. Half of a plastic blue egg has been placed in the center of the wrist portion of the gripper. The camera will look at the gripper and use the color coding to determine which portion of the viewing area is contained in the gripper. It then checks to see if the center of the target is within this region. If it is then it moves onto the Move Target step, otherwise it returns to the Align with Target step to try grabbing it again.

This step closely resembles how a human might check to see if they had grabbed a target. Naturally a human would be able to feel the target but ignoring that sense for a moment a person would have to look and see if the target is within the bounds of their fingers. If not then they would probably back off slightly and try again. This checking algorithm applied well to Tekkotsu as described above.

The arm will move the target to a new location on the table, release the target and move the arm away from the target. This completes the algorithm.

This simply models a human moving an object they have grabbed and letting go of it. No feedback loop is required as a person can assume they have a good grasp of the target. This may not be entirely true in all cases but will suffice for this program.