There is a behavior built into Tekkotsu which allows a webcam to track a target. This code was adapted to this program however some limitations were encountered. Firstly there is a lot of "noise" in the room. That is, there are several large objects such as couches which have colors recognized by the webcam which can confuse it. This noise was acounted for by defining colors for objects which confused the camera so it could differentiate between noise and actual targets.

Another challenge encountered when trying to aquire the target is the limitations with the webcamera. Even after programming the camera to recognize the correct colors it had trouble detecting the correct color of small objects. This was accounted for by grouping together several blocks of the same color so the camera could target them together.

The camera code which allowed the program to focus on the target is based on a behavior built into Tekkotsu (see above). This portion of the code was rapidly developed and made functional early in the project. However using this functionality to calculate where the target is located in space took a few tries.

It was accomplished though the use of the tilt and pan angles of the camera servos. The height from the table to the camera was measured and used with the tilt angle to determine the distance from the base of the camera directly to the target. This distance was then used with a rotation formula along with the pan angle to find the horizontal and vertical coordinates of the target on the table. This formula went through several iterations and calibration steps until finally working correctly.

There is a built-in function in Tekkotsu called moveToPoint which was called once the gripper was aligned to move towards the target automatically. However this function moves the arm in very obscure ways. It seems to extend the arm to the correct distance and rotate it to the target point. This led to the gripper slamming into the target from the side rather then grabbing it.

This challenge was overcome by writing a function which is called continuously while the arm is moving towards the target. This function calculates the current heading of the gripper and where it should point to be pointing straight at the target. This new rotation is then applied to the wrist so that the gripper points at the target as it moves towards the target in its obscure built-in path.

When the arm is done moving to grab the target the camera will check to see if the gripper is surrounding the target. This was accomplished by coloring portions of the gripper. First, the tops of both halves of the gripper mechanism were colored blue and half of a blue plastic egg was placed in the center of the mechanism.

Code was written to detect the blue lines on the gripper halves and the blue egg. The lines were defined to be the blue lines detected closest to the egg. The endpoints of the lines were used to define a grab region for the gripper. The center of the target was checked to see if it existed in this region, if it did then the gripper would be able to grab the target.

Another challenge faced when Checking to see if Target was grabbed. This challenge was due to the Tekkotsu language itself. I described the target zone in sketch space. However I needed a way to add a line shape object to a sketch. There was no simple way to accomplish this. As such I wrote a function which could take a line shape object and add it to a sketch object. The function finds the endpoints of the line and re-draws the line in sketch space using the Bresenham Line algorithm.