Kasparov vs. Deep Blue game. Baxter is playing with red pieces as Kasparov.

Baxter Chess - SVM for Computer Vision (Project is currently in progress)


Implement Baxter robot to play chess, recognize the location and state of the chess board and chess pieces by leveraging machine learning and image processing techniques.   


Baxter Robot by Rethink Robotics, Chess board, Chess pieces.


ROS, MoveIt!, OpenCV, Python.


Kinematics of the robot were implemented using MoveIt! libraries. Baxter's right arm was used for image capturing and thus had the grippers removed and the left arm was used for the pick and place actions.

The computer vision part was done as follows. The board was detected using the Canny Edge detection algorithm and the Harris corner detection algorithms. The chess pieces were detected using linear SVM classifier with HOG features. 

Currently there are three nodes (processes) and 4 topics implemented. vision_rightarm_node is controlling the movement of the right arm, the gathering and processing of images, detection of the chess board using Canny edge detection and Harris corner detection algorithms, feature extraction using HOG and classification of chess pieces using a linear SVM. The move_leftarm_node it responsible for the movement of the left arm and the execution of the chess move action. And the chess_engine node is responsible for keeping track of the board state and proving the best possible chess moves to execute based on the current chess board state. Currently the chess moves are hard-coded, later they will be generated by the integrated chess engine. 

The following topics are currently implemented:  

  • the ChessMove topic containing the desired chess type, source and destination locations in Baxter coordinates. chess_engine node publishes to this topic and move_rightarm_node subscribes to it.
  •  the GameMode topic containing one field that can correspond to either calibration mode, Baxter vs Human mode, Human vs Human mode, and Baxter vs Baxter mode. (Not curently used)
  • RobotState topic that contains one field specifying whether right arm, left arm or neither is active. Both vision_rightarm_node and move_leftarm_node subscribe and publish to this topic.
  • SquareState containing  square ID, square color, square center location in Baxter coordinates, square occupancy, chess piece color, chess piece type, time step, and user that modified that square. vision_rightarm_node publishes to this topic and chess_engine node subscribes to it.

GitHub link. [Private Repo. Please ask permission to view. ]