Robo Referee
Overview
The motivation of this project is to build a system that can act as a tennis referee. A ZED stereo camera and a side-view camera are used.
Transformation
The camera is mounted behind the court on a tripod. The original stereo image is shown here.
A bird’s eye view of the court can make line judgement and game analytics more convenient. Since the camera is not directly above the court, a transformation is needed. The stereo camera allows disparity calculation and estimation of the ground plane ax + by + cz = d. Homography between the camera frame and the bird’s eye view frame can be obtained.
Line and Ball Detection
Ball detection is performed with HSV space color thresholding, and court line detection utilizes Hough Transform. The front and side view videos are synced so the contact location can be estimated. The blue arrow is the velocity vector, and the red dot shows the point of contact. The tennis ball is tracked with the green circle.
Judging
The contact location is transformed to the bird’s eye view frame and compared with court lines. Since a homography on the ground plane is used, the location of the ball in the bird’s eye view is only accurate when the ball is on the ground plane. However, this is sufficient for judging whether the ball should be called in or out.
The videos show the side view, the original view, and the transformed bird's eye view. The ball is correctly judged as "In" in the first video and "Out" in the second video.