Cantzler, H.Hoile, C.Peter Hall and Philip Willis2016-02-092016-02-0920033-905673-54-1https://doi.org/10.2312/vvg.20031008This paper presents a novel approach for man machine interaction applying real time computer vision techniques. We use a handheld camera to control the mouse cursor on a computer display. The camera captures an image of the display in its eld of view and this can be used to judge the camera's position and orientation relative to the display. The problem is modelled as a plane-to-plane projection (homography). Once the mapping of the display in the camera view to the real world display is known, the intersection between the central axis of the camera and the surface of the display can be computed. The mouse pointer is then moved to the corresponding display position. This calculation can be iterated continuously to update the mouse cursor position as the camera position and orientation changes. The camera can then be used to control the mouse cursor just like a laser pointer controls a laser dot. A prototype has been developed to demonstrate the approach.10.2312/vvg.20031008