Use MatLab to move and click mouse, to press keyboard

1 min read

We have an interesting challenge in one of our projects. In our neuroimaging experiment, we need the participant to play a computer game while his brain is scanned (using a NIRS device ETG 4000 in this case). As you can imagine, we need to start the computer game and brain data collection at the same time to make sure the behavior data and neuroimaging data are synchronized. What we usually do is to write some code to start ETG 4000 programmaticly inside the game program; but we can not do it this time because this computer game is developed by others and we can’t inject code into it.

What we want to achieve, simply put, is to click the “Go” button of the game at the same time when we start ETG 4000.

Fortunately there is a solution. We can write a MatLab program to simulate mouse movement and click. Below is the matlab code which will automatically move the mouse to point (640,640) and click it after 5s. If your computer game program requires keyboard input, the code below also contains a snippet for that.

import java.awt.Robot;
import java.awt.event.*;
robot = Robot;

% 5s later, move the mouse to point (640,640) where the 'go' button is,
% then click it.
pause(5);
robot.mouseMove(640, 640);
robot.mousePress(InputEvent.BUTTON1_MASK);
robot.mouseRelease(InputEvent.BUTTON1_MASK);
% fill in the code to start ETG 4000 here

% 5s later, press key SHIFT and W at the same time
%pause(5);
robot.keyPress(java.awt.event.KeyEvent.VK_SHIFT)
robot.keyPress(java.awt.event.KeyEvent.VK_W)
robot.keyRelease(java.awt.event.KeyEvent.VK_W)
robot.keyRelease(java.awt.event.KeyEvent.VK_SHIFT)

As you can see in the following short screen shot, after we run the matlab program (called testmouse), 5s later the mouse moves to (640,640) and clicks, then key SHIFT+W is pressed.

Refer: http://www.mathworks.com/matlabcentral/answers/100545


Don't want to miss new papers in your field? Check out Stork we developed:

After fNIRS experiment …

Xu Cui
13 sec read

nirs2img, create an image file from NIRS data

I was asked where to get nirs2img script. Here it is. The download link is at the bottom of this article. nirs2img is to...
Xu Cui
51 sec read

3 Replies to “Use MatLab to move and click mouse, to press…”

  1. Thank you very much for this code. It was exactly what I needed – to run the EEG recording software with GetSecs in order to sync the experiment with the EEG acquisition without trigger. ( although it is sub-optimal it is good enough for my purposes). Thank you

Leave a Reply

Your email address will not be published. Required fields are marked *