Use MatLab to move and click mouse, to press keyboard

1 min read

We have an interesting challenge in one of our projects. In our neuroimaging experiment, we need the participant to play a computer game while his brain is scanned (using a NIRS device ETG 4000 in this case). As you can imagine, we need to start the computer game and brain data collection at the same time to make sure the behavior data and neuroimaging data are synchronized. What we usually do is to write some code to start ETG 4000 programmaticly inside the game program; but we can not do it this time because this computer game is developed by others and we can’t inject code into it.

What we want to achieve, simply put, is to click the “Go” button of the game at the same time when we start ETG 4000.

Fortunately there is a solution. We can write a MatLab program to simulate mouse movement and click. Below is the matlab code which will automatically move the mouse to point (640,640) and click it after 5s. If your computer game program requires keyboard input, the code below also contains a snippet for that.

import java.awt.Robot;
import java.awt.event.*;
robot = Robot;

% 5s later, move the mouse to point (640,640) where the 'go' button is,
% then click it.
pause(5);
robot.mouseMove(640, 640);
robot.mousePress(InputEvent.BUTTON1_MASK);
robot.mouseRelease(InputEvent.BUTTON1_MASK);
% fill in the code to start ETG 4000 here

% 5s later, press key SHIFT and W at the same time
%pause(5);
robot.keyPress(java.awt.event.KeyEvent.VK_SHIFT)
robot.keyPress(java.awt.event.KeyEvent.VK_W)
robot.keyRelease(java.awt.event.KeyEvent.VK_W)
robot.keyRelease(java.awt.event.KeyEvent.VK_SHIFT)

As you can see in the following short screen shot, after we run the matlab program (called testmouse), 5s later the mouse moves to (640,640) and clicks, then key SHIFT+W is pressed.

Refer: http://www.mathworks.com/matlabcentral/answers/100545

第十一期 fNIRS Journal Club 通知 2020/8/29,10am

北京时间2020年8月29日周六上午10点, 深圳大学成晓君助理教授将为大家讲解她去年发表的一篇用近红外超扫描揭示人际协调的神经机制的文章。同时,她还会和大家介绍如何用Granger causality分析两个信号的因果关系。欢迎大家参加并参与讨论。 她要讲的文献如下: Cheng, Pan, Hu, Hu (2019) Coordination Elicits Synchronous Brain Activity Between Co-actors: Frequency Ratio Matters Frontiers in neuroscience 13() 1071 Abstract: People...
Xu Cui
48 sec read

第十期 fNIRS Journal Club 视频

在2020/7/25日, 北京航空航天大学的汪待发副教授讲解了他发表的一篇BCI文章。视频如下: Youtube:https://youtu.be/gAQQrmbWSOcYouku:https://v.youku.com/v_show/id_XNDc2OTAxNzkzMg==.html
Xu Cui
4 sec read

第十期 fNIRS Journal Club 通知 2020/7/25,10am

北京时间2020年7月25日周六上午10点,北京航空航天大学的汪待发副教授,博士生导师,将为大家讲解他们组去年发表的一篇脑机交互(BCI)的近红外文章。欢迎大家参加并参与讨论。 他要讲的文献如下:Y. Zheng,D. Zhang, L. Wang, Y. Wang, H. Deng, S. Zhang, D. Li, D. Wang, “Resting-State-Based Spatial Filtering for an fNIRS-Based Motor Imagery...
Xu Cui
1 min read

3 Replies to “Use MatLab to move and click mouse, to press…”

  1. Thank you very much for this code. It was exactly what I needed – to run the EEG recording software with GetSecs in order to sync the experiment with the EEG acquisition without trigger. ( although it is sub-optimal it is good enough for my purposes). Thank you

Leave a Reply

Your email address will not be published. Required fields are marked *