Automating Desktop using RobotJS

RobotJS package gives users ability to control mouse and keyboard of the desktop. In our demo, we will try to draw a circle in MS Paint by controlling mouse actions. Here we will be using version 0.47 of RobotJS.

Development Environment:

  1. JetBrains Webstorm
  2. NPM 5.5.1

We will start by creating a simple node application in WebStorm.

Installing RobotJS:

Enter following command in terminal:

npm install robotjs --save

WebStorm creates a lot of boilerplate for us.

Next, we will set a route in our app.js as ‘draw’ where we will be adding our actions to draw different types of shapes.

... ... var index = require('./routes/index'); var users = require('./routes/users'); var draw = require('./routes/draw'); ... ... app.use('/', index); app.use('/users', users); app.use('/draw', draw); ... ...
Code language: PHP (php)

Next, we will create ‘draw.js’ file in ‘routes’ directory of the project.

We will be using the polar form of the circle ie.:

x = R * cosθ y = R * sinθ

for drawing the points of the circle by using angles ranging from 0-360.

Robot class provides following methods for keyboard and mouse events:

  1. robot.keyTap(“tab”, “alt”) -> Performs a single keypress (first argument is a key and second argument is a modifier).
  2. robot.getScreenSize() -> Returns the size of the desktop screen.
  3. robot.setMouseDelay(0) -> Sets the delay in milliseconds to sleep after a mouse event.
  4. robot.moveMouse(x, y) -> Moves mouse to x, y human like, with the mouse button up.
  5. robot.mouseClick(“left”, false) -> Clicks the mouse (first argument is the mouse key and second argument is for double key press).

The draw.js class is as below:

var express = require('express'); var robot = require("robotjs"); var router = express.Router(); router.get('/circle', function(req, res) { robot.keyTap("tab", "alt"); var screenSize = robot.getScreenSize(); var height = screenSize.height; var width = screenSize.width; var centerX = width / 2; var centerY = height / 2; var radius = Math.min(width, height) / 5; robot.setMouseDelay(0); for (var angle = 0; angle < 360; angle += 0.25) { var x = centerX + (radius * Math.cos(toDegrees(angle))); var y = centerY + (radius * Math.sin(toDegrees(angle))); robot.moveMouse(x, y); robot.mouseClick("left", false); } res.send('Bot at work'); }); function toDegrees (angle) { return angle * (180 / Math.PI); } module.exports = router;
Code language: JavaScript (javascript)

Final Drawing:

Circle drew by Robot events

Posted

in

by

Tags:

Recent Post

  • How to Implement In-Order, Pre-Order, and Post-Order Tree Traversal in Python?

    Tree traversal is an essential operation in many tree-based data structures. In binary trees, the most common traversal methods are in-order traversal, pre-order traversal, and post-order traversal. Understanding these tree traversal techniques is crucial for tasks such as tree searching, tree printing, and more complex operations like tree serialization. In this detailed guide, we will […]

  • Mastering Merge Sort: A Comprehensive Guide to Efficient Sorting

    Are you eager to enhance your coding skills by mastering one of the most efficient sorting algorithms? If so, delve into the world of merge sort in Python. Known for its powerful divide-and-conquer strategy, merge sort is indispensable for efficiently handling large datasets with precision. In this detailed guide, we’ll walk you through the complete […]

  • Optimizing Chatbot Performance: KPIs to Track Chatbot Accuracy

    In today’s digital age, chatbots have become integral to customer service, sales, and user engagement strategies. They offer quick responses, round-the-clock availability, and the ability to handle multiple users simultaneously. However, the effectiveness of a chatbot hinges on its accuracy and conversational abilities. Therefore, it is necessary to ensure your chatbot performs optimally, tracking and […]

  • Reinforcement Learning: From Q-Learning to Deep Q-Networks

    In the ever-evolving field of artificial intelligence (AI), Reinforcement Learning (RL) stands as a pioneering technique enabling agents (entities or software algorithms) to learn from interactions with an environment. Unlike traditional machine learning methods reliant on labeled datasets, RL focuses on an agent’s ability to make decisions through trial and error, aiming to optimize its […]

  • Understanding AI Predictions with LIME and SHAP- Explainable AI Techniques

    As artificial intelligence (AI) systems become increasingly complex and pervasive in decision-making processes, the need for explainability and interpretability in AI models has grown significantly. This blog provides a comprehensive review of two prominent techniques for explainable AI: Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive Explanations (SHAP). These techniques enhance transparency and accountability by […]

  • Building and Deploying a Custom Machine Learning Model: A Comprehensive Guide

    Machine Learning models are algorithms or computational models that act as powerful tools. Simply put, a Machine Learning model is used to automate repetitive tasks, identify patterns, and derive actionable insights from large datasets. Due to these hyper-advanced capabilities of Machine Learning models, it has been widely adopted by industries such as finance and healthcare.  […]

Click to Copy