Automating Desktop using RobotJS

RobotJS package gives users ability to control mouse and keyboard of the desktop. In our demo, we will try to draw a circle in MS Paint by controlling mouse actions. Here we will be using version 0.47 of RobotJS.

Development Environment:

  1. JetBrains Webstorm
  2. NPM 5.5.1

We will start by creating a simple node application in WebStorm.

Installing RobotJS:

Enter following command in terminal:

npm install robotjs --save

WebStorm creates a lot of boilerplate for us.

Next, we will set a route in our app.js as ‘draw’ where we will be adding our actions to draw different types of shapes.

... ... var index = require('./routes/index'); var users = require('./routes/users'); var draw = require('./routes/draw'); ... ... app.use('/', index); app.use('/users', users); app.use('/draw', draw); ... ...
Code language: PHP (php)

Next, we will create ‘draw.js’ file in ‘routes’ directory of the project.

We will be using the polar form of the circle ie.:

x = R * cosθ y = R * sinθ

for drawing the points of the circle by using angles ranging from 0-360.

Robot class provides following methods for keyboard and mouse events:

  1. robot.keyTap(“tab”, “alt”) -> Performs a single keypress (first argument is a key and second argument is a modifier).
  2. robot.getScreenSize() -> Returns the size of the desktop screen.
  3. robot.setMouseDelay(0) -> Sets the delay in milliseconds to sleep after a mouse event.
  4. robot.moveMouse(x, y) -> Moves mouse to x, y human like, with the mouse button up.
  5. robot.mouseClick(“left”, false) -> Clicks the mouse (first argument is the mouse key and second argument is for double key press).

The draw.js class is as below:

var express = require('express'); var robot = require("robotjs"); var router = express.Router(); router.get('/circle', function(req, res) { robot.keyTap("tab", "alt"); var screenSize = robot.getScreenSize(); var height = screenSize.height; var width = screenSize.width; var centerX = width / 2; var centerY = height / 2; var radius = Math.min(width, height) / 5; robot.setMouseDelay(0); for (var angle = 0; angle < 360; angle += 0.25) { var x = centerX + (radius * Math.cos(toDegrees(angle))); var y = centerY + (radius * Math.sin(toDegrees(angle))); robot.moveMouse(x, y); robot.mouseClick("left", false); } res.send('Bot at work'); }); function toDegrees (angle) { return angle * (180 / Math.PI); } module.exports = router;
Code language: JavaScript (javascript)

Final Drawing:

Circle drew by Robot events

Posted

in

by

Tags:

Recent Post

  • 12 Essential SaaS Metrics to Track Business Growth

    In the dynamic landscape of Software as a Service (SaaS), the ability to leverage data effectively is paramount for long-term success. As SaaS businesses grow, tracking the right SaaS metrics becomes essential for understanding performance, optimizing strategies, and fostering sustainable growth. This comprehensive guide explores 12 essential SaaS metrics that every SaaS business should track […]

  • Bagging vs Boosting: Understanding the Key Differences in Ensemble Learning

    In modern machine learning, achieving accurate predictions is critical for various applications. Two powerful ensemble learning techniques that help enhance model performance are Bagging and Boosting. These methods aim to combine multiple weak learners to build a stronger, more accurate model. However, they differ significantly in their approaches. In this comprehensive guide, we will dive […]

  • What Is Synthetic Data? Benefits, Techniques & Applications in AI & ML

    In today’s data-driven era, information is the cornerstone of technological advancement and business innovation. However, real-world data often presents challenges—such as scarcity, sensitivity, and high costs—especially when it comes to specific or restricted datasets. Synthetic data offers a transformative solution, providing businesses and researchers with a way to generate realistic and usable data without the […]

  • Federated vs Centralized Learning: The Battle for Privacy, Efficiency, and Scalability in AI

    The ever-expanding field of Artificial Intelligence (AI) and Machine Learning (ML) relies heavily on data to train models. Traditionally, this data is centralized, aggregated, and processed in one location. However, with the emergence of privacy concerns, the need for decentralized systems has grown significantly. This is where Federated Learning (FL) steps in as a compelling […]

  • Federated Learning’s Growing Role in Natural Language Processing (NLP)

    Federated learning is gaining traction in one of the most exciting areas: Natural Language Processing (NLP). Predictive text models on your phone and virtual assistants like Google Assistant and Siri constantly learn from how you interact with them. Traditionally, your interactions (i.e., your text messages or voice commands) would need to be sent back to […]

  • What is Knowledge Distillation? Simplifying Complex Models for Faster Inference

    As AI models grow increasingly complex, deploying them in real-time applications becomes challenging due to their computational demands. Knowledge Distillation (KD) offers a solution by transferring knowledge from a large, complex model (the “teacher”) to a smaller, more efficient model (the “student”). This technique allows for significant reductions in model size and computational load without […]

Click to Copy