NODE.JS TUTORIAL – HOW TO USE REQUEST MODULE

In any application framework one basic need is to be able to make REST calls over HTTP/HTTPS, they are widely used and very useful in cases of API interactions and web scraping. We can use the Node.js core modules http or https to perform these calls but that that can be a cumbersome process and the Request module makes it simple to perform these HTTP requests. In this tutorial we will learn about how to get started with the request module to perform HTTP calls.

Installing Request Module

Request is a third party module, hence in order to install the request module, simply run the following command in your project directory:

> npm install request

If you’re unaware, npm is a package manager, which provides a central repository for custom open source modules for Node.js and JavaScript. npm makes it simple to manage modules versions and distribution. We used the npm install command to install the required module in our project.

Getting started with the Request Module

The Request module is built in a way that it makes it very simple to make HTTP(S)calls. It also follows redirects by default, however that is configurable. In this section we will use shorthand notations of the request module to get our job done.

Simple GET call over HTTP

Lets try to print the HTML off the modulus.io homepage by making a GET request to the modulus.io homepage URL:

//Load the request module
var request = require('request');
//Lets try to make a HTTP GET request to modulus.io's website.
request('http://www.modulus.io', function (error, response, body) {
    if (!error && response.statusCode == 200) {
        console.log(body); // Show the HTML for the Modulus homepage.
    }
});

In the above example we did not specify the GET method anywhere but by default the request module makes a GET request, if not specified explicitly. Alternatively, we can also use request.get, using request.get makes the code more readable.

The callback function contains three parameters:

  1. Error: This is the first parameter of the callback function. It will be null in case there is no error or it will contain the error object with appropriate details. You should always check for the error before continuing to process the response.
  2. Response: This is the second parameter in the callback function. This contains the http.IncomingMessage object which contains additional data about the http/https request i.e status code, headers, etc.
  3. Body: This is the third parameter and contains the body of the http/https response. This is a string type containing the contents of the body if the response is in text format, the body is a buffer if the response data is in the binary / octet streamencoding, finally the body will be a JSON object if the response is in JSON encoding.

Simple GET call over HTTPS

For making HTTPS calls the request module automatically detects from the URL and routes your request via https module internally. For making HTTPS calls all you have to do is, in the UEL specify the protocol as the https. Below is an example of the HTTPS call:

//Load the request module
var request = require('request');

//Lets try to make a HTTPS GET request to modulus.io's website.
//All we did here to make HTTPS call is changed the `http` to `https` in URL.
request('https://modulus.io', function (error, response, body) {
    //Check for error
    if(error){
        return console.log('Error:', error);
    }

    //Check for right status code
    if(response.statusCode !== 200){
        return console.log('Invalid Status Code Returned:', response.statusCode);
    }

    //All is good. Print the body
    console.log(body); // Show the HTML for the Modulus homepage.

});

The Request module has some shortcut methods for making calls in common REST methods like POST, PUT or DELETE, these notations are as follows:

  • request.post is for POST request.
  • request.put is for PUT request.
  • request.delete is for DELETE request.
  • request.post is for GET request.
  • request.patch is for PATCH request.

Some More Configurations

In the previous section we discussed on how to make simple request calls via quick shortcut methods. Here we will discuss another syntax where you can configure the request per your needs. In place of a string URL, the request module also optionally takes a config object where we can pass the desired configurations. The syntax is: request(optionsObject, callback). Lets see an example below:

//Load the request module
var request = require('request');

//Lets configure and request
request({
    url: 'http://modulus.io', //URL to hit
    qs: {from: 'blog example', time: +new Date()}, //Query string data
    method: 'GET', //Specify the method
    headers: { //We can define headers too
        'Content-Type': 'MyContentType',
        'Custom-Header': 'Custom Value'
    }
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
    }
});

In the above example you can see that we passed a configuration object with various configurations, there is a huge list of configurations, which you can use for your request. Here we just demonstrated how you pass headers and query data as well as define the HTTP method explicitly.

Posting a string body

In this example we will see how we can post a String or Buffer body using the request module. In this example we will use the body configuration property for posting the data.

//Load the request module
var request = require('request');

//Lets configure and request
request({
    url: 'https://modulus.io/contact/demo', //URL to hit
    qs: {from: 'blog example', time: +new Date()}, //Query string data
    method: 'POST',
    headers: {
        'Content-Type': 'MyContentType',
        'Custom-Header': 'Custom Value'
    },
    body: 'Hello Hello! String body!' //Set the body as a string
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
    }
});

Posting as a form

The request module has a helper configuration to post forms. You just need to pss the form data as an object to the form configuration property and it will automatically url-encode it and put it in the body with the desired application/x-www-form-urlencodedcontent type. Lets see an example below:

//Load the request module
var request = require('request');

//Lets configure and request
request({
    url: 'https://modulus.io/contact/demo', //URL to hit
    qs: {from: 'blog example', time: +new Date()}, //Query string data
    method: 'POST',
    //Lets post the following key/values as form
    form: {
        field1: 'data',
        field2: 'data'
    }
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
    }
});

Posting JSON data

If we have an object and we want to post it as the JSON data we can do that by passing that object in the json configuration property. Lets see an example below:

//Load the request module
var request = require('request');

//Lets configure and request
request({
    url: 'https://modulus.io/contact/demo', //URL to hit
    qs: {from: 'blog example', time: +new Date()}, //Query string data
    method: 'POST',
    //Lets post the following key/values as form
    json: {
        field1: 'data',
        field2: 'data'
    }
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
}
});

Streams

The Request module has a fantastic first citizen interface for Streams. Streams are like a continuous flow of data from one source to a target. Using streams can really save us memory as well as help us to work with large data. If possible use of streaming API is recommended. We will cover some very basic streaming based approaches here, but this is just a start. Streams have infinite possibilities. Lets consider an example; we want to send a request for an image to a third party server and stream the response to a file.

//Load the request module
var request = require('request');
//Load fs module
var fs = require('fs');
//Lets define a write stream for our destination file
var destination = fs.createWriteStream('./savedImage.png');
//Lets save the modulus logo now
request('https://my.modulus.io/img/modulus-logoSmall-gray20.png').pipe(destination);

Here all we are doing is piping the readStream from the request we made to the write stream of our destination file. It’s that simple. We can also listen to error events as described below:

//Lets save the modulus logo now
request('https://my.modulus.io/img/modulus-logoSmall-gray20.png')
.pipe(destination)
.on('error', function(error){
    console.log(error);
});

Sending data from a file to a request via streams

We can also read data from a file and send it to a remote server via our request. Lets consider an example below:

//Load the request module
var request = require('request');
//Load fs module
var fs = require('fs');
//Lets define a read stream from our source file, it could be any JSON file.
var source = fs.createReadStream('./sampleData.json');
//Let’s send our data via POST request
source.pipe(request.post('https://modulus.io/contact/demo'));

In this example we created a read stream from our source file and piped it to the request.

Conclusion

In this tutorial we learned the basics of the request module. We saw how we can use shorthand methods, custom configurations, as well as the basics of the streaming API. For future reference, you can read more detailed streams usage in the request module as well as some of its auth features.


Posted

in

,

by

Tags:

Recent Post

  • 12 Essential SaaS Metrics to Track Business Growth

    In the dynamic landscape of Software as a Service (SaaS), the ability to leverage data effectively is paramount for long-term success. As SaaS businesses grow, tracking the right SaaS metrics becomes essential for understanding performance, optimizing strategies, and fostering sustainable growth. This comprehensive guide explores 12 essential SaaS metrics that every SaaS business should track […]

  • Bagging vs Boosting: Understanding the Key Differences in Ensemble Learning

    In modern machine learning, achieving accurate predictions is critical for various applications. Two powerful ensemble learning techniques that help enhance model performance are Bagging and Boosting. These methods aim to combine multiple weak learners to build a stronger, more accurate model. However, they differ significantly in their approaches. In this comprehensive guide, we will dive […]

  • What Is Synthetic Data? Benefits, Techniques & Applications in AI & ML

    In today’s data-driven era, information is the cornerstone of technological advancement and business innovation. However, real-world data often presents challenges—such as scarcity, sensitivity, and high costs—especially when it comes to specific or restricted datasets. Synthetic data offers a transformative solution, providing businesses and researchers with a way to generate realistic and usable data without the […]

  • Federated vs Centralized Learning: The Battle for Privacy, Efficiency, and Scalability in AI

    The ever-expanding field of Artificial Intelligence (AI) and Machine Learning (ML) relies heavily on data to train models. Traditionally, this data is centralized, aggregated, and processed in one location. However, with the emergence of privacy concerns, the need for decentralized systems has grown significantly. This is where Federated Learning (FL) steps in as a compelling […]

  • Federated Learning’s Growing Role in Natural Language Processing (NLP)

    Federated learning is gaining traction in one of the most exciting areas: Natural Language Processing (NLP). Predictive text models on your phone and virtual assistants like Google Assistant and Siri constantly learn from how you interact with them. Traditionally, your interactions (i.e., your text messages or voice commands) would need to be sent back to […]

  • What is Knowledge Distillation? Simplifying Complex Models for Faster Inference

    As AI models grow increasingly complex, deploying them in real-time applications becomes challenging due to their computational demands. Knowledge Distillation (KD) offers a solution by transferring knowledge from a large, complex model (the “teacher”) to a smaller, more efficient model (the “student”). This technique allows for significant reductions in model size and computational load without […]

Click to Copy