CODEWITHSUNDEEP
javascriptseleniumprotractor
dhru soni
dhru soni

Reputation: 37

not able to fetch text data from web url using javascript

I need to extract text data from web url (http://www.africau.edu/images/default/sample.pdf)

I used two node_module.

1) crawler-Request

  it('Read Pdf Data using crawler',function(){
        const crawler = require('crawler-request');
        function response_text_size(response){
            response["size"] = response.text.length;
            return response;
        }
        crawler("http://www.africau.edu/images/default/sample.pdf",response_text_size).then(function(response){
            // handle response

            console.log("Reponse =" + response.size);
        });

    });

What happen for this it will not print anything on console.

2) pfd2json/pdfparser

it('Read Data from url',function(){
        var request = require('request');
        var pdf = require('pfd2json/pdfparser');
        var fs = require('fs');
        var pdfUrl = "http://www.africau.edu/images/default/sample.pdf";
        let databuffer = fs.readFileSync(pdfUrl);
        pdf(databuffer).then(function(data){
            var arr:Array<String> = data.text;
            var n = arr.includes('Thursday 02 May');
            console.log("Print Array " + n);
        });

    });

I am able to access data from local path but not able to extract it from url.

Upvotes: 1

Views: 224

Answers (1)

salixor
salixor

Reputation: 122

The issue here is that you are using the fs module (File System) to read a file on a distant server.

You also mistyped the pdf2json module, which should give you an error ?

You did require the request module. This module will make it possible to access that distant file. Here's one way to do this :

it('Read Data from url', function () {
    var request = require('request');
    var PDFParser = require('pdf2json');

    var pdfUrl = 'http://unec.edu.az/application/uploads/2014/12/pdf-sample.pdf';

    var pdfParser = new PDFParser(this, 1);

    // executed if the parser fails for any reason
    pdfParser.on("pdfParser_dataError", errData => console.error(errData.parserError));
    // executed when the parser finished
    pdfParser.on("pdfParser_dataReady", pdfData => console.log(pdfParser.getRawTextContent()));

    // request to get the pdf's file content then call the pdf parser on the retrieved buffer
    request({ url: pdfUrl, encoding: null }, (error, response, body) => pdfParser.parseBuffer(body));
});

This will make it possible to load the distant .pdf file in your program.

I'd recommend looking at the pdf2json documentation if you want to do more. This will simply output the textual content of the .pdf file when the parser has completed reading data.

Upvotes: 1

Related Questions