CODEWITHSUNDEEP
jsonperlhttpgetmojolicious
Lucas
Lucas

Reputation: 61

Fetching JSON in Perl

I've been working building an emulator in Perl and one of the issues I'm facing is parsing JSON files located in the computer. When I try fetching them from my server, they work fine...

    method getContent(\@arrURLS) {
    my %arrInfo;
    my $resUserAgent = Mojo::UserAgent->new;
    foreach my $strURL (@arrURLS) {
        $resUserAgent->get($strURL => sub {
            my($resUserAgent, $tx) = @_;
            if ($tx->success) {
                my $strName = basename($strURL, '.json');
                my $arrData = $tx->res->body;
                $arrInfo{$strName} = $arrData;
            }
            Mojo::IOLoop->stop;
        });
        Mojo::IOLoop->start;
    }
    return \%arrInfo;
}

Let's assume @arrURLS is:

my @arrURLS = ("file:///C:/Users/Test/Desktop/JSONS/first.json", "file:///C:/Users/Test/Desktop/JSONS/second.json");

The above url's are the one's that aren't working, however if I change that to:

my @arrURLS = ("http://127.0.0.1/test/json/first.json", "http://127.0.0.1/test/json/second.json");

it works.

Also I would like to use something better than Mojo::UserAgent because it seems a bit slow, when I was using Coro with LWP::Simple it was much faster but unfortunately Coro is broken in Perl 5.22...

Upvotes: 3

Views: 412

Answers (3)

clt60
clt60

Reputation: 63892

Myself using WWW::Mechanize for all such tasks. From the doc:

WWW::Mechanize is a proper subclass of LWP::UserAgent and you can also use any of LWP::UserAgent's methods.

what means you can feed it with file:// type URLs too.

For example, the following one-liner dumps your passwd file.

perl -MWWW::Mechanize -E 'say WWW::Mechanize->new->get("file://etc/passwd")->content'

or an example without any error handling...

use 5.014;
use warnings;
use WWW::Mechanize;
my $mech = WWW::Mechanize->new;
$mech->get('file://some/path');
say $mech->content;

Anyway, probably is better to use for local files some file-based utility, myself using for all file-things the Path::Tiny module, which has (not limited only) an method for file slurping, such:

use Path::Tiny;
my $content = path('/some/path')->slurp;

or just plain perl:

open my $fh, '<', '/some/file' or die "...";
my $content = do { local $/; <$fh> };
close $fh;

Upvotes: 2

Borodin
Borodin

Reputation: 126722

It's important always to say what additional modules you're using. I think your code uses Method::Signatures, and I've tested the code below only to check that it compiles with that module in place

Mojolicious is an excellent tool for its purpose, but it is focused on HTTP URLs. LWP::UserAgent is much more general-purpose, and the documentation for LWP says this

Provides an object oriented model of HTTP-style communication. Within this framework we currently support access to http, https, gopher, ftp, news, file, and mailto resources

Your method becomes something like this. It's untested

method get_content(\@urls) {

    my %info;

    my $ua = LWP::UserAgent->new;
 
    for my $url (@urls) {

        my $res = $ua->get($url);
        die $res->status_line unless $res->is_success;

        my $name     = basename($url) . '.json';
        my $data     = $res->decoded_content;
        $info{$name} = $data;
    }

    \%info;
}

I would also encourage you to drop Hungarian notation in the context of Perl code, as the language already has its sigils that denote the data type

@arrURLS duplicates the information that this arrURLS is an array, while %arrInfo is just wrong as this arrInfo is a hash. $arrData is actually a scalar, although perhaps some indicator that it is also a reference may help, and $arrURLS[0] is also a scalar (hence the dollar)

There is also nothing to stop you using $arrURLS (which is a completely separate variable from @arrURLS)

Upvotes: 1

simbabque
simbabque

Reputation: 54323

User Agents are mainly for downloading files through http. They are usually not expected to handle filesystem URIs. You need to open and read the file yourself, or use a module like File::Slurp that does it for you.

It could look something like this.

use File::Slurp 'read_file';

method getContent(\@arrURLS) {
    my %arrInfo;
    my $resUserAgent = Mojo::UserAgent->new;
    foreach my $strURL (@arrURLS) {
        if (substr($strURL, 0, 4) eq 'file') {
            $arrInfo{basename($strURL, '.json')} = read_file($strURL);
        } else {
            $resUserAgent->get($strURL => sub {
                my($resUserAgent, $tx) = @_;
                if ($tx->success) {
                    my $strName = basename($strURL, '.json');
                    my $arrData = $tx->res->body;
                    $arrInfo{$strName} = $arrData;
                }
                Mojo::IOLoop->stop;
            });
            Mojo::IOLoop->start;
        }
    }
    return \%arrInfo;
}

Upvotes: 4

Related Questions