CODEWITHSUNDEEP
javascriptjsonnode.jsregexexpress
lycrake
lycrake

Reputation: 45

Parsing Reddits JSON in Node. Javascript parser doesn't like comments

Hope you're good

I've got a project where I want to scan the JSON of reddit.com/r/pics. Pull a random image and show it on a webpage.

Reddits json at https://www.reddit.com/r/pics/.json?jsonp=1 pulls some characters at the start before the JSON begins '/**/1(' which is throwing off the Javascript parser.

I can regex edit some of the characters out (see code), but ultimately it fails

SyntaxError: Unexpected string in JSON at position 1

If I remove these, save the file locally, then it seems to work fine and as expected.

What is the best way to tackle this problem?

Am I missing something totally?

Thanks in advance!

Here is my code, its in Node.js

var express = require('express');
var router = express.Router();
var request = require('request');
var fs = require('fs');
const https = require('https');


let detailedObject;
let pageNo = 1;
let object;
let lengthofObject;
let urlImg;
let newObject;
var file = fs.createWriteStream("reddit.json");



// SAVES REDDIT.JSON TO FILE
var request = https.get("https://www.reddit.com/r/pics/.json?jsonp=1", response => {
  response.pipe(file);
});


// RANDOM NUMBER GENERATOR
function randomNo(num) {
var randomNo = Math.floor(Math.random() * num) + 1 ;
  return randomNo;
}


//REMOVE WEIRD CHARATCERS IN JSON
function removeChar(object) {

  object = object.replace(/\(/g,"").replace(/\)/g,"").replace(/\//g,"").replace(/\*/g,"").replace(/\{/g,"");

  newObject = JSON.parse(object);

  return newObject;

}


function fetchImg() {

fs.readFile('./reddit.json', 'utf8', function (err, data) {
    if (err) throw err; // we'll not consider error handling for now

    if (data) {


        data = data.trim();

        var storage = removeChar(data);

        detailedObject = storage.data.children;

        lengthofObject = detailedObject.length;

        var randomValue = randomNo(lengthofObject);

        urlImg = detailedObject[randomValue].data.url

        return urlImg;

     };


});

}

router.get('/', (req, res, next) => {
  //query needs to be inside function

fetchImg();

return res.render('index', {"urlImg": urlImg});

});



module.exports = router;

UPDATE

I've found the issue, Reddits JSON is wrapped in

/**/1(

and there is also a closing bracket at the end of the JSON that I missed

)

If I remove all of these then it parses without stringify. What I need to figure out is how to remove the starting 5 characters (which will increase depending on the amount of page numbers) and the final 1 character dynamically...

If I regex it all out then all of the slashes and brackets are removed throughout the query...

SOLVED

Once the .json has been downloaded by fs, I immediately use slice to remove the first 6 characters and the last character. I have a variable pageReducer that reduces more characters from the JSON depending on which random JSON my app pulls

sliceObject = object.slice(6 + pageReducer, -1);

Upvotes: 1

Views: 87

Answers (1)

Chris
Chris

Reputation: 993

//REMOVE WEIRD CHARATCERS IN JSON
function removeChar(object) {

  object = object.replace(/\(/g,"").replace(/\)/g,"").replace(/\//g,"").replace(/\*/g,"").replace(/\{/g,"");

  newObject = JSON.parse(object);

  return newObject;

}

The error is here: newObject = JSON.parse(object);

JSON.stringify it first and then you will be able to use JSON.parse and proceed with your code.

I think the reason is because the "object" parameter is still just a javascript string after removing the invalid characters, but it isn't yet a valid JSON editable object. JSON.stringify converts it to valid JSON string. JSON.parse will then be able to parse the valid JSON string.

Upvotes: 1

Related Questions