CODEWITHSUNDEEP
jsonnode.jscsvparsingextract
Edon
Edon

Reputation: 1216

Methods to convert CSV to unique JSON

I need to convert a .CSV to a specific .JSON format, and I decided to use the csvtojson package from NPM since it seemed to be designed for this sort of thing.

First, a little background on my problem. I have .CSV data that looks similar to this:

scan_type, date/time, source address, source-lat, source-lng, dest address, dest-lat, dest-lng

nexpose,2016-07-18 18:21:44,1008,40.585260,-10.124120,10.111.131.4,10.844880,-10.933360

I have a plain csv-to-json converter here , but there is a problem. It outputs a rather plain file, and I need to split the source/destination, and add some special formatting that I will show later on.

[
 {
 "scan_type": "nexpose",
 "date/time": "2026-07-28 28:22:44",
 "source_address": 2008,
 "source-lat": 10.58526,
 "source-lng": -105.08442,
 "dest_address": "11.266.282.0",
 "dest-lat": 11.83388,
 "dest-lng": -111.82236
 }
]

The first thing I need to do is be able to separate the "source" values, from the "destination" values. Here is an example of what I want the "source" values to look like:

(var destination will be exactly the same format)

var source={
    id: "99.58926-295.09492",
    "source-lat": 49.59926,
    "source-lng": -209.98942,
    "source_address": 2009,
    x: {
        valueOf: function() {
            var latlng=[
                49.58596,
                -209.08442
            ]; 
var xy = map.FUNCTION_FOR_CONVERTING_LAT_LNG_TO_X_Y(latlng);
return xy[0]; //xy.x
        },

        y: {
            valueOf: function(){
                varlatlng=[
                    49.58596,
                    -209.08442
                ];                          
var xy = map.FUNCTION_FOR_CONVERTING_LAT_LNG_TO_X_Y(latlng);
return xy[1]; //xy.y
        }
    }

So, my question is, how should I approach converting my data? Should I convert everything with csvtojson? Or should I convert it from the plain .JSON file I generated?

Does anyone have any advice, or similar examples, they could share on how to approach this problem?

Upvotes: 1

Views: 139

Answers (1)

Hans Strausl
Hans Strausl

Reputation: 614

I do a lot of work with parsing CSV data and as I am sure you have seen, CSV is very hard to parse and work with correctly as there are a huge number of edge cases that can break even the most rugged of parsers (although it looks like your dataset is fairly plain so that isn't a huge concern). Not to mention you could potentially run into corruption by performing operations while reading from disk, so it is a much better idea to get the data from CSV into a JSON file and then make any manipulations to a JSON object loaded from that "plain" JSON file.

tl;dr: convert your data from the plain .JSON file

Upvotes: 0

Related Questions