CODEWITHSUNDEEP
c#jsonparsingfilestream
Cosmin
Cosmin

Reputation: 2214

Parsing large json

I'm trying to parse a json file and then operate on it to insert data into a SQL Server 2008 Database.

Example:

var sr = new StreamReader("C:\\path to file\file.json");
var json = JsonSerializer.SerializeToString(sr);
var o = JsonObject.Parse(json);

But I always get this error at the second line - "Timeouts are not supported on this stream."

The Json file looks like this:

"main":{
    "prg": [
               {
                   "Id": 1,
                   "name": "A&E",
                   more fields
               }
    "prg": [
               {
                   "Id": 2,
                   "name": "asda",
                   more fields
               }
 }

I need to make something like this 

foreach (prg in main)
    entity.id = prg.id
    entity.name = prg.name

How can I do this and why I get that timeout exception?

EDIT: To better understand my question this is how I do for an XML file

XmlDocument sourceDoc = new XmlDocument();
sourceDoc.Load(SourcesElement2); // where SourcesElement2 is the path to my XML
XmlNodeList prg = sourceDoc.GetElementsByTagName("prg");

foreach (XmlNode item in prg)
{
    entity.Name= item.SelectSingleNode("name").InnerText;
    ...
}

I have converted the XML to Json and I want to do same thing. For every "prg" node in the Json File insert a new item in the database

EDIT2:

This is what I've done.

    using (
            StreamReader stream =
                File.OpenText(
                    "C:\\path\\Sources.json")
            )
        {
            JObject sources = (JObject) JToken.ReadFrom(new JsonTextReader(stream));

            var a = sources["on"];
            var b = a["sources"];
            var c = b["prgs"];
            foreach (var item in c)
            {
                var d= item.SelectToken("prg");

                // Here d is null

            }

I have the same question as the one from above. For every "prg" node in the Json File insert a new item in the database. How can I do this ? ( path to prg is on/sources/prgs/ )

Upvotes: 1

Views: 4932

Answers (3)

Sam
Sam

Reputation: 1376

I would approach it by converting that JSON object into a C# class and then applying logic to the C# object and/or use DataTables [Note: There are solutions online that show you would to easily pass an Object or List<Object> into a DataTable and then pass it "easily" to SQL]

The first step is still your hiccup in either solution, how do I pull in a large JSON string from filesystem?

if you have the JSON, use json2csharp.com and/or jsonutils.com to retrieve the classes in order to Deserialize it to your object. 

StreamReader re = new StreamReader(@"C:\path to file\file.json");
JsonTextReader reader = new JsonTextReader(re);
YourClass DeserializedObject = se.Deserialize<YourClass>(reader);
Console.WriteLine(DeserializeObject.SomeProperty);

Upvotes: 1

Cosmin
Cosmin

Reputation: 2214

For everyone with the same problem here is what I've done ( NOTE: this is just an example )

By the way, thank you for everyone who tried to answer my question and I'm sorry for my mistakes. 

        List<string> d = new List<string>();

        using (
            StreamReader stream =
                File.OpenText(
                    "C:\\path\\Sources.json")
            )
        {
            JObject sources = (JObject) JToken.ReadFrom(new JsonTextReader(stream));

            var a = sources["on"];
            var b = a["sources"];
            var c = b["prgs"];
            foreach (JObject item in c["prg"].ToList())
            {
                d.Add(item.Value<string>("name"));
            }

        }

        //part below is just for testing
        foreach (var VARIABLE in d)
        {
            Console.WriteLine(VARIABLE);
        }
        Console.ReadLine();

Upvotes: 3

Jakub Konecki
Jakub Konecki

Reputation: 46008

I don't think you want to serialize the stream.

JsonSerializer.SerializeToString(sr)

You want to deserialize from the stream.

JsonSerializer.Deserialize

You might want to use JsonReader for performance reasons.

Your XML example load the whole file in the memory - you don't want to do that for large documents. reader.Read() pattern is better suited for processing large files.

Upvotes: 3

Related Questions