CODEWITHSUNDEEP
azure-data-lakeu-sql
Sandesh
Sandesh

Reputation: 3004

Load a JSON file in a C# object using U-SQL

I have a JSON file stored in the Data Lake Store. I can extract the JSON file using the JsonExtractor from Microsoft.

Is it possible to load the JSON file in a POCO object without using EXTRACT command? If I use EXTRACT command is it possible for me combine all the rows in a single C# object?

Below is a sample JSON file which I want to de-serialize and store in a C# object 

{
  "sourcePath": "wasb://[email protected]/Input/{*}.txt",
  "destinationPath": "wasb://[email protected]/Output/myfile.txt",
  "errorPath": "wasb://[email protected]/Error/error.txt",
  "schema": [
    {
      "name": "column1",
      "type": "string",
      "allowNull": true,
      "minLength": 12,
      "maxLength": 50
    },
    {
      "name": "column2",
      "type": "int",
      "allowNull": true,
      "minLength": 0,
      "maxLength": 0
    },
    {
      "name": "column3",
      "type": "bool",
      "allowNull": true,
      "minLength": 0,
      "maxLength": 0
    },
    {
      "name": "column4",
      "type": "DateTime",
      "allowNull": false,
      "minLength": 0,
      "maxLength": 0
    }
  ]
}

Upvotes: 0

Views: 340

Answers (1)

Michael Rys
Michael Rys

Reputation: 6684

You can write your own custom Extractor that reads the data (input.baseStream) and you can create your object. Take a look at the Microsoft JSON Extractor for the pattern.

Note that you will have 1/2 GB of main memory limit for your extractor.

Upvotes: 1

Related Questions