CODEWITHSUNDEEP
c#csvfilehelpers
L.Fuentes
L.Fuentes

Reputation: 35

Parsing csv with optional columns using FileHelper

I am using Filehelper 3.1.5 to parse a CSV file but my problem that the CSV file should support many optional columns and I have not found out to configure FileHelper for this task.

Here is an example:

[DelimitedRecord(";")]
[IgnoreEmptyLines]
public class TestRecord
{
    //Mandatory
    [FieldNotEmpty]
    public string A;

    [FieldOptional]
    public string B;

    [FieldOptional]
    public string C;
}

I would like it to be possible to handle data like this:

A;C
TestA1;TestC1
TestA2;TestC1

But when I parse it, I will get "TestC1" as a result of records[1].B

var engine = new FileHelperEngine<TestRecord>();
var records = engine.ReadFile("TestAC.csv");

string column = records[1].C;
Assert.IsTrue(column.Equals("TestC1"));  //Fails, returns ""

column = records[1].B;
Assert.IsTrue(column.Equals("TestC1"));  //True, but that was not what I wanted

Thankful for any advice!

Upvotes: 2

Views: 4318

Answers (2)

netniV
netniV

Reputation: 2418

I think you should decorate your columns with titles, such as:

[DelimitedRecord(";")]
[IgnoreEmptyLines]
public class TestRecord
{
    //Mandatory
    [FieldNotEmpty, FieldOrder(0), FieldTitle("A")]
    public string A;

    [FieldOptional, FieldOrder(1), FieldTitle("B")]
    public string B;

    [FieldOptional, FieldOrder(2), FieldTitle("C")]
    public string C;
}

This way, the runtime knows what the column names are and will parse them accordingly. Otherwise, all it knows is that you have two columns in the file and expects extra semi-colons. So, the following would have worked with your original setup:

A;;C TestA1;;TestC1 TestA2;;TestC1

This only works on FileHelpers v2 as v3 no longer has FieldTitle

Upvotes: 0

netniV
netniV

Reputation: 2418

Tested against File Helpers Version 3.2.5

In order to make the FileHelper.Engine correctly identify your columns, you would have to dynamically remove the fields no longer in use. The following is based on your code with a few added bits and run from a console program:

        string tempFile = System.IO.Path.GetTempFileName();
        System.IO.File.WriteAllText(tempFile, @"A;C\r\n\TestA1;TestC1\r\nTestA2;TestC1");
        var engine = new FileHelperEngine<TestRecord>();
        var records = engine.ReadFile(tempFile, 1);

        // Get the header text from the file
        var headerFile = engine.HeaderText.Replace("\r", "").Replace("\n", "");

        // Get the header from the engine record layout
        var headerFields = engine.GetFileHeader();

        // Test fixed string against column as column could be null and Debug.Assert can't use .Equals on a null object!
        string column = records[0].C;
        Debug.Assert("TestC1".Equals(column), "Test 1 - Column C does not equal 'TestC1'");  //Fails, returns ""

        // Test fixed string against column as column could be null and Debug.Assert can't use .Equals on a null object!
        column = records[0].B;
        Debug.Assert(!"TestC1".Equals(column), "Test 1 - Column B does equal 'TestC1'");  //True, but that was not what I wanted

        // Create a new engine otherwise we get some random error from Dynamic.Assign once we start removing fields
        // which is presumably because we have called ReadFile() before hand.
        engine = new FileHelperEngine<TestRecord>();

        if (headerFile != headerFields)
        {
            var fieldHeaders = engine.Options.FieldsNames;
            var fileHeaders = headerFile.Split(';').ToList();

            // Loop through all the record layout fields and remove those not found in the file header
            for (int index = fieldHeaders.Length - 1; index >= 0; index--)
                if (!fileHeaders.Contains(fieldHeaders[index]))
                    engine.Options.RemoveField(fieldHeaders[index]);
        }

        headerFields = engine.GetFileHeader();
        Debug.Assert(headerFile == headerFields);

        var records2 = engine.ReadFile(tempFile);

        // Test fixed string against column as column could be null and Debug.Assert can't use .Equals on a null object!
        column = records2[0].C;
        Debug.Assert("TestC1".Equals(column), "Test 2 - Column C does not equal 'TestC1'");  //Fails, returns ""

        // Test fixed string against column as column could be null and Debug.Assert can't use .Equals on a null object!
        column = records2[0].B;
        Debug.Assert(!"TestC1".Equals(column), "Test 2 - Column B does equal 'TestC1'");  //True, but that was not what I wanted

        Console.WriteLine("Seems to be OK now!");
        Console.ReadLine();

Note: One important thing I found is that in the current version 3.2.5, removing a field after already reading the first line of a file will cause the engine to blow a fuse!

I also added a IgnoreFirst() attribute to your class so that it skips the header row and sets the text that is ignored into engine.HeaderText. This results in the following class:

    [DelimitedRecord(";")]
    [IgnoreEmptyLines]
    [IgnoreFirst()]
    public class TestRecord
    {
        //Mandatory
        [FieldNotEmpty]
        public string A;

        [FieldOptional]
        public string B;

        [FieldOptional]
        public string C;
    }

Upvotes: 1

Related Questions