C# CsvHelper - Records with different number of properties

Question

Im using CsvHelper on a legacy project (version 7.1.0 , I'd rather keep that version for now if possible)

I have a list of objects with different number of properties that I want to save as a csv.

For example:

var list = new List<dynamic>(){
  {id=1, image1="xxx", state="ok"},
  {id=2, image1="xxx", image2=yyy, state="ok"}
  {id=3, image1="xxx", state="ok"}
}

---

edit for clarification: Im receiving the objects as part of a publicly shipped API A client can decide to send me

{
  {id=1, image1="xxx", state="ok"},
  {id=2, image1="xxx", image2="yyy", state="ok"}
  {id=3, image1="xxx", state="ok"}
}

while another one could send

{
  {id=1, MyImage1="xxx", state="ok"},
  {id=2, MyImage1="xxx", MyImage2="yyy", state="ok"}
  {id=3, MyImage1="xxx", MyComment="please discard", state="ok"}
}

Which is why I (at least I think) need to use some sort of dynamic or expandoObject

Easiest case is where all records have the same properties, but Im having issues with a specific edge case

---

When I write:

var csv = new CsvWriter(writer);
csv.WriteRecords(list);

The headers are based on the first item in the list, so in my case, the csv will be messed up as some headers are missing, and I will get something like that:

id	image1	state
1	xxx		ok
2	xxx	yyy	ok
3	xxx		ok

What I would like is the following:

id	image1	image2	state
1	xxx		ok
2	xxx	yyy	ok
3	xxx		ok

I need to keep the order of the headers and the sorting of the list.

The best solution I have so far is to identify the item with the highest property count, and add the missing properties with a null value to all the other items ...

Is there a smarter way of achieving what I need?

Thanks!

Answer 1

The only way I can get it to work is to go through all of the records and find all of the potential headers. Then loop through that list of headers for each record to see if the record contains the header or not. If so, write out the value, if not, write an empty field.

void Main()
{
    var records = new List<dynamic>(){
      new {id=1, image1="xxx", state="ok"},
      new {id=2, image1="xxx", image2="yyy", state="ok"},
      new {id=3, image1="xxx", state="ok"}
    };
    
    var headers = new List<string>();
    
    foreach (var type in records.Select(r => r.GetType()).Distinct())
    {
        var properties = type.GetProperties();
        var previousItem = string.Empty;
        
        foreach (var property in properties)
        {
            var previousItemIndex = headers.FindIndex(a => a == previousItem);          
            var currentItemIndex = headers.FindIndex(a => a == property.Name);
            
            if (currentItemIndex == -1)
            {
                headers.Insert(previousItemIndex + 1, property.Name);
            }
            
            previousItem = property.Name;
        }
    }
    
    using (var csv = new CsvWriter(Console.Out))
    {
        foreach (var header in headers)
        {
            csv.WriteField(header);
        }
        
        csv.NextRecord();
        
        foreach (var record in records)
        {
            foreach (var header in headers)
            {
                var property = record.GetType().GetProperty(header);
                
                if (property != null)
                {
                    csv.WriteField(property.GetValue(record, null));
                }
                else
                {
                    csv.WriteField(string.Empty);
                }
            }
            csv.NextRecord();
        }
    }
}

You will likely need to change the logic of how you get the order of the headers. That is going to depend a bit on what things you know about the incoming data. One possible solution to having name1, name2, name3 would be to find the last index of the previous item with any numbers removed.

var previousItemIndex = headers.FindLastIndex(a => a.Contains(string.Concat(previousItem.Where(char.IsLetter))));