Dissecting Values From FileHelpers? Or this out of the programs functionality

Question

I am wondering if I got to ditch filehelpers and do this myself as I think I might be going beyond what it was designed for.

I want a user to be able to upload any csv file(maybe in the future excel file). The first row would have the header

C1 C2 C3 C4 C5 C6

once upload it would look like

C1,C2,C3,C4,C5,C6 a,b,c,d,e,f

Now I want to look at the header and basically take certain ones. For instance I want C2, C3, C4. The rest are extra information I don't care about.

Now someone might upload a file that has this header

C1 C2 C3 C4

Again I am looking only for C2, C3, C4.

I know I can have multiple formats but what I am getting at is I want them to be able to basically upload any file with any number of headers(could be 1000 for all I care) and then have my application try to find the information I care about(so in the case of the 1000 headers I maybe only want 3)

Is this possible?

Edit

(based on shamp00 comments)

My goal is to fill in as much data as I possibly can determine, however cases like this might happen. I want C1, C2, C3. They give a file with C1,C3,C4. I got 2 columns of data I need but I don't have C2.

Now I had 2 ideas one was to display the data into 2 tables. Table 1 would have C1, C2, C3 and table 2 would have C1,C3,C4 and they basically take the data they have in table 2 and move the appropriate data into my expected columns.

With this approach I am basically saying "you did not give me 100% what I expected, now you have to format every single row into my format".

The second approach would be 1 table and try to fill in as much data as possible.

For example the user upload the file that has C1,C3,C4. I determine that their are 2 columns that are known but I don't have the full amount of expected data yet.

So I would display all the rows back in a html table to the user with headers of

C1, C2, C3, C4

C1 would be filled in, C2 cells would be blank (as this is the data I am missing from them), C3 would be filled in, C4 would be filled in with (this data was unexpected but who knows it might actually be the data C2 should hold but since they misspelled the header name my program could not figure it out).

Then essentially they would just fill in C2 with data they got from somewhere else or maybe from C4.

Now they only have to fill in 1 column in instead of all the columns that where expect. So in a sense I need a concrete class like MyClass was with C1,C2,C3 but at the same time I need to dynamic so I can hold C4,C5.....Cn.

I would always display C1,C2,C3 first and the rest of these unexpected ones would come after and through the magic of javascript and stuff they could edit the missing info. If nothing is missing they nothing would show up to be edited.

Based on shamp00 comments I am now wondering if I need to return the data as a Data Table(fortunately this seems to be a system class as right now my code is in a service layer and I was return back a Domain Transfer Class as I want to keep my code independent from like web code classes and hence why I was trying to figure out how to the dynamic class FileHelpers generated.).

Then somehow (not 100% sure yet) just keep track where those 3 columns I am really interested are so I know which data is what.

Answer 1

I am not familiar with FileHelpers, but I have done something very similar to what you describe by using a tool called LogParser (http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=24659) in conjunction with my own "DelimitedTextFileData" class. If you decide FileHelpers isn't going to do what you need, I would recommend you look into LogParser next. Even if it's overkill for your current project, it IS an excellent tool to be aware of for future projects.

LogParser is a tool that allows "SQL like" queries against a wide variety of sources - including CSV text files. It is a command line based .exe, but also comes with an API you can reference in your .NET project. In my situation, I was dealing with text files that could be delimited by any character, so I developed my own class to let me specify the delimiter on class instantiation and then use a simple API to tap the larger LogParser API. I also had to parse files with an unknown number of (and name for) columns, so my custom class has a function to retrieve a list of columns found in the file. You may not need to take these extra steps if you're always dealing with a CSV, and you know exactly what columns you want. Nevertheless, I'd be happy to share my custom class if you'd like; just let me know the best way to send it.

LogParser is meant to let you "query anything using SQL-like syntax", and it occurred to me that one purpose of Linq is to do the same. Have you searched online for any "Linq to Text File" libraries? If there is a good one out there, it may solve your problem as well.

Answer 2

You can use FileHelpers using a technique like the one described in my answer to your other question.

You read the header line to determine which columns are relevant and then traverse the resulting DataTable processing only those columns.

Something like

public class MyClass
{
    public string SomeImportantField { get; set; }
    public string SomeOtherField { get; set; }
    public string AnotherField { get; set; }
}

public IList<MyClass> GetObjectsFromStream(Stream stream)
{
    var cb = new DelimitedClassBuilder("temp", ",") { IgnoreFirstLines = 1, IgnoreEmptyLines = true, Delimiter = "," };
    var sr = new StreamReader(stream);
    var headerArray = sr.ReadLine().Split(',');
    foreach (var header in headerArray)
    {
        var fieldName = header.Replace("\"", "").Replace(" ", "");
        cb.AddField(fieldName, typeof(string));
    }

    var engine = new FileHelperEngine(cb.CreateRecordClass());

    List<MyClass> objects = new List<MyClass>();
    DataTable dt = engine.ReadStreamAsDT(sr);
    foreach (DataRow row in dt.Rows) // Loop over the rows.
    {
        MyClass myClass = new MyClass();
        for (int i = 0; i < row.ItemArray.Length; i++) // Loop over the items.
        {
            if (headerArray[i] == "ImportantField")
                myClass.SomeImportantField = row.ItemArray[i].ToString();
            if (headerArray[i] == "OtherField")
                myClass.SomeOtherField = row.ItemArray[i].ToString();
            if (headerArray[i] == "AnotherField")
                myClass.AnotherField = row.ItemArray[i].ToString();
            objects.Add(myClass);
        }
    }
    return objects;
}