CODEWITHSUNDEEP
csvrustserde
galop1n
galop1n

Reputation: 8824

Deserializing a CSV with Serde when the header row may not have the last column name in it

I am using the csv and serde crates to deserialize csv files. The thing is that the last field is actually a comma separated list.

field1,field2,field3
xx, xx, str1, ..., strN
xx, xx,
xx, xx, str1, ..., strM

And this is how it maps in Rust, reading it with .flexible(true) on the reader:

#[derive(Debug, Deserialize)]
struct Row {
  field1: isize,
  field2: isize,
  field3: Vec<String>,
}

Everything works fine if the CSV has the ,field3 the header row. But some of the files does not have it, and i can't find a solution to have serde still fill the Vec. All I was able to do is #[serde(default)] that just let field3 empty.

Here a rust playground showing the problem: 

extern crate csv;
#[macro_use]
extern crate serde_derive;

#[derive(Debug, Deserialize)]
struct Row {
    field1: String,
    field2: String,
    #[serde(default)]
    field3: Vec<String>,
}

fn test(str: String) {
    let mut reader = csv::ReaderBuilder::new()
        .flexible(true)
        .from_reader(str.as_bytes());

    for row in reader.deserialize() {
        if let Ok(row) = row {
            let row: Row = row;
            println!("{:?}", row);
        }
    }
}

fn main() {
    let csv_data = "
field1,field2,field3
xx,yy,one,two,three
zz,ww,
aa,bb
cc,dd,foo,bar,ban
";
    println!("With full header");
    test(csv_data.to_string());

    let csv_alt_data = "
field1,field2
xx,yy,one,two,three
zz,ww,
aa,bb
cc,dd,foo,bar,ban
";
    println!("With incomplet header");
    test(csv_alt_data.to_string());
}

Upvotes: 5

Views: 3457

Answers (1)

Stargateur
Stargateur

Reputation: 26727

Add the field header by the hand before read rows should do it, using headers() and set_headers():

let rdr = reader.headers().unwrap();
if let None = rdr.get(2) {
    let mut rdr = rdr.clone();
    rdr.push_field("field3");
    reader.set_headers(rdr);
}

However, that quick and dirty.

Upvotes: 4

Related Questions