CODEWITHSUNDEEP
javacharacter-encodingopencsv
roizpi
roizpi

Reputation: 3648

Different encodings every time I append to a CSV file

I am appending text encoded in Windows-1252 into to a CSV in this way:

public static final Charset CHARSET = Charset.forName("Windows-1252");


public void dumpToCSV(final List<String[]> content, 
                      final char delimiter, 
                      final String enc,
                      final int csvDays) {

    File file = new File(Constants.CSV_FILENAME);

    // Convert the Character Format before dumping to file:
    try (
        OutputStreamWriter os = new OutputStreamWriter(
            new FileOutputStream(file, true),  
            CHARSET);
        CSVWriter cw = new CSVWriter(os, delimiter)) {

        // Remove old lines
        clearCsvByDays(file, csvDays, Character.toString(delimiter));
        // Dump new content into file.  
        cw.writeAll(content);
  } catch (IOException e) {}
}

private void clearCsvByDays(final File file, final int csvDays, final String delim) 
             throws IOException {

    List<String> out = Files.lines(file.toPath(), CHARSET)
                            .filter(line -> mustFilter(line, csvDays, delim))
                            .collect(Collectors.toList());
    Files.write(file.toPath(), out, 
                StandardOpenOption.WRITE, 
                StandardOpenOption.TRUNCATE_EXISTING);
}

The first writing to the file, the result is as expected, the characters are Windows-1252 encoded and are shown well on the target program .

"ʳpⲠ t촴";"ADN";"26-09-2017";"0";"0";"0";"0"     <-- This result is fine.

The second dump, it appends the new data on UTF-8, I don't know why.

"Éspáñà tëst";"ADN";"26-09-2017";"0";"0";"0";"0" <-- 2nd dump (new)
"ʳpⲠ t촴";"ADN";"26-09-2017";"0";"0";"0";"0"     <-- 1st dump (old)

The third dump, it appends the new data on another different encoding, but keeps the first correct line dumped on Windows-1252.

"Ãspáñà tëst";"ADN";"26-09-2017";"0";"0";"0";"0"   <-- 3rd dump (new)
"Éspáñà tëst";"ADN";"26-09-2017";"0";"0";"0";"0"       <-- 2nd dump (old)
"ʳpⲠ t촴";"ADN";"26-09-2017";"0";"0";"0";"0"           <-- 1st dump (old)

If I keep appending, each time it is a different encoding.

Why is this happening and how can I fix it?

Upvotes: 0

Views: 239

Answers (1)

Joop Eggen
Joop Eggen

Reputation: 109547

CSVWriter has been given a correct OutputStreamWriter.

And on writing, Files.write needs an encoding too.

Files.write(file.toPath(), out, CHARSET,
    StandardOpenOption.WRITE, StandardOpenOption.TRUNCATE_EXISTING);

So I suspect hacks elsewhere:

new String(string.getBytes(...), ...)

Upvotes: 3

Related Questions