Unexpected amount of lines when writing to a csv file

Question

A part of my application writes data to a .csv file in the following way:

public class ExampleWriter {

    public static final int COUNT = 10_000;
    public static final String FILE = "test.csv";

    public static void main(String[] args) throws Exception {
        try (OutputStream os = new FileOutputStream(FILE)){         
            os.write(239);
            os.write(187);
            os.write(191);
            BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, StandardCharsets.UTF_8));         
            for (int i = 0; i < COUNT; i++) {               
                writer.write(Integer.toString(i));
                writer.newLine();               
            }           
        } catch (IOException e) {                       
            e.printStackTrace();
        }

        System.out.println(checkLineCount(COUNT, new File(FILE)));
    }

    public static String checkLineCount(int expectedLineCount, File file) throws Exception {
        BufferedReader expectedReader = new BufferedReader(new FileReader(file));
        try {
            int lineCount = 0;
            while (expectedReader.readLine() != null) {
                lineCount++;
            }
            if (expectedLineCount == lineCount) {
                return "correct";       
            } else {
                return "incorrect"; 
            }
        }
        finally {
            expectedReader.close();
        }
    }
}

The file will be opened in excel and all kind of languages are present in the data. The os.write parts are for prefixing the file with a byte order mark as to enable all kinds of characters.

Somehow the amount of lines in the file do not match the count in the loop and I can not figure out how. Any help on what I am doing wrong here would be greatly appreciated.

score 3 · Answer 1 · answered Apr 16 '17 at 14:42

3

You simply need to flush and close your output stream (forcing fsync) before opening the file for input and counting. Try adding:

writer.flush();
writer.close();

inside your try-block. after the for-loop in the main method.

answered Apr 16 '17 at 14:42

Johan Thomsen

174
4

Obviously, if you want to reuse your writer after calling _checkLineCount_, you don't have to close the stream. Invoking _flush()_ is enough to perform fsync. – Johan Thomsen Apr 16 '17 at 14:49

score 2 · Answer 2 · answered Apr 16 '17 at 14:57

(As a side note).

Note that using a BOM is optional, and (in many cases) reduces the portability of your files (because not all consuming app's are able to handle it well). It does not guarantee that the file has the advertised character encoding. So i would recommend to remove the BOM. When using Excel, just select the file and and choose UTF-8 as encoding.

score 1 · Accepted Answer · answered Apr 16 '17 at 14:51

You are not flushing the stream,Refer oracle docs for more info which says that

Flushes this output stream and forces any buffered output bytes to be written out. The general contract of flush is that calling it is an indication that, if any bytes previously written have been buffered by the implementation of the output stream, such bytes should immediately be written to their intended destination. If the intended destination of this stream is an abstraction provided by the underlying operating system, for example a file, then flushing the stream guarantees only that bytes previously written to the stream are passed to the operating system for writing; it does not guarantee that they are actually written to a physical device such as a disk drive.

The flush method of OutputStream does nothing.

You need to flush as well as close the stream. There are 2 ways

manually call close() and flush().
use try with resource

As I can see from your code that you have already implemented try with resource and also BufferedReader class also implements Closeable, Flushable so use code as per below

public static void main(String[] args) throws Exception {
        try (OutputStream os = new FileOutputStream(FILE); BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, StandardCharsets.UTF_8))){         
            os.write(239);
            os.write(187);
            os.write(191);

            for (int i = 0; i < COUNT; i++) {               
                writer.write(Integer.toString(i));
                writer.newLine();               
            }          
        } catch (IOException e) {                       
            e.printStackTrace();
        }

        System.out.println(checkLineCount(COUNT, new File(FILE)));
    }

I have chosen to implement your version as it seems more elegant by using the try-with-resources nicely. — Sander_M, Apr 16 '17 at 14:55

score 0 · Answer 4 · answered Apr 16 '17 at 14:38

When COUNT is 1, the code in main() will write a file with two lines, a line with data plus an empty line afterwards. Then you call checkLineCount(COUNT, file) expecting that it will return 1 but it returns 2 because the file has actually two lines. Therefore if you want the counter to match you must not write a new line after the last line.

score 0 · Answer 5 · answered Apr 16 '17 at 15:04

0

(As another side note).

Notice that writing CSV-files the way you are doing is really bad practice. CSV is not so easy as it may look at first sight! So, unless you really know what you are doing (so being aware of all CSV quirks), use a library!

answered Apr 16 '17 at 15:04

rmuller

10,058
3
50
80

Unexpected amount of lines when writing to a csv file

5 Answers5