How to know when a buffer flush is needed?

Question

Considering two functions that flush buffers:

1. fflush()
2. sync()

How can I know when a call to either one is needed?

I know adding a '\n' to printf() will flush the output buffer, but if the string contains no such character, when can I skip this call, and when not (multi-threaded systems?)?

Same goes for sync. I have a function that saves files into the file system (the saving is done via a series of system calls), and it seems that without a call to sync the files are not saved in a specific case

Unfortunately I don't have all of the details on that case currently [What I do know is that the files are saved and a power off occurs right after that (don't know exactly how soon) and the files are not there after reboot]. On all the tests I run, the files were saved correctly.

So, how can I figure out when the system will flush the file data/metadata buffers and when it will not and I'm needed to explicitly call sync()?

Quoting man (which does not specify when an explicit call is needed):

sync, syncfs - commit buffer cache to disk
sync() causes all buffered modifications to file metadata and data to be written to the underlying file systems.

fflush - flush a stream
For  output  streams,  fflush() forces a write of all user-space buffered data for the given output or update stream via the stream's underlying write function.  For input streams, fflush() discards any buffered data that has been fetched from the underlying file, but has not been consumed by the application.  The open status  of the stream is unaffected.

A side question:

The 1st comment on this answer shows a way to disable stdout buffering by using setbuf(stdout, NULL);. Is there something like for sync (what buffer is used for that?)?

Answer 1

sync()

You seldom need a call to sync(). When you do, you have something that it is crucial must be recorded on disk ASAP. However, sync() will return having scheduled the writing of buffers in kernel memory, not after they've been written, so you won't know that they've actually been written — so it isn't wholly reliable. If you need more control over the writing for your file, look at the O_SYNC, O_DSYNC, O_RSYNC flags for open(). You will probably have to use fcntl() and fileno() to set these flags if you use file streams rather than file descriptors.

Two caveats:

1. sync() won't write buffers from your process (or any other process) to the kernel buffer pool; it is wholly unrelated to fflush().
2. sync() affects all data written by all processes on the system — you can become unpopular if your application uses it very often; it subverts the good work the kernel does caching data.

fflush()

The fflush() function ensures that data has been written to the kernel buffer pools from your application's buffer (either for a single file, or for all output files if you use fflush(0) or fflush(NULL)). It doesn't directly affect other processes. Again, you use this when you need to be confident that pending output has been sent to the kernel for onwards transmission. One place where you might use it is before an input operation where you want the prompt to appear, even if it hasn't got a newline at the end. Otherwise, you don't often use it, but you can use it whenever you want to be sure data has been sent to the kernel for writing. If you're debugging and your program crashes, a sprinkling of fflush() statements can ensure that pending output is written before the crash. This can help reveal more accurately where the problem is (but so can using a debugger).

Note that setting unbuffered output (setbuf(stdout, NULL) or setvbuf(stdout, NULL, _IONBF, 0)) means all output occurs 'immediately'. This is not necessarily good for performance. You use it sometimes, but only fairly rarely.