2007-11-12

atomic file writing

It still shocks me when I see code that is not writing files atomically.
Most people still think that it is OK when changing something in a text file to do something like:
file = open("filename.txt", O_RDONLY);
read(file,...
close(file)
now change whatever in memory and write the file back:
file = open("filename.txt", O_RDWR|O_CREAT|O_TRUNC|O_LARGEFILE, 0666)
write(file,...
close(file)

This is the correct way to save the file back, also known as atomic save:
file = open("filename.txt.tmp", O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE, 0666)
if(write(file,...
if(fsync(file) /* this is needed as Linux can reorder operations on files */
if(close(file)
if(rename("file.txt.tmp", "file.txt") /* commit */

Remember you have to check every write() and close() and the rename() for errors and rollback with unlink("file.txt.tmp") in case of any error.

This is not something some newbies do, as this problem is in software like Mozilla Firefox with critical bugs like:
The main problem with not doing atomic file writes is that any problem while writing the file can leave the old one corrupted and that means data loss (one of the most common problems is full non-volatile memory)

Good implementations of atomic file writing are in Pidgin
purple_util_write_data_to_file_absolute() (that I helped to debug) and glib g_file_set_contents(). But there are still lots of programs and libraries out there that are still not using atomic file writing.

There is still no agreement on the correct way to save a file atomically as noted by Benoît Dejean. Even the ext3 & ext4 developer (Ted Ts'o) does not do proper research on this.

No comments: