1

I need to force OS to purge the pages used for a mapped file. I don't have the file descriptor, so posix_fadvise cannot be used.

Our application caches a lot of files by mapping them into memory. After the file has been mapped (i.e. we've got the pointer from mmap()), we close the file. When at some later point we have to clean the cache, we want to purge the pages in OS cache as well. That is, we want to unmap the file, and do something like posix_fadvise(POSIX_FADV_DONTNEED), but the file descriptor is not available at this point.

The flow looks like this:

//caching stage
fd = open("file");
data = mmap(fd, <mmap flags>);
close(fd);

//clean-up stage
munmap(data);
// posix_fadvise(???, POSIX_FADV_DONTNEED);

Is there a way to clear the cached pages without file descriptor?

I have thought about following two workarounds:

  • Keeping the files open, so that I have valid descriptors at the time of cleanup. However, there may be tens of thousands files, and keeping them all open may affect OS performance.
  • Keep the file path, and reopen it just to get a descriptor and call posix_fadvise(). But the question is: will the old mapped area be associated with the same file? And will fadvise() purge the cached pages in this scenario?
Vladislav Ivanishin
  • 1,872
  • 13
  • 20
me76
  • 79
  • 7
  • The cache is associated with the file, not a particular mapping, so I think reopening the file should work. – Barmar May 03 '19 at 19:38
  • I just found this in man page for munmap(): closing the file descriptor does not unmap the region. It looks like the region is associated with the file, and will be associated with new fd when the file is opened next time. I will test it and will post the results. – me76 May 03 '19 at 20:07
  • 2
    You already knew that closing the descriptor doesn't unmap the file, since your application keeps using the mapping after closing. – Barmar May 03 '19 at 20:15
  • I misinterpreted that part – I thought the pages would just lose the link with the file and become data in memory. – me76 May 04 '19 at 00:24
  • Most Linux systems use a unified file cache. – Barmar May 04 '19 at 00:33

1 Answers1

0

The second option worked. When the file is reopened later, the mapped area is associated with it, and calling posix_fadvise with new file descriptor unloads the mapped pages:

//caching stage
fd = open("file");
data = mmap(fd, <mmap flags>);
close(fd);

//clean-up stage
fd = open("file");
munmap(data);
posix_fadvise(fd, POSIX_FADV_DONTNEED);
close(fd);
me76
  • 79
  • 7