I am analyzing astronomical data with python. I use the astropy.io fits library to handle data.
Using the psutil library, I recently noticed that the library does not appear to properly close files once the data has been read in.
Consider this example, in which the data from an open fits file is read in to an hdulist:
from astropy.io import fits
sky_image = 'sky_2020_11_22_315.fits'
import psutil
proc = psutil.Process()
hdul = fits.open(sky_image)
print (proc.open_files())
hdul.close()
print (proc.open_files())
hdul = fits.open(sky_image)
print (proc.open_files())
header, table = [hdul[0].header, hdul[0].data]
print (proc.open_files())
hdul.close()
print (proc.open_files())
The output of this code is as follows:
[popenfile(path='sky_2020_11_22_315.fits', fd=4)]
[]
[popenfile(path='sky_2020_11_22_315.fits', fd=4)]
[popenfile(path='sky_2020_11_22_315.fits', fd=4), popenfile(path='sky_2020_11_22_315.fits', fd=5)]
[popenfile(path='sky_2020_11_22_315.fits', fd=5)]
In the first instance, the hdulist is opened and closed without issue. However, the second time, the file is apparently opened twice, once for the hdulist and once for the associated array. The issue I have is that, as I analyze data, I open and close several hundred fits images and Python fill sometimes crash, complaining about having too many files open:
OSError: [Errno 24] Too many open files
How can I close the additional opened file, once I am done with the read in data? Thank you!
