I'm working on a Python script that needs to create about 50 distinct temporary files, which are all appended frequently during the course of the script and merged at the end. I'm sure that the tempfile
module can do what I need, but I haven't been able to figure out how from reading the documentation.
I want to use temporary files--as opposed to variables--to conserve system memory, as these data chunks grow large as the script processes tens of thousands of other files.
The following chunk of code is the hack I'm currently using to create these files (untemporarily) in an untemporary directory:
item = (string from another file) # string must id file for future use
tmpfile = 'tmpfiles/' + item
if item not in totalitems:
totalitems.add(item)
with open(tmpfile, 'w') as itemfile:
output = some stuff
tmpfile.write(output)
else:
with open(tmpfile, 'a') as itemfile:
output = different stuff
tmpfile.write(output)
I think what I need is tempfile.NamedTemporaryFile()
. According to the documentation:
That name can be retrieved from the name member of the file object.
Unfortunately, I don't understand what that means. I just need to be able to call each file again later when I run across its corresponding "item" again in the files I'm processing. I presume this is rather straight forward and I'm just being dense. In case it matters, I have versions of this script for both Python 2.7.1 and 3.2.3. I only really need for one or the other to work; I created both just as a learning exercise.