3

I need to iterate over the lines of a couple of text files a couple of times. This is currently done with multiple

with open("file.txt") as f: 
    for line in f:
        # do something

Although performance is not an issue yet, I'd like to read the files only once into an io.StringIO buffer and then work with that.

Python io docs:

This is a working snippet

import io
sio = io.StringIO( open("file.txt").read() )
for line in sio:
    print(line)
sio.seek(0)
for line in sio:
    print(line)
sio.close()

or wrapping it in a with statement context manager

import io
with io.StringIO( open("file.txt").read() ) as sio:
    for line in sio:
        print(line)
    sio.seek(0)
    for line in sio:
        print(line)
    #sio.close()

Questions

  1. Is this a "good" way of doing it, what are alternatives?
  2. What happens to the file object used to read the file (there's no way to explicitly close() it this way)?
  3. Where can I read more about Python's io buffering (I think I read something about Python optimizing multiple file accesses by buffering automatically)?
handle
  • 5,063
  • 2
  • 40
  • 69

1 Answers1

1

What you're doing is already the right way. Quoting from this answer: How to read large file, line by line in python

The with statement handles opening and closing the file, including if an exception is raised in the inner block. The for line in f treats the file object f as an iterable, which automatically uses buffered IO and memory management so you don't have to worry about large files.

Community
  • 1
  • 1
Dwaxe
  • 109
  • 1
  • 6
  • Yes, but my questions are about processing a file _multiple_ times. Should it be `open()`ed multiple times? Also there are no references about the buffering. – handle May 17 '17 at 14:57