1

I am trying to manipulate/strip the output of 7zip command and intimate user about the progress of process. The sample code i am trying to use is below:

import subprocess
proc = subprocess.Popen(['7zip','arg', 'archive'],shell=True, stdin=None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while True:
  line = proc.stdout.readline()
  if line != '':
    #Do some striping and update pyqt label
    print "test:", line.rstrip()
    sys.stdout.flush()
  else:
    break

However, the real problem is that print statement only print stdout after completion of the process. Is there a way to capture the stdout line by line then manipulate and print again?

Update Updated the script to include sys.stdout.flush()

sundar_ima
  • 2,842
  • 3
  • 29
  • 46
  • The problem is that program output is handled differently depending on whether the output is going to a terminal (assume human reader and flush often) or another program (assume no human and delay output for more efficient pipelining). In linux you fix this by opening a pty (see pexpect for instance) but I don't know of any good solution for windows. – tdelaney Jan 31 '14 at 19:47
  • Am I correct in understanding that your issue is that you're only getting `stdout` after the process completes, but you'd rather stream `stdout` in real time? – ernie Jan 31 '14 at 19:49
  • If you are just trying to fix the print in python, do `sys.stdout.flush()` after each print. – tdelaney Jan 31 '14 at 19:50
  • Yes. Idid add `sys.stdout.flush()` after print statement. But the result is same. It print the stdout as a whole content. – sundar_ima Jan 31 '14 at 19:55
  • @J.F.Sebastian provides a `pty`-based (Linux-only) [solution here](http://stackoverflow.com/a/12471855/190597). – unutbu Jan 31 '14 at 20:02
  • Yeah, the delay is in 7zip (most console based apps work the same way). Easily fixed in linux and I've heard that you can use pseudo terminals in Cygwin, but I don't know of a generic windows solution. – tdelaney Jan 31 '14 at 20:04
  • Possible duplicate: [Python: read streaming input from subprocess.communicate()](http://stackoverflow.com/questions/2715847/python-read-streaming-input-from-subprocess-communicate/17698359#17698359). – ernie Jan 31 '14 at 20:04
  • I have already read different thread on the same subject but there is no solution for reading the stdout. – sundar_ima Jan 31 '14 at 20:32
  • Breaking my head to solve this issue. But it is very clear that `proc.stdout.readline()` is blocking the code to complete. Not sure if some one has already solved this issue. – sundar_ima Jan 31 '14 at 21:30
  • Again I can confirm that above code perfectly works when I replace the `proc = subprocess.Popen(['7zip','arg', 'archive'],shell=True, stdin=None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)` with this system command `dir` as `subprocess.Popen("dir", shell=True, stdin=None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)` – sundar_ima Jan 31 '14 at 21:54

1 Answers1

0

Yes, the popen family of calls.

You can see the documentation here 

(child_stdin,
 child_stdout,
 child_stderr) = os.popen3("cmd", mode, bufsize)
==>
p = Popen("cmd", shell=True, bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
(child_stdin,
 child_stdout,
 child_stderr) = (p.stdin, p.stdout, p.stderr)

they give you filedescriptors to the streams and you can use them to read the output of the called program.

Jörg Beyer
  • 3,447
  • 19
  • 35
  • I don't think this solves his problem. He already use Popen. The problem is that you can't get a pseudo-terminal on Windows. – tdelaney Jan 31 '14 at 19:48