4

My code is as follows, basically this module will run the required command and capture its ouput line by line but in my case when the command runs, it takes just more than a second to return to the command prompt and thats where child.stdout.read(1) hangs, if I run a normal command using this it prints everthing as expected. but in a particular case where, the command prints somthing to STDOUT and then takes some time to return to the prompt, it hangs.. Please help

New code:

def run_command(shell_command):
'''run the required command and print the log'''
child = subprocess.Popen(shell_command, shell=True,stdout=subprocess.PIPE)
(stdoutdata, stderrdata) = child.communicate()
print stdoutdata

print "Exiting.."

Error:

  File "upgrade_cloud.py", line 62, in <module>
stop_cloud()
File "upgrade_cloud.py", line 49, in stop_cloud
run_command(shell_command)
 File "upgrade_cloud.py", line 33, in run_command
 (stdoutdata, stderrdata) = child.communicate()
 File "/usr/lib/python2.6/subprocess.py", line 693, in communicate
stdout = self.stdout.read()
KeyboardInterrupt
rohit vyawahare
  • 85
  • 7
  • Hm... I suspect this may be [issue 7213](http://bugs.python.org/issue7213). Do you have multiple Popen instances running at the same time? – Kevin Dec 16 '14 at 16:16
  • No, at a time it just runs a single command and comes out. – rohit vyawahare Dec 16 '14 at 16:19
  • Please help, I am stuck on this issue – rohit vyawahare Dec 16 '14 at 16:25
  • The only other thing I can think of is that the actual command itself is hanging. If that's the case, there's little Python can do. – Kevin Dec 16 '14 at 16:27
  • No, thats not the case, I am able to run the command manually – rohit vyawahare Dec 16 '14 at 16:28
  • Does the command prompt for input? To debug, you could temporarily remove `stdout=subprocess.PIPE` and see what's printed to the screen. – tdelaney Dec 16 '14 at 17:29
  • related: [Python: read streaming input from subprocess.communicate()](http://stackoverflow.com/q/2715847/4279) – jfs Dec 16 '14 at 17:59
  • *"I am able to run the command manually"*. What happens if you copy-paste the command into `subprocess.check_call(shell_command, shell=True)` (without redirecting)? – jfs Dec 16 '14 at 18:01

1 Answers1

3

Here's your problem:

child.wait()

This line causes Python to wait for the child process to exit. If the child process tries to print a lot of data to stdout, it will block waiting for Python to read said data. Since Python is waiting for the child process and the child process is waiting for Python, you get a deadlock.

I would recommend using subprocess.check_output() instead of subprocess.Popen. You could also use the Popen.communicate() method instead of the .wait() method.

Kevin
  • 25,251
  • 7
  • 54
  • 78
  • Python 2.6.5 (r265:79063, Feb 27 2014, 19:43:51) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. Version is 2.6, it does not have check_output and the thing is i cant upgrade the environment to latest python – rohit vyawahare Dec 16 '14 at 15:43
  • It still hangs, File "upgrade_cloud.py", line 14, in run_command child.communicate() File "/usr/lib/python2.6/subprocess.py", line 693, in communicate stdout = self.stdout.read() KeyboardInterrupt .. I had to use the keyboard interrupt – rohit vyawahare Dec 16 '14 at 15:49
  • Once you've done a `communicate()`, `child.stdout` is a useless object. You need to save the return value from `communicate()` and work with that. – Kevin Dec 16 '14 at 15:55
  • `child = subprocess.Popen(shell_command, shell=True,stdout=subprocess.PIPE) (stdoutdata, stderrdata) = child.communicate() print stdoutdata` So ideally this should print all the lines of the stdoutput, right?, but it is not able to do so – rohit vyawahare Dec 16 '14 at 16:00
  • Communicate is definitely the way to go. Reading one character at a time is highly inefficient. – Dunes Dec 16 '14 at 16:00
  • With the code, in my latest above comment, ideally i should be able to print the data, but i am not able to, however for the commands that doesnt wait, it works fine – rohit vyawahare Dec 16 '14 at 16:02
  • @rohit: Please post the full error you are getting in the question so we can look at it. – Kevin Dec 16 '14 at 16:05