3

I am attempting to follow this answer here: https://stackoverflow.com/a/5087695/343381

I have a need to execute multiple bash commands within a single environment. My test case is simple:

import subprocess
cmd = subprocess.Popen(['bash'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)

# Write the first command
command = "export greeting=hello\n"
cmd.stdin.write(command)
cmd.stdin.flush() # Must include this to ensure data is passed to child process
result = cmd.stdout.read()
print result

# Write the second command
command = "echo $greeting world\n"
cmd.stdin.write(command)
cmd.stdin.flush() # Must include this to ensure data is passed to child process
result = cmd.stdout.read()
print result

What I expected to happen (based on the referenced answer) is that I see "hello world" printed. What actually happens is that it hangs on the first cmd.stdout.read(), and never returns.

Can anyone explain why cmd.stdout.read() never returns?

Notes:

  • It is absolutely essential that I run multiple bash commands from python within the same environment. Thus, subprocess.communicate() does not help because it waits for the process to terminate.
  • Note that in my real test case, it is not a static list of bash commands to execute. The logic is more dynamic. I don't have the option of running all of them at once.
Community
  • 1
  • 1
Adam S
  • 8,245
  • 17
  • 60
  • 95
  • 1
    Ah, see it now; I suspect you need to poll the pipes to see if there is any content at all. You could also try to just push the second command `stdin` pipe without reading from `stdout`. – Martijn Pieters Mar 20 '13 at 21:35

1 Answers1

2

You have two problems here:

  1. Your first command does not produce any output. So the first read blocks waiting for some.
  2. You are using read() instead of readline() -- read() will block until enough data is available.

The following modified code (updated with Martjin's polling suggestion) works fine:

import subprocess
import select

cmd = subprocess.Popen(['bash'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)

poll = select.poll()
poll.register(cmd.stdout.fileno(),select.POLLIN)

# Write the first command
command = "export greeting=hello\n"
cmd.stdin.write(command)
cmd.stdin.flush() # Must include this to ensure data is passed to child process
ready = poll.poll(500)
if ready:
   result = cmd.stdout.readline()
   print result

# Write the second command
command = "echo $greeting world\n"
cmd.stdin.write(command)
cmd.stdin.flush() # Must include this to ensure data is passed to child process
ready = poll.poll(500)
if ready:
   result = cmd.stdout.readline()
   print result

The above has a 500ms timeout - adjust to your needs.

isedev
  • 16,514
  • 3
  • 51
  • 58
  • @MartjinPieters - Apologies (just noticed your comment). Was going to delete, but the read() vs readline() is important. – isedev Mar 20 '13 at 21:58
  • Thank you for including the polling code. For someone who has never had to use that function, it was not obvious how to poll stdout for new data. – Adam S Mar 20 '13 at 22:46
  • Doesnt work for me on Python 2.7.10 gives AttributeError: 'module' object has no attribute 'poll' – avimehenwal Mar 15 '16 at 09:35