57

How can I call an external program with a python script and retrieve the output and return code?

cfischer
  • 23,024
  • 36
  • 125
  • 209
  • 2
    There are some existing questions and answers on SO that will help you: http://stackoverflow.com/questions/89228/how-to-call-external-command-in-python – Jarret Hardie Apr 01 '09 at 19:27

5 Answers5

75

Look at the subprocess module: a simple example follows...

from subprocess import Popen, PIPE

process = Popen(["ls", "-la", "."], stdout=PIPE)
(output, err) = process.communicate()
exit_code = process.wait()
Victor Uriarte
  • 439
  • 4
  • 9
jkp
  • 70,446
  • 25
  • 98
  • 102
  • 10
    I've edited the answer above to reflect Ambroz's suggestion in case someone does not read the comments and uses the previously incorrect code. – Ben McCann Oct 24 '13 at 17:30
  • if this does not for some reason work, you might want to add shell=True to the params ( when in windows? ) – ntg Oct 31 '16 at 10:05
  • It seems that [the above solution](https://stackoverflow.com/a/707001/1321680) can be replaced by simple call [`subprocess.run()`](https://docs.python.org/dev/library/subprocess.html#subprocess.run) (Python >= 3.5 is required). – patryk.beza Jun 24 '17 at 12:56
  • 1
    can I get the line number of error line like a line has error and the function returns me the line number like 1.0? so that I can add a tag and highlight the error. – Kumar Saptam May 15 '19 at 16:19
  • In order to grab the error outputs "stderr=PIPE" should also be added: `process = Popen(["ls", "-la", "."], stdout=PIPE, stderr=PIPE)` – HmT May 31 '19 at 07:18
18

Following Ambroz Bizjak's previous comment, here is a solution that worked for me:

import shlex
from subprocess import Popen, PIPE

cmd = "..."
process = Popen(shlex.split(cmd), stdout=PIPE)
process.communicate()
exit_code = process.wait()
Jabba
  • 15,755
  • 4
  • 43
  • 41
  • 5
    This is by far the best answer. – dotancohen Aug 22 '13 at 09:43
  • 3
    I have a similar post [here](https://stackoverflow.com/questions/1996518/retrieving-the-output-of-subprocess-call/21000308#21000308) that shows how to get three things from a process: exitcode, stdout, stderr. – Jabba Mar 18 '14 at 20:58
4

After some research, I have the following code which works very well for me. It basically prints both stdout and stderr in real time. Hope it helps someone else who needs it.

stdout_result = 1
stderr_result = 1


def stdout_thread(pipe):
    global stdout_result
    while True:
        out = pipe.stdout.read(1)
        stdout_result = pipe.poll()
        if out == '' and stdout_result is not None:
            break

        if out != '':
            sys.stdout.write(out)
            sys.stdout.flush()


def stderr_thread(pipe):
    global stderr_result
    while True:
        err = pipe.stderr.read(1)
        stderr_result = pipe.poll()
        if err == '' and stderr_result is not None:
            break

        if err != '':
            sys.stdout.write(err)
            sys.stdout.flush()


def exec_command(command, cwd=None):
    if cwd is not None:
        print '[' + ' '.join(command) + '] in ' + cwd
    else:
        print '[' + ' '.join(command) + ']'

    p = subprocess.Popen(
        command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=cwd
    )

    out_thread = threading.Thread(name='stdout_thread', target=stdout_thread, args=(p,))
    err_thread = threading.Thread(name='stderr_thread', target=stderr_thread, args=(p,))

    err_thread.start()
    out_thread.start()

    out_thread.join()
    err_thread.join()

    return stdout_result + stderr_result
Jake W
  • 2,559
  • 31
  • 36
3

Check out the subprocess module here: http://docs.python.org/library/subprocess.html#module-subprocess. It should get what you need done.

David Ackerman
  • 11,809
  • 6
  • 21
  • 19
2

I've developed a little library (py-execute) that allows you to execute external programs, retrieve the output and the retcode and, at the same time get output in console in real time:

>>> from py_execute.process_executor import execute
>>> ret = execute('echo "Hello"')
Hello
>>> ret
(0, 'Hello\n')

You can avoid printing to console passing a mock user_io:

>>> from mock import Mock
>>> execute('echo "Hello"', ui=Mock())
(0, 'Hello\n')

I wrote it because with plain Popen (In Python 2.7) I was having trouble executing commands with a long output

hithwen
  • 2,057
  • 25
  • 42