Executing command line programs from within python

Question

I'm building a web application that will is going to manipulate (pad, mix, merge etc) sound files and I've found that sox does exactly what I want. Sox is a linux command line program and I'm feeling a little uncomfortable with having the python web app starting new sox processes on my server on a per request basis.

Example:

import os
os.system('sox input.wav -b 24 output.aiff rate -v -L -b 90 48k')

This whole setup seems a little unstable to me.

So my question is, what's the best practice for running command line programs from within a python (or any scripting language) web app?

Message queues would be one thing to implement in order to get around the whole request response cycle. But is there other ways to make these things more elegant?

Related: http://stackoverflow.com/questions/89228/how-to-call-external-command-in-python, http://stackoverflow.com/questions/311601/python-as-a-batch-script-i-e-run-commands-from-python — S.Lott, Jan 16 '09 at 12:55
@S.Lott Not quite, as it is much more specific to the web server / web application context. — ThomasH, May 11 '11 at 16:48
@Thomas. I can't see how "web" changes anything in this case. Can you explain why "web" matters when running a subprocess? — S.Lott, May 11 '11 at 17:00
@S.Lott The OP expresses the concern of starting a subprocess on a per-request basis. So it's not only about starting subprocesses, but also e.g. about limiting their overall number. — ThomasH, May 11 '11 at 17:20
@ThomasH: "limiting their overall number"? It seemed more like the request-response of a web site is fast, but this subprocess might be slow. It's hard to tell. I don't get how the other references are not related. — S.Lott, May 11 '11 at 17:35
> I'm feeling a little uncomfortable > with having the python web app > starting new sox processes on my > server on a per request basis. To me this seems to mean that he fears that, if he opens up his webserver to the public, that there's not a lot he can do to prevent his server resources from being consumed if 15.000 people decide to click on that button that will launch sox in this way. — Alex Boschmans, Jan 16 '09 at 14:39

score 306 · Accepted Answer · edited Apr 23 '20 at 01:14

306

The subprocess module is the preferred way of running other programs from Python -- much more flexible and nicer to use than os.system.

import subprocess
#subprocess.check_output(['ls', '-l'])  # All that is technically needed...
print(subprocess.check_output(['ls', '-l']))

edited Apr 23 '20 at 01:14

Anonsage

7,194
3
41
50

answered Jan 16 '09 at 12:48

dF.

67,375
27
125
135

6

import subprocess; subprocess.check_output(['ls', '-l']). for using ls -l in the command line. check_output() return the output of the command too. – eleijonmarck Mar 11 '15 at 16:26
6

by using `subprocess.check_call(['ls','-l'])` you don't need to print the output. – Yar Dec 12 '18 at 23:00
what is wrong with using the `os`? – Charlie Parker Mar 14 '21 at 13:58

score 28 · Answer 2 · edited Jul 03 '20 at 12:52

This whole setup seems a little unstable to me.

Talk to the ffmpegx folks about having a GUI front-end over a command-line backend. It doesn't seem to bother them.

Indeed, I submit that a GUI (or web) front-end over a command-line backend is actually more stable, since you have a very, very clean interface between GUI and command. The command can evolve at a different pace from the web, as long as the command-line options are compatible, you have no possibility of breakage.

score 3 · Answer 3 · answered Jan 16 '09 at 18:11

If you're concerned about server performance then look at capping the number of running sox processes. If the cap has been hit you can always cache the request and inform the user when it's finished in whichever way suits your application.

Alternatively, have the n worker scripts on other machines that pull requests from the db and call sox, and then push the resulting output file to where it needs to be.

score 2 · Answer 4 · answered Jan 16 '09 at 12:36

2

I am not familiar with sox, but instead of making repeated calls to the program as a command line, is it possible to set it up as a service and connect to it for requests? You can take a look at the connection interface such as sqlite for inspiration.

answered Jan 16 '09 at 12:36

z -

6,990
3
36
65

You are spot on, but unfortunately sox does not run as a server daemon. At least not as far as I can tell. – Mattias Jan 16 '09 at 14:21

Executing command line programs from within python

4 Answers4

Linked

Related