0

I have a page that submits data to this cgi script, this script then calls another executable. The issue is that this executable takes a very long time to run and as a result, it causes the cgi script to halt until finished. I'm trying to find a way to run it in the background. I'm coding in python on a linux machine. The issue that I'm running into is that when the script ends (the parent process is killed), the child process is also terminated by the init (in accordance with the linux spec). I googled around a little, and I came across this thread: Calling an external command in Python

I then tried the following in my code: 

print <html code>
pid = subprocess.Popen([sys.executable, '/path/to/executable.py '+arg1],
                       stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
                       stdin=subprocess.PIPE)
...
retval = pid.wait()
#For debugging purposes:
print str(retval) + "<br/>"
print </html code>

This is giving me a retval of '2' which I'm sure means something wrong (because i'm not seeing the right output) I tried some other methods but it either runs the script and takes a while to load (Meaning that it's running in the foreground), or it just fails to run the script all together.

Ideas? Thank you. Oh, btw, I'm using python 2.6

Community
  • 1
  • 1
de1337ed
  • 2,545
  • 9
  • 29
  • 48
  • What does the external process do? If it's slow, either cache results, or, use AJAX to have the page load instantly, and the data show on the page, when done. – ninMonkey Jun 26 '12 at 20:28
  • It does a lot of data driven calculations. It isn't something that updates the immediate page it's on, it updates an html page that displays all the data. I don't care if the page is updated immediately, I just don't want it so that when hitting the submit button, it doesn't hang. I want it to go to a submit successful page (The python script) that spawns the background process – de1337ed Jun 26 '12 at 20:36
  • Then you don't want to wait for the sub process to finish, before finishing outputting HTML, since that will hang. wait() waits for the child to end. http://docs.python.org/library/subprocess.html#popen-objects . See also: http://docs.python.org/library/subprocess.html#replacing-older-functions-with-the-subprocess-module – ninMonkey Jun 28 '12 at 21:59

1 Answers1

0

The solution is to generate some output before the script is finished. As for me, the best way is to output a script execution log using chunked transfer encoding.

Normally, CGI script output needs to be generated completely before output starts. This is required to calculate "Content-Length" header. Chunked transfer encoding works around this - you don't need to know whole response to output chunks.

Maksym Polshcha
  • 16,722
  • 8
  • 47
  • 73