reticulate doesn't print to console in real time

Question

Using Python's print() (or to the best of my knowledge, any other console output generating) function in loop structures and running the code via reticulate in R, the output is only printed after the execution finished. For example, take the following loop which goes to sleep for 1.5 seconds after each iteration; the run numbers are all printed in one go after the loop is through. The same applies when saving the Python code to a separate .py file and then running reticulate::py_run_file().

library(reticulate)

py_run_string("
import time

for i in range(5):
   print(str(i))
   time.sleep(1.5) # sleep for 1.5 sec
")

Does anyone know where this behavior comes from and, if possible, how to circumvent it?

This seems to be an issue with python. When you put that into a script and run it using `nohup thisscript.py` you can see that the output will be be held back until the script finishes. That's just from my experience. Unfortunately, I have no idea why — loki, Mar 18 '20 at 09:20
Amazing, inserting `sys.stdout.flush()` right after the `print()` call indeed sends a status update to the console at each iteration. Thanks! — fdetsch, Mar 18 '20 at 09:31

score 4 · Accepted Answer · answered Mar 18 '20 at 09:28

Apparently, you need to force pyhton to export the standard output at times. You can do this by adding sys.stdout.flush() to your code:

library(reticulate)

py_run_string("
import time
import sys

for i in range(5):
   print(str(i))
   time.sleep(1.5) # sleep for 1.5 sec
   sys.stdout.flush()
")

see over here described with nohup

score 2 · Answer 2 · answered Mar 27 '20 at 05:46

For future reference, enforcing to flush the stream via print(..., flush = True) works as well (as suggested here) and renders importing sys obsolete:

library(reticulate)

py_run_string("
import time

for i in range(5):
   print(str(i), flush = True)
   time.sleep(1.5)
")

reticulate doesn't print to console in real time

2 Answers2