1

On Python 3.5.1, I have the following:

output = subprocess.check_output(cmd).decode(encoding="UTF-8")

This calls the properly invoked command cmd. C++14 code in cmd looks like:

HANDLE handle = GetStdHandle(STD_OUTPUT_HANDLE);
assert(handle!=INVALID_HANDLE_VALUE); //Always passes
assert(handle!=nullptr);              //Always passes

CONSOLE_SCREEN_BUFFER_INFO csbi;
BOOL result = GetConsoleScreenBufferInfo(handle,&csbi);
assert(result!=0); //Always fails.  `GetLastError()` returns 6 (invalid handle)

Running the above Python code causes the subprocess cmd to fail at the indicated line. According to the Python docs, in this case, the stdout/stderr should be inherited from the parent process (i.e., the Python interpreter). So, it shouldn't. In fact, the above works just fine for e.g. printfed output.

Attempting to redirect explicitly also fails:

#Traceback (most recent call last):
#  File "C:\dev\Python35\lib\subprocess.py", line 914, in __init__
#    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
#  File "C:\dev\Python35\lib\subprocess.py", line 1145, in _get_handles
#    c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
#io.UnsupportedOperation: fileno
p = subprocess.Popen(cmd.split(" "),stdout=sys.stdout,stderr=sys.stderr)

#Using `subprocess.PIPE` instead fails in the same way as the `subprocess.check_output(...)`
#example originally given above.

What's going wrong? How do I fix it?

imallett
  • 13,288
  • 9
  • 49
  • 115
  • 1
    `check_output` sets `stdout=PIPE`. You can't use that handle with console functions such as `GetConsoleScreenBufferInfo` and `WriteConsole`. `printf` works because ultimately it calls `WriteFile`, which writes bytes to any kind of file handle. – Eryk Sun Jun 03 '16 at 23:27
  • 1
    Is `sys.stdout` a filelike object in an IDE? In that case it's not a operating system file with a handle that can be inherited in the standard handles of the child process. If your C++ code falls back to using C standard I/O (preferably line unbuffered) when stdout isn't a console, then your Python code can use a thread to read from the `p.stdout` pipe. `check_output` and `Popen.communicate` do this for you, but you may need something more interactive. – Eryk Sun Jun 03 '16 at 23:35
  • 2
    @HarryJohnston, we know that `sys.stdout` isn't associated with a Windows file (console, pipe, disk) because it doesn't have a `fileno()`. I think imallett wants standard output from the child process to show up on Python's `sys.stdout`, regardless of what `sys.stdout` is (e.g. `sys.stdout` could be a proxy object that talks to an IDE's interactive window using a pipe or socket). In that case a thread can loop over `p.stdout.read(1)` or `p.stdout.readline()` and write to `sys.stdout`. – Eryk Sun Jun 04 '16 at 02:20
  • @eryksun: right you are. The call to `.Popen` should have worked as shown if the OP were talking about the simple scenario I was thinking of. At least, it works for me. :-) – Harry Johnston Jun 04 '16 at 02:43

1 Answers1

2

A pipe is not a console. check_output() uses stdout=PIPE internally. It won't redirect the console output (produced by WriteConsoleW()).

The first error (invalid handle) suggests that the standard output is not the console device.

Unless it is a desired behavior (to print outside of the standard output); use WriteFile() if stdout is not a console device.

The second error (io.UnsupportedOperation: fileno) suggests that sys.stdout is not a real file and therefore you can't pass it as stdout parameter to a subprocess (you could redirect subprocess' stdout using stdout=PIPE and print the output using print() or sys.stdout.write() method directly).

Community
  • 1
  • 1
jfs
  • 346,887
  • 152
  • 868
  • 1,518
  • After a bit of outside reading in addition to this answer, I think what I really wanted was for `cmd` to use the console opened by the Python interpreter. Since `cmd` is a subprocess, I'm not sure that desire makes sense.¶ When invoking `cmd`, however, `cmd` seems to open its own console window (a new one appears when run). Can you please instead explain why `cmd` doesn't recognize this window and write to it when, for example, there is no redirection in the Python invocation? – imallett Jun 04 '16 at 21:03
  • @imallett: "How a C++ application can use the console opened by its parent Python script (via subprocess)" sounds like a good question but it would be the third issue (try to limit your questions to a single issue per question). Have you tried the obvious? (don't redirect stdout, just don't pass `stdout` parameter e.g., use `subprocess.check_call(cmd)`—the subprocess should print to the console in this case). Also, it seems your code assumes that the stdout is the console (I initially missed it—I've updated the answer accordingly). – jfs Jun 04 '16 at 21:12
  • I didn't mean to ask how I could use the parent's console, as that would indeed be a separate question. I meant I was unclear on how what I did failed.¶ Attempting the suggested `subprocess.check_call(cmd)` works (the assertions pass, and the output is printed to the console). When redirected, it crashes.¶ I think I understand: the handle is generated from the stdout handle. When redirected, that's a pipe, not a file directed into a terminal. So the terminal still exists, but since stdout is not attached to it, getting the handle returns the pipe instead, So terminal-ish things fail. Yes? – imallett Jun 04 '16 at 22:06
  • @imallett yes. The gist is correct: *"the handle is generated from the stdout handle. When redirected, that's a pipe, not a file directed into a terminal."* – jfs Jun 04 '16 at 22:19
  • @imallett, if you know you're attached to a console (e.g. `GetConsoleCP` returns a non-zero value), you can open `\\.\CONOUT$` to write to it. It should be opened with `"r+"` access. – Eryk Sun Jun 05 '16 at 01:24