How to detect when subprocess asks for input in Windows

Question

I have a subprocess that either quits with a returncode, or asks something and waits for user input.

I would like to detect when the process asks the question and quit immediately. The fact that the process asks the question or not is enough for me to decide the state of the system.

The problem is that I cannot read the question because the child process probably does not flush standard output. So I cannot rely on parsing subprocess.Popen().stdout: when trying to read it, well, it blocks because input is being read first.

A bit like this

# ask.py, just asks something without printing anything if a condition is met
# here, we'll say that the condition is always met
input()

Of course, the actual subprocess is a third party binary, and I cannot modify it easily to add the necessary flush calls, which would solve it.

I could also try the Windows equivalent of unbuffer (What is the equivalent of unbuffer program on Windows?) which is called winpty, which would (maybe) allow me to detect output and solve my current issue, but I'd like to keep it simple and I'd like to solve the standard input issue first...

I tried... well, lots of things that don't work, including trying to pass a fake file as stdin argument, which doesn't work because subprocess takes the fileno of the file, and we cannot feed it rubbish...

p = subprocess.Popen(["python","ask.py"],...)

Using communicate with a string doesn't work either, because you cannot control when the string is read to be fed to the subprocess (probably through a system pipe).

Those questions were promising but either relied on standard output, or only apply to Linux

What I'm currently doing is running the process with a timeout, and if the timeout is reached, I then decide that the program is blocked. But it costs the timeout waiting time. If I could decide as soon as stdin is read by the subprocess, that would be better.

I'd like to know if there's a native python solution (possibly using ctypes and windows extensions) to detect read from stdin. But a native solution that doesn't use Python but a non-Microsoft proprietary language could do.

do you have any control over the subprocess you are trying to detect the state of? — Ma0, Mar 21 '18 at 13:59
If you can, you should go with the `pexepct` package available at https://pypi.python.org/pypi/pexpect. Otherwise you probably need to use the `pty` package to open up a pseudo-terminal, to make your subprocess believe it is communcating with a terminal and not being a (terminal free) subprocess. Or, if your third-party file is a python script you can force it to not buffer output by using the `-u` flag when calling the script (`['python', '-u', 'ask.py']`) but then you could probably edit it as well. — JohanL, Mar 21 '18 at 23:11
I'll have a go a pexpect. pty doesn't import on windows. cannot use -u option on subprocess: this isn't python... thanks for your suggestions. — Jean-François Fabre, Mar 22 '18 at 06:47
If it's a console application and doesn't write much to stdout, I suggest passing it a new console screen buffer, initialized to all NULs to make it easy to read lines as null-terminated strings. Then poll the buffer for output or wait for console WinEvents. winpty no doubt works similarly since Windows doesn't have pty devices. But if your needs are simple, then you can do without it. You just need ctypes or PyWin32. — Eryk Sun, Mar 22 '18 at 07:36
thanks. That's out of my windows abilities, but I trust your expertise in the field. — Jean-François Fabre, Mar 22 '18 at 08:36
you want general detection for any possible console process, or only for your concrete binary ? — RbMm, Apr 02 '18 at 15:01
let's say a binary written in C and reading from `stdin`. Possibly ported from unix/linux to windows. — Jean-François Fabre, Apr 02 '18 at 15:09
how do this (with write 1 byte to asynchronous pipe with 0 size buffer) on pyton i don`t know and are this possible at all, but on c/c++ this is very easy. then we wait on 2 objects - process itself and write complete event. but this is only solution, if you not want that child process got real user input. simply plan kill it in this case — RbMm, Apr 03 '18 at 12:20
that looks good, but the 0-biffer & recieve event part is kind of difficult for me to figure out. Maybe you can post a C++ solution ? — Jean-François Fabre, Apr 03 '18 at 12:22

RbMm · Accepted Answer · 2018-04-03T16:26:45.010

if we not want let to child process process user input, but simply kill it in this case, solution can be next:

start child process with redirected stdin to pipe.
pipe server end we create in asynchronous mode and main set pipe buffer to 0 size
before start child - write 1 byte to this pipe.
because pipe buffer is 0 size - operation not complete, until another side not read this byte
after we write this 1 byte and operation in progress (pending) - start child process.
finally begin wait what complete first: write operation or child process ?
if write complete first - this mean, that child process begin read from stdin - so kill it at this point

one possible implementation on c++:

struct ReadWriteContext : public OVERLAPPED
{
    enum OpType : char { e_write, e_read } _op;
    BOOLEAN _bCompleted;

    ReadWriteContext(OpType op) : _op(op), _bCompleted(false)
    {
    }
};

VOID WINAPI OnReadWrite(DWORD dwErrorCode, DWORD dwNumberOfBytesTransfered, OVERLAPPED* lpOverlapped)
{
    static_cast<ReadWriteContext*>(lpOverlapped)->_bCompleted = TRUE;
    DbgPrint("%u:%x %p\n", static_cast<ReadWriteContext*>(lpOverlapped)->_op, dwErrorCode, dwNumberOfBytesTransfered);
}

void nul(PCWSTR lpApplicationName)
{
    ReadWriteContext wc(ReadWriteContext::e_write), rc(ReadWriteContext::e_read);

    static const WCHAR pipename[] = L"\\\\?\\pipe\\{221B9EC9-85E6-4b64-9B70-249026EFAEAF}";

    if (HANDLE hPipe = CreateNamedPipeW(pipename, PIPE_ACCESS_DUPLEX|FILE_FLAG_OVERLAPPED, 
        PIPE_TYPE_BYTE|PIPE_READMODE_BYTE|PIPE_WAIT, 1, 0, 0, 0, 0))
    {
        static SECURITY_ATTRIBUTES sa = { sizeof(sa), 0, TRUE };
        PROCESS_INFORMATION pi;
        STARTUPINFOW si = { sizeof(si)};
        si.dwFlags = STARTF_USESTDHANDLES;
        si.hStdInput = CreateFileW(pipename, FILE_GENERIC_READ|FILE_GENERIC_WRITE, 0, &sa, OPEN_EXISTING, 0, 0);

        if (INVALID_HANDLE_VALUE != si.hStdInput)
        {
            char buf[256];

            if (WriteFileEx(hPipe, "\n", 1, &wc, OnReadWrite))
            {
                si.hStdError = si.hStdOutput = si.hStdInput;

                if (CreateProcessW(lpApplicationName, 0, 0, 0, TRUE, CREATE_NO_WINDOW, 0, 0, &si, &pi))
                {
                    CloseHandle(pi.hThread);

                    BOOLEAN bQuit = true;

                    goto __read;
                    do 
                    {
                        bQuit = true;

                        switch (WaitForSingleObjectEx(pi.hProcess, INFINITE, TRUE))
                        {
                        case WAIT_OBJECT_0:
                            DbgPrint("child terminated\n");
                            break;
                        case WAIT_IO_COMPLETION:
                            if (wc._bCompleted)
                            {
                                DbgPrint("child read from hStdInput!\n");
                                TerminateProcess(pi.hProcess, 0);
                            }
                            else if (rc._bCompleted)
                            {
__read:
                                rc._bCompleted = false;
                                if (ReadFileEx(hPipe, buf, sizeof(buf), &rc, OnReadWrite))
                                {
                                    bQuit = false;
                                }
                            }
                            break;
                        default:
                            __debugbreak();
                        }
                    } while (!bQuit);

                    CloseHandle(pi.hProcess);
                }
            }

            CloseHandle(si.hStdInput);

            // let execute pending apc
            SleepEx(0, TRUE);
        }

        CloseHandle(hPipe);
    }
}

another variant of code - use event completion, instead apc. however this not affect final result. this variant of code give absolute the same result as first:

void nul(PCWSTR lpApplicationName)
{
    OVERLAPPED ovw = {}, ovr = {};

    if (ovr.hEvent = CreateEvent(0, 0, 0, 0))
    {
        if (ovw.hEvent = CreateEvent(0, 0, 0, 0))
        {
            static const WCHAR pipename[] = L"\\\\?\\pipe\\{221B9EC9-85E6-4b64-9B70-249026EFAEAF}";

            if (HANDLE hPipe = CreateNamedPipeW(pipename, PIPE_ACCESS_DUPLEX|FILE_FLAG_OVERLAPPED, 
                PIPE_TYPE_BYTE|PIPE_READMODE_BYTE|PIPE_WAIT, 1, 0, 0, 0, 0))
            {
                static SECURITY_ATTRIBUTES sa = { sizeof(sa), 0, TRUE };
                PROCESS_INFORMATION pi;
                STARTUPINFOW si = { sizeof(si)};
                si.dwFlags = STARTF_USESTDHANDLES;
                si.hStdInput = CreateFileW(pipename, FILE_GENERIC_READ|FILE_GENERIC_WRITE, 0, &sa, OPEN_EXISTING, 0, 0);

                if (INVALID_HANDLE_VALUE != si.hStdInput)
                {
                    char buf[256];

                    if (!WriteFile(hPipe, "\n", 1, 0, &ovw) && GetLastError() == ERROR_IO_PENDING)
                    {
                        si.hStdError = si.hStdOutput = si.hStdInput;

                        if (CreateProcessW(lpApplicationName, 0, 0, 0, TRUE, CREATE_NO_WINDOW, 0, 0, &si, &pi))
                        {
                            CloseHandle(pi.hThread);

                            BOOLEAN bQuit = true;

                            HANDLE h[] = { ovr.hEvent, ovw.hEvent, pi.hProcess };

                            goto __read;
                            do 
                            {
                                bQuit = true;

                                switch (WaitForMultipleObjects(3, h, false, INFINITE))
                                {
                                case WAIT_OBJECT_0 + 0://read completed
__read:
                                    if (ReadFile(hPipe, buf, sizeof(buf), 0, &ovr) || GetLastError() == ERROR_IO_PENDING)
                                    {
                                        bQuit = false;
                                    }
                                    break;
                                case WAIT_OBJECT_0 + 1://write completed
                                    DbgPrint("child read from hStdInput!\n");
                                    TerminateProcess(pi.hProcess, 0);
                                    break;
                                case WAIT_OBJECT_0 + 2://process terminated
                                    DbgPrint("child terminated\n");
                                    break;
                                default:
                                    __debugbreak();
                                }
                            } while (!bQuit);

                            CloseHandle(pi.hProcess);
                        }
                    }

                    CloseHandle(si.hStdInput);
                }

                CloseHandle(hPipe);
                // all pending operation completed here.
            }

            CloseHandle(ovw.hEvent);
        }

        CloseHandle(ovr.hEvent);
    }
}

I think I won't even try to port that to python. I'll use the code as-is in a small tool exe which takes the command as argument, so I can run it from my python code. Thanks. I'm going to test that soon enough. — Jean-François Fabre, Apr 03 '18 at 14:55
@Jean-FrançoisFabre - in my test with cmd.exe/notepad (xp-win10) this work well. of course can litle change code, for use instead apc completion, event completion in overlapped. but this not affect result — RbMm, Apr 03 '18 at 15:42
@Jean-FrançoisFabre - DbgPrint this is only for debug output. you can at all remove it (without replace) - it not have any functional point. and i use UNICODE (W) api. you look like try ansi. i use CL.EXE (msvc) compiler — RbMm, Apr 03 '18 at 15:54
@Jean-FrançoisFabre - declare direct `STARTUPINFOW` with `W`. i simply use unicode defined and it auto added for me. change to `STARTUPINFOW si = { sizeof(si)};` — RbMm, Apr 03 '18 at 15:55
DbgPrint or print - help test, but always the best test under debugger. — RbMm, Apr 03 '18 at 16:04
this seems to work, but `TerminateProcess` seems to kill only `cmd`, and I'm running a python script. Typical annoyance. Will test with a C/C++ program directly. — Jean-François Fabre, Apr 03 '18 at 16:14
@Jean-FrançoisFabre of course that `TerminateProcess` kill only 1 process, for which it will be called. i not sure what you want do, if you detect read from *stdin* in child. in place `TerminateProcess` can be another actions. this is only detection of read — RbMm, Apr 03 '18 at 16:20
@Jean-FrançoisFabre - i add second variant of code (with event completion). may be it will be even more simply for you. however this not change main idea. only bit different thin implementation details. anyway both variants will be ok, of fail at the same place — RbMm, Apr 03 '18 at 16:28
ok, with a python subprocess it didn't work, probably because of python itself, but with custom made C programs it worked. Thanks a lot. — Jean-François Fabre, Apr 03 '18 at 20:22
@Jean-FrançoisFabre - with *python subprocess it didn't work* - you mean if exec some code (may be another exe) via python ? and what is not work in this case ? process terminate ? detect stdin read ? — RbMm, Apr 03 '18 at 20:28
stdin read detection. But it doesn't matter, as a plain C program works fine. — Jean-François Fabre, Apr 03 '18 at 20:35

Aran-Fey · Answer 2 · 2018-04-01T23:58:47.553

3

My idea to find out if the subprocess reads user input is to (ab)use the fact that file objects are stateful: if the process reads data from its stdin, we should be able to detect a change in the stdin's state.

The procedure is as follows:

Create a temporary file that'll be used as the subprocess's stdin
Write some data to the file
Start the process
Wait a little while for the process to read the data (or not), then use the tell() method to find out if anything has been read from the file

This is the code:

import os
import time
import tempfile
import subprocess

# create a file that we can use as the stdin for the subprocess
with tempfile.TemporaryFile() as proc_stdin:
    # write some data to the file for the subprocess to read
    proc_stdin.write(b'whatever\r\n')
    proc_stdin.seek(0)

    # start the thing
    cmd = ["python","ask.py"]
    proc = subprocess.Popen(cmd, stdin=proc_stdin, stdout=subprocess.PIPE)

    # wait for it to start up and do its thing
    time.sleep(1)

    # now check if the subprocess read any data from the file
    if proc_stdin.tell() == 0:
        print("it didn't take input")
    else:
        print("it took input")

Ideally the temporary file could be replaced by some kind of pipe or something that doesn't write any data to disk, but unfortunately I couldn't find a way to make it work without a real on-disk file.

edited Apr 01 '18 at 23:58

answered Apr 01 '18 at 23:06

Aran-Fey

30,995
8
80
121

nice try, but if the process is allowed to take all the input, it will reach end of file and will perform the task I don't want it to perform. I need to detect one char read so I can kill the process. – Jean-François Fabre Apr 02 '18 at 07:16
this would answer if the question was "How to detect when subprocess _asked_ for input in Windows". I need an instant detection. – Jean-François Fabre Apr 02 '18 at 07:22
@Jean-FrançoisFabre - *I need to detect one char read so I can kill the process* - if you want kill process, if it begin read from *stdin* - you need create asynchronous pipe with 0 size buffer. connect child *stdin* to this pipe, and write 1 byte to pipe with event as completion. because pipe buffer is 0 - operation not complete until somebody not read from connected pipe end (*stdin*) or you not close pipe handle. then you start child process with redirected *stdin* and wait for event + child exit. if event fire first - child begin read from *stdin*. – RbMm Apr 03 '18 at 00:56
don`t know in pyton, but in c++ this is very easy to implement – RbMm Apr 03 '18 at 00:56
`time.sleep(1)` - why not `time.sleep(748)` ? – RbMm Apr 03 '18 at 00:57
Because I don't want to wait 12 minutes every time I test this code? – Aran-Fey Apr 03 '18 at 00:58
but anyway why sleep 1 but not another value ? what if 1 not enough ? this indeterminate time interval say that this is not solution – RbMm Apr 03 '18 at 06:57
@RbMm sorry, your ping failed :) I didn't see your comment earlier. that makes sense. the event part I'm not aware how to do, but sounds a promising solution. – Jean-François Fabre Apr 03 '18 at 12:13

How to detect when subprocess asks for input in Windows

2 Answers2

Linked