2

I'm working on an automation process in C# that is going to remotely reboot a Windows (2008/2012/2016) server and I need to wait until that server is back online before proceeding.

I know 'back online' can be ambiguous, so for my requirements, I need the server to be back at the Ctrl-Alt-Del screen. The reason for this is to have the server in a consistent state before proceeding. In my experience, there are several factors that could prevent the server from reaching this screen, such as installing windows updates that gets stuck in a reboot cycle or getting stuck at 'Waiting for Local Session Manager' etc.

I've spent a few days looking in to this to no avail:

  • The server obviously starts responding to ping requests before it is available
  • System Boot Time occurs before the Server reaches the desired state
  • Any events indicating the system has booted are logged before the desired state
  • I can't simply poll for an essential service - when Windows is applying computer updates prior to logon these services can be already started. Additionally, sometimes a server will reboot itself whilst installing updates at this stage which could result in false positives.
  • Polling CPU activity could also produce false positives or introduce delays

Is there anyway to detect a Windows server has finished booting and is available for an interactive logon?

Cœur
  • 32,421
  • 21
  • 173
  • 232
sunts
  • 21
  • 1
  • Maybe run your detection code as a delayed start service? https://stackoverflow.com/questions/11015189/automatic-vs-automatic-delayed-start/11015576#11015576 – Tim Medora Jul 18 '18 at 21:21
  • I forgot to mention I cannot install a Windows Service to these server. I could create an executable and invoke via PsExec or PowerShell though. The issue is what would the detection code be, I'm at a loss as to how to detect the desired state. – sunts Jul 18 '18 at 21:27
  • Maybe create a service and deploy it to auto start...and ping that service – Dan Hunex Jul 18 '18 at 21:27
  • The issue with a service is it is likely to be started prior to reaching the Ctrl-Alt-Del screen, so polling it until it's running could produce a false positive. Additionally, as above, I'm unable to install a Windows Service to these servers :( – sunts Jul 18 '18 at 21:35
  • You know you can configure windows to autologon with a given user and pwd stored in registry. Why are you waiting for the logon screen? – Alois Kraus Jul 18 '18 at 21:38
  • Autologon is not an option for me in our enterprise and logging in interactively requires MFA. Plus I need to do this remotely rather than interactively as it needs to scale to potentially thousands of servers – sunts Jul 18 '18 at 21:43
  • You can check if a reboot is pending (https://stackoverflow.com/questions/47867949/how-can-i-check-for-a-pending-reboot) and if not and the logon screen is present you can proceed. It can also help to check the setup logs before continuing. – Alois Kraus Jul 18 '18 at 22:06
  • I can check for logonui.exe in session 1 which would indicate the server has loaded the logon screen, but this window is also visible when installing Windows Updates, or a boot issue has occurred such as it being stuck at Waiting for Local Session Manager etc. – sunts Jul 18 '18 at 22:45

2 Answers2

0

It sounds like you've covered most of the possible ways I know of. Which makes me revert to brute force ideas. I am curious what you're doing where you can't install a windows service on the box (or is that just not very viable because of the number)

First would just be trying to remote login or whatever, and having some way to test if it fails or not, wait 1 minute, try again. But seems like that might cause side-issues for you somehow?

My idea of a brute force method that wouldn't affect state:

  • Ping every 1-5seconds
  • Once it starts responding
  • wait 5 or 10 or even 15 minutes, whilst still pinging it
  • If pings fail reset that timer (windows updates restart case)
  • Then be pretty confident you're at the right state.

With potentially thousands of servers, I can't imagine 15 minutes each would be a big deal, especially if it is consistent enough to be able to run in larger batches

Thymine
  • 7,739
  • 1
  • 28
  • 42
  • I am thinking some kind of 'brute force' approach may be the only option. I can check for the logonui.exe process is running in session 1 which indicates the server has booted somewhat, there's just no way that I can find to pull the text from the window looking for Ctrl-Alt-Del. Obviously this is by design since winlogon is supposed to be a secure desktop. I don't really like the idea of pinging for a set amount of time since it will introduce unnecessary delays. Installing Windows Updates sometimes takes a lot longer than 15 mins too which means the poll would need to be extended. – sunts Jul 18 '18 at 22:43
  • 1
    Maybe I've been lucky with hardware and keeping the OS up-to-date on service packs at least. Watching for `logonui` seems like it should be fairly reliable, maybe also check for processes with `Windows*KB*.exe` (and whatever other patterns you know of) in them to also be able to detect windows updates (basically develop a white-list and a black-list of processes to watch for and decide to wait or proceed) – Thymine Jul 19 '18 at 13:53
0

So I've been able to accomplish this by using a hacky method put seems to work in my test environment.

Note that the el.Current.Name property will equate to the Ctrl-Alt-Del text, so on 2008R2 this is 'Press CTRL-ALT-DEL to log on' and 'Press CTRL-ALT-DEL to sign in.' on 2012R2

I've built a C# console application that uses UI Automation:

using System;
using System.Windows.Automation;

namespace WorkstationLocked
{
    class Program
    {
        static void Main()
        {
            AutomationElement el = AutomationUI.FindElementFromAutomationID("LockedMessage");

            if (el !=null)
            {
                Console.WriteLine(el.Current.Name);
            }
        }
    }
    class AutomationUI
    {
        public static AutomationElement FindElementFromAutomationID(string automationID)
        {
            string className = "AUTHUI.DLL: LogonUI Logon Window";

            PropertyCondition condition = new PropertyCondition(AutomationElement.ClassNameProperty, className);
            AutomationElement logonui = AutomationElement.RootElement.FindFirst(TreeScope.Children, condition);

            if (logonui != null)
            {
                condition = new PropertyCondition(AutomationElement.AutomationIdProperty, automationID);
                return logonui.FindFirst(TreeScope.Descendants, condition);
            }
            else
            {
                return null;
            }

        }
    }
}

I can then execute this console application via PsExec, however, because this needs to be launched in the winlogon desktop, which can only be done by running under the local system, PsExec is invoked twice. For example:

psexec.exe \\ServerA -s -d C:\PsTools\PsExec.exe -accepteula -d -x C:\Utils\WorkstationLocked.exe

This is very much a work in progress right now as I can't get the output of the command to pass through to the calling process so I may just look to populate a registry value or write to a file that can be subsequently interrogated.

HoldOffHunger
  • 10,963
  • 6
  • 53
  • 100
sunts
  • 21
  • 1