0

We have a very strange problem on our production web app. Our web application is external facing and uses impersonation for app pool identity using a DMZ account. However randomly we see our application crashing while the user trying to login to the application. We get an exception as shown below-

As you can see when the authenticate method is called we get a failure indicating:

Is authenticated: False Authentication Type:
Thread account name: DMZ account

It appears as though the SQL Server does not allow the asp.net app pool identity to connect randomly, which is the DMZ account. Now this happens intermittently , not always. The asp.net application impersonates as pool identity, however it connects to SQL Server via a SQL login.

The weirdest thing is if we connect to production sever from Visual Studio, it connects to the same production SQL Server ok and the production site magically starts working ! It's almost as if the local domain account that Visual Studio impersonated as unlocked the production DMZ account. 

Exception information: 
    Exception type: NullReferenceException 
    Exception message: Object reference not set to an instance of an object.
   at eCertsFormsAdmin.ControllerBase.OnException(ExceptionContext filterContext)
   at System.Web.Mvc.ControllerActionInvoker.InvokeExceptionFilters(ControllerContext controllerContext, IList`1 filters, Exception exception)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass25.<BeginInvokeAction>b__22(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
   at System.Web.Mvc.Controller.<>c__DisplayClass1d.<BeginExecuteCore>b__18(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.<>c__DisplayClass8.<BeginProcessRequest>b__3(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)



Request information: 
    Request URL: https://appurl:443/Home/Authenticate?param={'AccessCode' :'demo', 'UserName' :'user'} 
    Request path: /Home/Authenticate 
    User host address: web server ip address 
    User:  
    Is authenticated: False 
    Authentication Type:  
    Thread account name: DMZ account 

Thread information: 
    Thread ID: 5 
    Thread account name: DMZ account
    Is impersonating: True 
    Stack trace:    at eCertsFormsAdmin.ControllerBase.OnException(ExceptionContext filterContext)
   at System.Web.Mvc.ControllerActionInvoker.InvokeExceptionFilters(ControllerContext controllerContext, IList`1 filters, Exception exception)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass25.<BeginInvokeAction>b__22(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
   at System.Web.Mvc.Controller.<>c__DisplayClass1d.<BeginExecuteCore>b__18(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.<>c__DisplayClass8.<BeginProcessRequest>b__3(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)


Custom event details: 

Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="ASP.NET 4.0.30319.0" />
    <EventID Qualifiers="32768">1309</EventID>
    <Level>3</Level>
    <Task>3</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2017-03-28T13:56:18.000000000Z" />
    <EventRecordID>425417</EventRecordID>
    <Channel>Application</Channel>
    <Computer>Web server</Computer>
    <Security />
  </System>
  <EventData>
    <Data>3005</Data>
    <Data>An unhandled exception has occurred.</Data>
    <Data>3/28/2017 9:56:18 AM</Data>
    <Data>3/28/2017 1:56:18 PM</Data>
    <Data>3accff4bebb74aa5823c9d05fe7ae5c0</Data>
    <Data>135</Data>
    <Data>4</Data>
    <Data>0</Data>
    <Data>/LM/W3SVC/6/ROOT-1-131351820594338174</Data>
    <Data>Full</Data>
    <Data>/</Data>
    <Data>C:\www\apppath\</Data>
    <Data>Web server</Data>
    <Data>
    </Data>
    <Data>6768</Data>
    <Data>w3wp.exe</Data>
    <Data>GTSDMZ\eValid</Data>
    <Data>NullReferenceException</Data>
    <Data>Object reference not set to an instance of an object.
   at eCertsFormsAdmin.ControllerBase.OnException(ExceptionContext filterContext)
   at System.Web.Mvc.ControllerActionInvoker.InvokeExceptionFilters(ControllerContext controllerContext, IList`1 filters, Exception exception)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.&lt;&gt;c__DisplayClass25.&lt;BeginInvokeAction&gt;b__22(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
   at System.Web.Mvc.Controller.&lt;&gt;c__DisplayClass1d.&lt;BeginExecuteCore&gt;b__18(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.&lt;&gt;c__DisplayClass4.&lt;MakeVoidDelegate&gt;b__3(IAsyncResult ar)
   at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.&lt;&gt;c__DisplayClass4.&lt;MakeVoidDelegate&gt;b__3(IAsyncResult ar)
   at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.&lt;&gt;c__DisplayClass8.&lt;BeginProcessRequest&gt;b__3(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.&lt;&gt;c__DisplayClass4.&lt;MakeVoidDelegate&gt;b__3(IAsyncResult ar)
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean&amp; completedSynchronously)

</Data>
    <Data>https://appurl:443/Home/Authenticate?param={'AccessCode' :'demo', 'UserName' :'user', }</Data>
    <Data>/Home/Authenticate</Data>
    <Data>server ip</Data>
    <Data>
    </Data>
    <Data>False</Data>
    <Data>
    </Data>
    <Data>DMZ account</Data>
    <Data>5</Data>
    <Data>DMZ account</Data>
    <Data>True</Data>
    <Data>   at OurApp.ControllerBase.OnException(ExceptionContext filterContext)
   at System.Web.Mvc.ControllerActionInvoker.InvokeExceptionFilters(ControllerContext controllerContext, IList`1 filters, Exception exception)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.&lt;&gt;c__DisplayClass25.&lt;BeginInvokeAction&gt;b__22(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
   at System.Web.Mvc.Controller.&lt;&gt;c__DisplayClass1d.&lt;BeginExecuteCore&gt;b__18(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.&lt;&gt;c__DisplayClass4.&lt;MakeVoidDelegate&gt;b__3(IAsyncResult ar)
   at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.&lt;&gt;c__DisplayClass4.&lt;MakeVoidDelegate&gt;b__3(IAsyncResult ar)
   at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.&lt;&gt;c__DisplayClass8.&lt;BeginProcessRequest&gt;b__3(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.&lt;&gt;c__DisplayClass4.&lt;MakeVoidDelegate&gt;b__3(IAsyncResult ar)
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean&amp; completedSynchronously)
</Data>
  </EventData>
</Event>

Our Environment:

Windows 2008 R2 Enterprise IIS 7.5 Web server is in DMZ

Madhuri Mittal
  • 493
  • 1
  • 6
  • 9
  • it seems you are facing login issues randomly,for every login failure,sql server logs a message into error log by default.Can you check that and paste it – TheGameiswar Apr 04 '17 at 15:48
  • We ran SQL Profiler on Prod SQL server. All we see is Entity Framework doing a Database Initialization. After that rest of the queries are not executed. Does that help... – Madhuri Mittal Apr 04 '17 at 16:48
  • The infrastructure were unable to find any entries in SQL logs – Madhuri Mittal Apr 04 '17 at 16:50
  • Probably related: [What is a NullReferenceException, and how do I fix it?](http://stackoverflow.com/questions/4660142/what-is-a-nullreferenceexception-and-how-do-i-fix-it) – dorukayhan Apr 04 '17 at 18:39

0 Answers0