1

At the moment I've got a strange release-only crash in my program. It seems to be throwing a NullReferenceException on webPageLoader[i].Join(); but I doesn’t seem to fail in debug.

Could my problem be related to the fact that the thread has not had the chance to start so therefore can’t be joined? Has anyone got any tips to how I could go about debugging this as it doesn’t seem to manifest in debug at all.

If I put a sleep(50) before webPageLoader[i].Join(); it doesn’t seem to crash. But I would expected it to not crash without the sleep.

for (int i = 0; i < pagesToGet; i++)
{
    crawlContext.vistedURLs.AddURL(CurrentURL);                                
    document[i] = new DocumentToScrape();
    webPageLoader[i] = new Thread(document[i].LoadPage);
    webPageLoader[i].Start(CurrentURL);

    if (!crawlContext.unVistedURLs.MoveNext())
    {
        break;
    }
}

for (int i = 0; i < pagesToGet; i++)
{
    **webPageLoader[i].Join();**
    if (document[i].Doc != null)
    {
        document[i].Doc = GetSubSetXPath(document[i].Doc, crawlContext.xPath);                      
        if (document[i].Doc != null)
        {
            crawlContext.scrapedUrls = ScrapeURLS(crawlContext.scrapedUrls, document[i]);                                      
        }
        else
        {
            //System.Windows.Forms.MessageBox.Show("Null Page Found");
        }

    }
}
Kristof U.
  • 1,098
  • 8
  • 17
Gregory William Bryant
  • 390
  • 2
  • 12
  • 3
    At first sight it seems that the possible `break` in the first loop might mean that you end up with fewer threads than you expect in the second loop. – Roger Rowland Dec 05 '13 at 19:43
  • 9
    The use of `Sleep` in a multi-threaded program indicates a bug in the program. – John Saunders Dec 05 '13 at 19:44
  • Almost all cases of `NullReferenceException` are the same. Please see "[What is a NullReferenceException in .NET?](http://stackoverflow.com/questions/4660142/what-is-a-nullreferenceexception-in-net)" for some hints. – John Saunders Dec 05 '13 at 19:45
  • 1
    @JohnSaunders It sounds like the sleep was just a troubleshooting step they did that revealed a different behavior. He stated it didn't crash with the sleep, and if he believed that to be a viable solution, he probably wouldn't be here seeking help. – AaronLS Dec 05 '13 at 19:47
  • @JohnSaunders is there another post that explains why using sleep in a multi-threaded program is a bug? – AlG Dec 05 '13 at 19:48
  • @AaronLS: no, it sounds like he thinks the `Sleep` is required, and doesn't understand the behavior that caused it to "work" with the sleep. – John Saunders Dec 05 '13 at 19:48
  • 1
    @AlG: no, just experience. Sleep indicates that the developer doesn't understand multi-threading. – John Saunders Dec 05 '13 at 19:49
  • 2
    @John He didn't ask you "What does the use of sleep reveal about the programmer using it?" – AaronLS Dec 05 '13 at 19:51
  • undoubtedly in this case pagesToGet was bigger than crawlContext.vistedUrls. So, one of: **webPageLoader[i].Join();** didn't get created. Almost really nothing to do with threading. (except if it worked it would use a thread) –  Dec 05 '13 at 19:53
  • 1
    @AIG Usually using Sleep isn't reliable because it depends on timing of other threads, which will vary. So if you used Sleep to wait for another thread to release a resource, but that thread took a little longer than usual, you would have a program bug/crash because the Sleep wasn't long enough. Therefore, reliable thread management techniques use more **deterministic** approaches to manage dependencies, waits, and resource sharing. – AaronLS Dec 05 '13 at 19:54
  • @AaronLS: and yet, even though he didn't ask, I was kind enough to tell him. I have found a one to one correlation between the use of sleep for synchronization in a multi-threaded program, and a lack of understanding of multi-threading, and a number of difficult bugs. 1 to 1 to 1. – John Saunders Dec 05 '13 at 19:55
  • @ebyrob that's exactly what I thought (see first comment) but I don't see how a sleep would have changed the behaviour. – Roger Rowland Dec 05 '13 at 19:55
  • @AaronLS too short? I thought because too long when unexpected things happen causing threads to be too unresponsive for too long. –  Dec 05 '13 at 19:56
  • @ebyrob Both, too long and yes you are right, you will have unresponsive problem. If you are using it to solve resource contention problems or dependencies on other threads completing work, then you will have bugs as Sleep is nothing more than guessing at when those things will complete. In that case you have both problems, because good thread management techniques would allow the waiting thread to continue as soon as it needed to, but with a Sleep technique it is just a guess at how long to wait, and will thus be longer than needed making it inefficient. – AaronLS Dec 05 '13 at 20:01
  • @JohnSaunders "I was kind enough to tell him" I didn't see anything kind about it, frankly it was rude. You basically waved your hands and said *just because* which anyone can do without supporting rational and then threw a cheap jab in there. If you were as experienced as you make out you'd be able to convey the rational from your experience. – AaronLS Dec 05 '13 at 20:04
  • @AaronLS resource contention? You must do a lot of threading. I'm considering the humble timer and trying to decide why sleep "isn't good enough" to implement that. Perhaps only because "QuitVeryFast()" isn't Very enough. –  Dec 05 '13 at 20:04
  • @ebyrob I was speaking in more general terms of why sleep is bad all around. There are lots of scenarios sleep is used improperly, just trying my best to cover all of them in response to your question about why sleep is considered a bug. If what you're trying to implement is a timer, you could look at the .NET source code for how timers are implemented if you are really curious.(it is available under BCL reference source) – AaronLS Dec 05 '13 at 20:09
  • @Roger Rowland: This infact does seem to the problem, but I'm not sure why the problem didn’t manifest in release? – Gregory William Bryant Dec 05 '13 at 20:12
  • 1
    @AaronLS: if I had a rationale based on published articles, then I would have referred to the articles. Instead, my rationale is based on over 35 years of experience, so that's what I gave. In my experience, developers struggling with threading (including my own self back in about 2002) often compensate by using `Sleep`. It's a mistake. It's better to never use `Sleep`, and instead to find out what's actually wrong with the program. This also helps develop the skills necessary to not need to resort to `Sleep`. – John Saunders Dec 05 '13 at 20:33
  • @GregoryWilliamBryant You're sure both loops are in the same thread and only ever called from one thread (not perhaps two threads running same code at same time?) Also, could we get any more info about the context above this code? Like how we know pagesToGet matches crawlContext every time. –  Dec 05 '13 at 20:42
  • @JohnSaunders The use of Sleep in a multi-threaded program MAY WELL indicate a bug in the program. No, it is not a lock. – Martin James Dec 05 '13 at 20:42
  • sleep is NOT bad all around. – Martin James Dec 05 '13 at 20:43
  • If a requirement is for a five second wait, sometimes, five function levels deep inside a threaded subsystem, (maybe it's controlling some hardware), You can use Sleep() or rewrite your software as a state-machine so that you can use a timer. No-brainer. – Martin James Dec 05 '13 at 20:46
  • Whay IS totally wrong is to use Sleep() calls as a substitute for sane and correct inter-thread signaling, or as other similar bodge such as might be suggested by the OP question. – Martin James Dec 05 '13 at 20:49
  • Some good points: Sleep() calls require next-to-no external resources. No other threads are required to run timers or timed tasks on threadpools. Sleep() will work in any thread, anywhere in the flow of control. The fact that Sleep() cannot be aborted does not matter in many app-lifetime threads and, in any case, can usually be circumvented by a timeout wait on a synchro object, (essentially the same mechanism as sleep). – Martin James Dec 05 '13 at 20:55
  • The lack of break has fixed the problem, I seem to have other problems now :S. FYI - I am in process of learning threading, rather than being an experienced programmer. Hence why asked if the sleep was required to allow the thread to actually start. Thankyou for all the helpful comments. – Gregory William Bryant Dec 05 '13 at 21:10
  • Fixed, OK, good luck! – Martin James Dec 05 '13 at 21:45
  • 1
    @MartinJames: your points are valid - for advanced or demanding threading scenarios, such as in embedded systems. Not so much for a programmer beginning to learn threading, who is much more likely to have "simple" synchronization issues, which he may be masking by using `Sleep`. – John Saunders Dec 05 '13 at 21:57
  • 1
    @JohnSaunders - indeed. I 100% agree that sleep() calls should not be misused, and that such misuse is far too common :( – Martin James Dec 06 '13 at 01:25

0 Answers0