I need to monitor a list of websites through different ISPs to see if they are blocked. I have a different machine for each ISP. I'm trying to write a code to automatically check if the websites are blocked. A few things came to my mind but they are not working for different reasons:
ping: I thought I would ping websites but then some websites have their ICMP ports closed on their side.
get request (or javascript image trick): I thought maybe I just GET the webpage but that wouldn't help because the blocked pages still return some non-standard blockage page. 200 status.
Lastly, I thought maybe I get a copy of the website on a non-blocked machine and compare the page with the one on testing machines but there are 2 problems: I don't know how to compare two pages (like what element would I compare) and secondly, some websites are dynamic thus they return slightly different versions.
Any thoughts will be helpful.