I am writing a code where I am loading a lot of websites, and sometimes the link doesn't exists and instead it goes to another link (than the one I told it to).
So I want to be able to identify when the current site I am scrapping from isn't actually the address I told it to go to.
This is the code sample I am using. What should I add so I can find the name of the address it goes to?
req = Request(l, headers={'User-Agent': 'Mozilla/5.0'})
html_page = urlopen(req).read()
soup = BeautifulSoup(html_page, "lxml")