I want to open a link from a website using Python, so here is the flow:
I open the main URL (e.g. www.url1.com)
I scrape the page and find the button, it has a redirection link (www.url2.com)
when I use this link in browser it redirects to (www.url3.com) then immediately goes to another (Required link) (www.url4.com)
When I try the same flow using Python requests, it only goes to (www.url3.com)
I tried using the
allow_redirects
argument without any success
Here is my code:
import requests
headers = {
'User-Agent': '',
'authority': '',
'scheme': '',
'accept': '',
'x-requested-with': '',
'cookie': '',
'referer':
}
def download(req):
resp = requests.get(req, headers=headers, allow_redirects=True)
print(resp.text)
I also tried to print history using this answer.
but it keeps redirecting me too (url3)