I'm currently working on learning Python and I'm already stuck on a strange problem.
I'm getting urls from a website using beautifulsoup, which I can already save to a mysql database. Now I want to let the script first check if a crawled url is already in the database.
The query looks like this:
check_url = "SELECT url FROM shop_ig " \
"WHERE url = '%s'"
Here I'm getting the url:
soup = BeautifulSoup(plain_text, "html.parser")
for link in soup.findAll('a', {'class': 'ig-preorders-item'}):
href = link.get('href')
print(href)
Now that I have the url I would like to check if href is already in the database so I execute the query using:
## Check if url is in db
cursor.execute(check_url, href)
data = cursor.fetchone()
sqlconnect.commit()
Now to the strange thing, data returns None, however, if I enter an url permanently into the query it finds the url in the database.