1

I'm currently working on learning Python and I'm already stuck on a strange problem.

I'm getting urls from a website using beautifulsoup, which I can already save to a mysql database. Now I want to let the script first check if a crawled url is already in the database.

The query looks like this:

check_url = "SELECT url FROM shop_ig " \
    "WHERE url = '%s'"

Here I'm getting the url:

soup = BeautifulSoup(plain_text, "html.parser")

for link in soup.findAll('a', {'class': 'ig-preorders-item'}):
    href = link.get('href')
    print(href)

Now that I have the url I would like to check if href is already in the database so I execute the query using:

    ## Check if url is in db
    cursor.execute(check_url, href)
    data = cursor.fetchone()
    sqlconnect.commit()

Now to the strange thing, data returns None, however, if I enter an url permanently into the query it finds the url in the database.

Hamlett
  • 403
  • 6
  • 19
Nolibert
  • 11
  • 1

1 Answers1

0

Thanks to alexce I found the problem.

I'm now using:

cursor.execute(check_url, (href, ))

The select now looks like this ('' removed)

check_url = "SELECT url FROM shop_instantgaming " \
"WHERE url = %s"
Community
  • 1
  • 1
Nolibert
  • 11
  • 1