Time out exception with WebDriverWait despite faster internet and element present

Question

I am trying to scrape this: https://www.lanebryant.com/chiffon-faux-wrap-fit-flare-midi-dress/prd-355958#color/0000091393

And this is my code:

wait = WebDriverWait(d, 10)
close = wait.until(EC.element_to_be_clickable((By.XPATH, "//a[@id='closeButton']")))
close.click()
time.sleep(5)
chart = wait.until(EC.element_to_be_clickable((By.XPATH, "//div[contains(*,'Size Guide')][@class='size-chart-link']")))
chart.click()

It first closes the pop up and then clicks the size guide, However, it always gives timeout exception and works only a couple of times.

Answer 1

The PARTIAL_LINK_TEXT Size Guide is pretty much unique within the page so would be your best bet would be to:

• Induce WebDriverWait for invisibility_of_element() for the wrapper element
• Induce WebDriverWait for the element_to_be_clickable() for the desired element

• You can use the following Locator Strategy:

  • Code Block (using XPATH and PARTIAL_LINK_TEXT):

    from selenium import webdriver
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
    options = webdriver.ChromeOptions()
    options.add_argument('start-maximized')
    driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
    driver.get('https://www.lanebryant.com/chiffon-faux-wrap-fit-flare-midi-dress/prd-355958#color/0000091393')
    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@id='closeButton']"))).click()
    WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.XPATH, "//div[@id='tinymask']")))
    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "Size Guide"))).click()
    

  • Code Block (using CSS_SELECTOR and PARTIAL_LINK_TEXT):

    from selenium import webdriver
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
    options = webdriver.ChromeOptions()
    options.add_argument('start-maximized')
    driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
    driver.get('https://www.lanebryant.com/chiffon-faux-wrap-fit-flare-midi-dress/prd-355958#color/0000091393')
    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a#closeButton"))).click()
    WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.CSS_SELECTOR, "div#tinymask")))
    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "Size Guide"))).click()
    

• Browser Snapshot:

Answer 2

Use JavaScript Executor to click on the element.Seems like selenium webdriver unable to click on the element.Use the below xpath

d.get("https://www.lanebryant.com/chiffon-faux-wrap-fit-flare-midi-dress/prd-355958#color/0000091393")

wait = WebDriverWait(d, 10)
close = wait.until(EC.element_to_be_clickable((By.XPATH, "//a[@id='closeButton']")))
close.click()
chart = wait.until(EC.element_to_be_clickable((By.XPATH, "//div[@class='size-chart-link']/a[contains(.,'Size Guide')]")))
d.execute_script("arguments[0].click();", chart)

Browser snapshot: