How to click on Search button under For Loop (Python Selenium)

Question

I am hoping that I can select all the stock in the drop down menu, hit the "Search" button and hit "Export to CSV" button. However, I am stuck that I am not able to hit the Search button after choosing the option. Please advise. Thank you

   from selenium import webdriver
   from selenium.webdriver.support.ui import Select, WebDriverWait
   from selenium.webdriver.common.by import By
   from selenium.webdriver.support import expected_conditions as EC
   from time import ctime

   browser = webdriver.Chrome()
   browser.get("https://www.hkex.com.hk/eng/sorc/options/statistics_hv_iv.aspx")

   select = Select(WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,        
    "//select[@id='underlying']"))))


   browser.implicitly_wait(10)
   optionsList = []

   #iterate over the options, place attribute value in list
   for option in select.options:
       option.click()
       optionsList.append(option.get_attribute("value"))

       # below code is not able to work properly but this is only what I am hoping to work
       elem = browser.find_element_by_link_text("Search")
       elem.click()
       elem1 = driver.find_element_by_link_text("Export to CSV")
       elem1.click()
       print(elem)

   print(optionsList)
   print(len(optionsList))


   browser.quit

score 0 · Answer 1 · answered Jul 31 '20 at 17:56

To click on an option e.g. (00002) CLP and then to click on the Search button you to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

driver.get("https://www.hkex.com.hk/eng/sorc/options/statistics_hv_iv.aspx")
Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "select#underlying")))).select_by_value("00002")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "td a[title='Search']"))).click()

Using XPATH:

driver.get("https://www.hkex.com.hk/eng/sorc/options/statistics_hv_iv.aspx")
Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//select[@id='underlying']")))).select_by_value("00002")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//td//a[@title='Search']"))).click()

Browser Snapshot:

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

score 0 · Accepted Answer · answered Jul 31 '20 at 18:04

You need to induce WebDriverWait() and wait for element_to_be_clickable()

While clicking on the Export to CSV button the page will refresh and it will be no longer available to the page and you will get stale exception.To avoid this you need to re-assign the element inside for loop.

Code:

from selenium import webdriver
from selenium.webdriver.support.ui import Select, WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

browser = webdriver.Chrome()
browser.get("https://www.hkex.com.hk/eng/sorc/options/statistics_hv_iv.aspx")
browser.maximize_window()
select = Select(WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,"//select[@id='underlying']"))))
optionsList = []
items=select.options
for i in range(len(items)):
    #re-assigned select elements again to avoid stale exception
    select = Select(WebDriverWait(browser, 30).until(EC.element_to_be_clickable((By.XPATH, "//select[@id='underlying']"))))
    items = select.options
    print(items[i].get_attribute("value"))
    select.select_by_value(items[i].get_attribute("value"))
    optionsList.append(items[i].get_attribute("value"))
    #click on search button
    WebDriverWait(browser,10).until(EC.element_to_be_clickable((By.XPATH,"//span[@class='floatleft']//a[@title='Search']"))).click()
    #click on download button
    WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//a[@title='Export to CSV']"))).click()

print(optionsList)
print(len(optionsList))

How to click on Search button under For Loop (Python Selenium)

2 Answers2