Selenium with Python : how to click dojo combobox values

Question

I'm navigating a JS-heavy webpage with Selenium and I need to be able to interact with a dojo component on the page. The page I'm looking at has a dojo dijit form with a combobox that has subject names for my university. I want to expose and iteratively click on every item in the list in order to scrape the course names for that subject when it redirects. The list items are exposed when the dropdown arrow button is clicked.

the url I'm automating: http://sis.rutgers.edu/soc/#subjects?semester=12020&campus=NB,NK,CM&level=U,G

I'm inspecting the element for the dropdown button and copying the XPath.

dropdownButton = driver.find_element_by_xpath('//*[@id="widget_dijit_form_FilteringSelect_0"]/div[1]/input')

Running this yields:

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="widget_dijit_form_FilteringSelect_0"]/div[1]/input"}

EDIT: I've made some progress, turn out the element wasn't rendered by the time find_by_xpath was called. I added a wait in my program, and now Selenium is able to locate and click the drowdown button.

Please share your code and url if possible, or dropdown and options HTML in text format? — Sers, Jan 21 '20 at 15:57
What is the behavior of this element? For example, how do you "expose" every item in the dropdown -- do you click the main empty box? How does this HTML screenshot relate to your image? It's quite difficult to tell which elements in the HTML match up with the screenshot you've posted. Posting raw HTML text (not a screenshot) or a link to the page you are automating would be more helpful. — Christine, Jan 21 '20 at 15:57
@Christine the items are exposed when the dropdown arrow button is pressed. The url has been added to the question. — Greg M, Jan 21 '20 at 16:10
@GregM Which is that _...dojo dijit form with a combobox that has subject names for my university..._ — DebanjanB, Jan 21 '20 at 16:17
@GregM show what you tried until now and where do you have problems and I'll help or will give you good solution. — Sers, Jan 21 '20 at 16:50

Sers · Accepted Answer · 2020-01-21T17:12:12.633

Use WebDriverWait to wait require element conditions. Dropdown disappears on any action on the page, that's why to get option locator you can do one of the following:

all options loaded after first expand, that's why you can search option element by text in chrome dev tools and get locator
pause and inspect the element.

You can google best practice for locators,here and here.

from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)

with driver:
    driver.get("http://sis.rutgers.edu/soc/#subjects?semester=12020&campus=NB,NK,CM&level=U,G")
    wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#filteringSelectDiv .dijitArrowButtonInner"))).click()
    options = driver.execute_script('return [...arguments[0]].map(e=>e.textContent)',
                                    wait.until(EC.presence_of_all_elements_located(
                                        (By.CSS_SELECTOR, ".dijitComboBoxMenuPopup .dijitMenuItem[item]"))))

    for option in options:
        driver.find_element_by_css_selector(".dijitInputInner").clear()
        driver.find_element_by_css_selector(".dijitInputInner").send_keys(option, Keys.TAB)
        wait.until(lambda d: d.execute_script("return document.readyState === 'complete'"))
        # collect data

Selenium with Python : how to click dojo combobox values

1 Answers1