How to find the element using selenium in python

Question

I can't find the element of the html below.

<span class="tabComboBoxName" id="tab-ui-id-1565209097494" aria-hidden="true">20/07/2019</span>

I've tried the following codes:

elem = browser.find_elements_by_xpath("//class[@id='tab-ui-id-1565209097494']")
elem = browser.find_elements_by_class_name('tabComboBoxName')
elem = browser.find_elements_by_id('tab-ui-id-1565209097494')

For those tries I got an empty list.

Can you post a link to the page you are trying to scrape please? — Robert Kearns, Aug 07 '19 at 21:17
I can't send the link because its a Tableau Dashboard then I need to send user and password. — Rafael Schettini, Aug 07 '19 at 21:19
Okay, well my guess is that what you are after is dynamically generated. Load the page, and then right click view source. Then hit ctrl-f and paste in 'tabComboBoxName' and tell me if it finds it in the source code. — Robert Kearns, Aug 07 '19 at 21:21
@RobertKearns You are right, I can't find the "tabComboBoxName" — Rafael Schettini, Aug 07 '19 at 21:25
The 1565209097494 looks like a timestamp FYI, so don't go by that. — pguardiario, Aug 08 '19 at 01:26

score 1 · Answer 1 · answered Aug 07 '19 at 21:25

1

The element is a dynamically generated element so to locate the element you need to induce WebDriverWait for the desired visibility_of_element_located() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.tabComboBoxName[id^='tab-ui-id-']")))

Using XPATH:

element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[@class='tabComboBoxName' and starts-with(@id, 'tab-ui-id-')][contains(., '20/07/2019')]")))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

answered Aug 07 '19 at 21:25

DebanjanB

118,661
30
168
217

@RafaelSchettini How about the _xpath_? – DebanjanB Aug 07 '19 at 21:30
@RafaelSchettini _TimeoutException_ is the outcome of **failed** _expected-conditions_. Debug your code through `find_element_by_*` in-conjunction with `time.sleep()`. If you are able to locate the element, update the question with the observations. – DebanjanB Aug 07 '19 at 21:33
the element is hidden – Corey Goldberg Aug 08 '19 at 01:35

score 0 · Answer 2 · answered Aug 08 '19 at 05:33

Make sure that the element doesn't belong to an <iframe>, if it does - you will need to switch_to() the iframe where the element lives prior to attempting to find it
Make sure that the element doesn't belong to ShadowDOM, if it does - you will need to locate the relevant ShadowRoot element, cast it to the WebElement and use find_element() function of the WebElement instead of driver
Make sure to use Explicit Wait as it might be the case the element is not immediately available and it's being loaded later via AJAX request
Try using another locator strategy, for instance you can stick to the element text like:
```
//span[text()='20/07/2019']
```

How to find the element using selenium in python

2 Answers2