How to get the text inside span data-bind part of html using selenium Python?

Question

This is the html snippet I am trying to web scrape using Python Selenium.

I am trying to get the text Add to bag which is inside a span data-bind.

<div class="is-add-item-saving" data-bind="visible: isBusy" style="display: none;"></div>
<span class="aria-live" aria-role="status" aria-live="polite" data-bind="{ text: ariaLiveText }"></span>
<button data-bind="click: addToBag, css : buttonCss, attr: { 'aria-label': resources.pdp_cta_add_to_bag, disabled: isBusy }, markAndMeasure: 'pdp:add_to_bag_interactive'" data-test-id="add-button" aria-label="Add to bag">
    <span class="product-tick" data-bind="visible: showProductTick" style="display: none;"></span>
    <span data-bind="text: buttonText">Add to bag</span>

</button>

This is what I have tried so far.

instock_element = driver.find_elements_by_xpath("//span[contains(@data-bind,'text: buttonText')]")
instock_element = driver.find_elements_by_xpath("//*[contains(text(), 'Add to bag')]")

When I iterate over these instock_elements,

for value in instock_element:
     print("text : ",value.text)
     print(" id : ",value.id)
     if len(value.text) == 0:
          text = value.id
     else:
          print(value.text)
          text = value.text
          ins_list.append(text)

These are giving me random values like 6489355d-9dd3-4d77-a0d7-b134ce48fae7 but not the text Add to bag.

Answer 1

Try this (in particular the xpath):

from lxml import html

sample = """<div class="is-add-item-saving" data-bind="visible: isBusy" style="display: none;"></div>
<span class="aria-live" aria-role="status" aria-live="polite" data-bind="{ text: ariaLiveText }"></span>
<button data-bind="click: addToBag, css : buttonCss, attr: { 'aria-label': resources.pdp_cta_add_to_bag, disabled: isBusy }, markAndMeasure: 'pdp:add_to_bag_interactive'" data-test-id="add-button" aria-label="Add to bag">
    <span class="product-tick" data-bind="visible: showProductTick" style="display: none;"></span>
    <span data-bind="text: buttonText">Add to bag</span>

</button>"""

print(html.fromstring(sample).xpath("//*[@data-bind='text: buttonText']/text()"))

Output:

['Add to bag']

Answer 2

https://www.selenium.dev/selenium/docs/api/py/webdriver_remote/selenium.webdriver.remote.webelement.html#module-selenium.webdriver.remote.webelement

id

Internal ID used by selenium.

This is mainly for internal use. Simple use cases such as checking if 2 webelements refer to the same element, can be done using ==:

if element1 == element2: print("These 2 are equal")

use value.get_attribute("id") instead to get id

to get text use:

value.text

if it fails use:

value.get_attribute("textContent")

as value.text retrieves only text that is displayed in UI

Answer 3

To print the text Add to bag you can use either of the following Locator Strategies:

• Using css_selector and get_attribute("innerHTML"):

  print(driver.find_element_by_css_selector("button[data-test-id='add-button'][aria-label='Add to bag'] span").get_attribute("innerHTML"))
  

• Using xpath and text attribute:

  print(driver.find_element_by_xpath("//button[@data-test-id='add-button' and @aria-label='Add to bag']//span").text)
  

---

Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

• Using CSS_SELECTOR and text attribute:

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "button[data-test-id='add-button'][aria-label='Add to bag'] span"))).text)
  

• Using XPATH and get_attribute():

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//button[@data-test-id='add-button' and @aria-label='Add to bag']//span"))).get_attribute("innerHTML"))
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

---

References

Link to useful documentation:

• get_attribute() method Gets the given attribute or property of the element.
• text attribute returns The text of the element.
• Difference between text and innerHTML using Selenium

Answer 4

If you are having hard time finding the right element, Easy way is to instead of finding all the elements of associated with xpath, You must use the full xpath of the individual tag and then get the text of it using .text

Example: text = driver.find_element_by_xpath("full xpath of the element").text

Answer 5

You may also use BeautifulSoup for this :

from bs4 import BeautifulSoup

html = """<div class="is-add-item-saving" data-bind="visible: isBusy" style="display: none;"></div>
<span class="aria-live" aria-role="status" aria-live="polite" data-bind="{ text: ariaLiveText }"></span>
<button data-bind="click: addToBag, css : buttonCss, attr: { 'aria-label': resources.pdp_cta_add_to_bag, disabled: isBusy }, markAndMeasure: 'pdp:add_to_bag_interactive'" data-test-id="add-button" aria-label="Add to bag">
    <span class="product-tick" data-bind="visible: showProductTick" style="display: none;"></span>
    <span data-bind="text: buttonText">Add to bag</span>
</button>"""

soup = BeautifulSoup(html)

tag = soup.find('span',{'data-bind':'text: buttonText'})
print(tag.text)

Output

Add to bag