0

So I'm trying to get the left bar of this website: https://experience.arcgis.com/experience/685d0ace521648f8a5beeeee1b9125cd to get the current WHO coronavirus data. I'm locating the countries by Xpath, my code is:

from selenium import webdriver

import time


driver = webdriver.Firefox()

driver.get("link_to_the_page")

time.sleep(30)


countries = country = "/html/body/div/div/div[2]/div/div/div/margin-container/full-container/div[10]/margin-container/full-container/div/div[2]/nav/span/div/div/p/span"

countries_name = driver.find_elements_by_xpath(countries)

But when I run this, the list is empty which is weird because the page looks exactly the same when being displayed by Firefox using Selenium. So why can't the browser find the elements?

Svetlana Levinsohn
  • 1,450
  • 3
  • 8
  • 19
iamsmorrin
  • 11
  • 2

3 Answers3

1

Alternatively you can get the data directly here :

https://services.arcgis.com/5T5nSi527N4F7luB/arcgis/rest/services/Historic_adm0_v3/FeatureServer/0/query?f=json&where=1%3D1&returnGeometry=false&spatialRel=esriSpatialRelIntersects&outFields=ADM0_NAME%2Ccum_conf%2CDateOfDataEntry&orderByFields=DateOfDataEntry%20asc&resultOffset=2000&resultRecordCount=2000&cacheHint=true

Parse the json, and do whatever you want. covid_data_who

E.Wiest
  • 5,122
  • 2
  • 4
  • 11
0

Enter iframe first driver.switch_to.frame(driver.find_element_by_css_selector("/html/body/div[2]/div[1]/div/div/div/div/div/div/div/iframe"))

country = driver.find_element_by_xpath("//*[@id="Cases_by_country_pt_V3_6708_0_layer"]/circle[32]")
country .click()

don't copy "full xpath",Should be copy "xpath"

Ruyut
  • 135
  • 9
0

As the the desired element is within an <iframe> so to access the desired elements you have to:

  • Induce WebDriverWait for the desired frame_to_be_available_and_switch_to_it().
  • Induce WebDriverWait for the desired visibility_of_all_elements_located().

  • You can use the following Locator Strategies:

    driver.get("https://experience.arcgis.com/experience/685d0ace521648f8a5beeeee1b9125cd")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.TAG_NAME,"iframe")))
print([my_elem.text for my_elem in WebDriverWait(driver,20).until(EC.visibility_of_all_elements_located((By.XPATH, "//strong[contains(., 'areas or territories with cases')]//following::div[1]//div[@class='external-html']//span")))])

  • Console Output:

    ['China :81151 cases', 'Italy :31506 cases', 'Iran (Islamic Republic of) :16169 cases', 'Spain :11178 cases', 'Republic of Korea :8413 cases', 'France :7652 cases', 'Germany :7156 cases', 'United States of America :4356 cases', 'Switzerland :2650 cases', 'The United Kingdom :1954 cases', 'Netherlands :1705 cases', 'Belgium :1486 cases', 'Austria :1471 cases', 'Norway :1423 cases', 'Sweden :1167 cases', 'Denmark :1044 cases', 'Canada :993 cases', 'Japan :873 cases', 'International conveyance (Diamond Princess) :712 cases', 'Malaysia :673 cases', 'Australia :510 cases', 'Czechia :464 cases', 'Portugal :448 cases', 'Qatar :442 cases', 'Greece :387 cases', 'Finland :319 cases', 'Israel :304 cases', 'Ireland :292 cases', 'Slovenia :275 cases', 'Singapore :266 cases', 'Estonia :258 cases', 'Poland :246 cases', 'Romania :246 cases', 'Pakistan :241 cases', 'Bahrain :237 cases', 'Brazil :234 cases', 'Iceland :225 cases', 'Thailand :212 cases', 'Luxembourg :203 cases', 'Chile :201 cases', 'Egypt :196 cases', 'Philippines :187 cases', 'Indonesia :172 cases', 'Saudi Arabia :171 cases', 'Iraq :154 cases', 'India :147 cases', 'Kuwait :142 cases', 'Lebanon :120 cases', 'Peru :117 cases', 'Russian Federation :114 cases', 'United Arab Emirates :113 cases', 'San Marino :104 cases', 'Turkey :98 cases', 'Slovakia :97 cases', 'Mexico :93 cases', 'Panama :86 cases', 'Serbia :85 cases', 'South Africa :85 cases', 'Bulgaria :81 cases', 'Argentina :79 cases', 'Latvia :71 cases', 'Viet Nam :66 cases', 'Colombia :65 cases', 'Croatia :65 cases', 'Algeria :60 cases', 'Ecuador :58 cases', 'Hungary :58 cases', 'Brunei Darussalam :56 cases', 'Albania :55 cases', 'Armenia :52 cases', 'Costa Rica :50 cases', 'Morocco :49 cases', 'Jordan :48 cases', 'Faroe Islands :47 cases', 'Belarus :46 cases', 'occupied Palestinian territory :44 cases', 'Sri Lanka :42 cases', 'Georgia :38 cases', 'Malta :38 cases', 'Venezuela (Bolivarian Republic of) :36 cases', 'Azerbaijan :34 cases', 'Cambodia :33 cases', 'Cyprus :33 cases', 'Kazakhstan :33 cases', 'Oman :33 cases', 'North Macedonia :31 cases', 'Senegal :31 cases', 'Republic of Moldova :30 cases', 'Uruguay :29 cases', 'Guadeloupe :27 cases', 'Bosnia and Herzegovina :26 cases', 'Lithuania :26 cases', 'Tunisia :24 cases', 'Afghanistan :22 cases', 'Dominican Republic :21 cases', 'Burkina Faso :20 cases', 'New Zealand :20 cases', 'Andorra :16 cases', 'Martinique :16 cases', 'Uzbekistan :16 cases', 'Ukraine :14 cases', 'Maldives :13 cases', 'Bolivia (Plurinational State of) :12 cases', 'Jamaica :12 cases', 'Réunion :12 cases', 'French Guiana :11 cases', 'Paraguay :11 cases', 'Bangladesh :10 cases', 'Cameroon :10 cases', 'Monaco :9 cases', 'Honduras :8 cases', 'Rwanda :8 cases', 'Democratic Republic of the Congo :7 cases', 'Ghana :7 cases', 'Liechtenstein :7 cases', 'Ethiopia :6 cases', 'Guatemala :6 cases', 'Côte d’Ivoire :6 cases', 'Cuba :5 cases', 'Jersey :5 cases', 'Mongolia :5 cases', 'Trinidad and Tobago :5 cases', 'Guam :5 cases', 'Guyana :4 cases', 'Kenya :4 cases', 'Seychelles :4 cases', 'Aruba :4 cases', 'Curacao :3 cases', 'Equatorial Guinea :3 cases', 'French Polynesia :3 cases', 'Gabon :3 cases', 'Gibraltar :3 cases', 'Mayotte :3 cases', 'Nigeria :3 cases', 'Puerto Rico :3 cases', 'Saint Barthelemy :3 cases', 'Namibia :2 cases', 'Saint Martin :2 cases', 'Saint Lucia :2 cases', 'United States Virgin Islands :2 cases', 'Liberia :2 cases', 'Montenegro :2 cases', 'Barbados :2 cases', 'Antigua and Barbuda :1 cases', 'Bhutan :1 cases', 'Cayman Islands :1 cases', 'Central African Republic :1 cases', 'Congo :1 cases', 'Eswatini :1 cases', 'Guernsey :1 cases', 'Guinea :1 cases', 'Holy See :1 cases', 'Mauritania :1 cases', 'Nepal :1 cases', 'Saint Vincent and the Grenadines :1 cases', 'Sudan :1 cases', 'Togo :1 cases', 'Suriname :1 cases', 'Somalia :1 cases', 'Benin :1 cases', 'United Republic of Tanzania :1 cases', 'Bahamas :1 cases', 'Gambia :1 cases', 'Djibouti :1 cases', 'Montserrat :1 cases']

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
DebanjanB
  • 118,661
  • 30
  • 168
  • 217