Can't get details inside class in Python

Question

I am trying to scrape the following website using the python and selenium code. But when I try to print the text inside the "col-md-12 messageContentColumn" class, nothing seems to happen and nothing gets printed and I don't get an error either.

driver.get("https://www.gpug.com/resources/forum")
driver.find_element_by_xpath('//*[@id="MainCopy_ctl02_MessageThreadSummary_hyperlinksubject_1"]').click()

posts = driver.find_elements_by_class_name("col-md-12 messageContentColumn")
print("test")      ---gets printed
for post in posts:
print(post.text)   ---Nothing gets printed
print("end")       ---gets printed

Answer 1

find_elements_by_class_name() accepts only classname NOT Multi class names.Instead you can use css selector.

To handle dynamic element induce WebDriverWait And visibility_of_all_elements_located()

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


posts=WebDriverWait(driver,10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,".col-md-12.messageContentColumn")))
for post in posts:
   print(post.text)

Answer 2

Your code at this line return 0 :

posts = driver.find_elements_by_class_name("col-md-12 messageContentColumn")

It cause your loop block never execute.

*_by_class_name() just for single class name, instead you can use *_by_css_selector.

driver.get('https://www.gpug.com/resources/forum')
driver.maximize_window()
elmt = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="MainCopy_ctl02_MessageThreadSummary_hyperlinksubject_1"]')))
driver.execute_script("arguments[0].scrollIntoView();", elmt)
elmt.click()

posts = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, '.col-md-12.messageContentColumn')))
for post in posts:
    print(post.text)

driver.quit()

Following import:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Answer 3

You won't be able to pass multiple classnames when using find_elements_by_class_name() and can pass only a single classname.

You can find a relevant detailed discussion in Invalid selector: Compound class names not permitted using find_element_by_class_name with Webdriver and Python

However, you were so close. You need to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

• XPATH:

  driver.get("https://www.gpug.com/resources/forum")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='MainCopy_ctl02_MessageThreadSummary_hyperlinksubject_1']"))).click()
  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='col-md-12 messageContentColumn']")))])
  

• CSS_SELECTOR:

  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.col-md-12.messageContentColumn")))])
  

• Console Output:

  ['Dear Collaborators,\n\nWe recently started using GP in a Remote Desktop Server (RDS) environment. Eventually we\'ll uninstall GP from each of the user workstations.\n\nFor now we\'re keeping both installations to ensure that our add-ons work properly with RDS.\n\nOne thing that has bothered some of our users is the default printer. They go into Microsoft Dynamics GP >> Print Setup... and choose their preferred printer. They work and print and everything\'s sunshine and kittens.\n\nThen the next day they log into GP on the RDS and find that the default printer has been set to something else. My SWAG (Scientific Wild Ass Guess) is that the default printer is set to whatever the last user set it to. In other words, the default printer is not an individual setting... at least that\'s the way it appears.\n\nWhat am I missing?\n\nSincerely,\n\n------------------------------\n"Sparkly" Steve Erbach - Green Bay, WI\nCo-Chair, GPUG WI (Green Bay) Chapter\nBlog: https://www.gpug.com/blogs/steve-erbach\nTwitter: twitter.com/serbach\n\n\n---------------\nExcel Webinar List as of 22-Mar-2019\n------------------------------', 'Hi @Steve Erbach\n\nWhen they first log onto the remote computer is the default printer the same default printer as their local machine?\n\n------------------------------\nKristen Hosman\nIndependent Dynamics ERP Consultant\nMount Evans Consulting LLC\n------------------------------\n\nOriginal Message', '@Kristen Hosman,\n\nIn this particular user\'s case, when she logs into GP on the RDS, the default printer is always the Microsoft PDF printer. She wants it to be the small laser printer right on her desk -- which is a network printer, by the way.\n\nAnd another "by the way", that is a terrific profile pic, Kristen!\n\nSincerely,\n\n------------------------------\n"Sparkly" Steve Erbach - Green Bay, WI\nCo-Chair, GPUG WI (Green Bay) Chapter\nBlog: https://www.gpug.com/blogs/steve-erbach\nTwitter: twitter.com/serbach\n\n\n---------------\nExcel Webinar List as of 22-Mar-2019\n------------------------------\n\nOriginal Message', 'You may have already considered this, but what about utilizing Named Printers?\n  Only sets the default for a subset of forms/reports, but it can be done by individual user on each machine.\n  This has been a great tool for our accounting department.\n\n------------------------------\nMark LeRette\nApplication System Analyst II\nMuscatine Power and Water\nMuscatine IA\n------------------------------\n\nOriginal Message', '@Mark LeRette,\n\nI appreciate that suggestion. And here I was just working with Named Printers the other day for another purpose. I will check that out. Thank you.\n\nSincerely,\n\n------------------------------\n"Sparkly" Steve Erbach - Green Bay, WI\nCo-Chair, GPUG WI (Green Bay) Chapter\nBlog: https://www.gpug.com/blogs/steve-erbach\nTwitter: twitter.com/serbach\n\n\n---------------\nExcel Webinar List as of 22-Mar-2019\n------------------------------\n\nOriginal Message', '@Steve Erbach\n\nHave a look at my articles on the topic\n\nhttps://winthropdc.wordpress.com/2008/08/15/using-named-printers-with-terminal-server/\n\nhttps://winthropdc.wordpress.com/2013/05/14/using-named-printers-with-terminal-server-revisited/\n\nOriginal creator/Developer of Named Printers\nDavid\n\n------------------------------\nDavid Musgrave MVP, GPUG All-Star\n\nManaging Director\nWinthrop Development Consultants\n\nPerth, Western Australia\n\nhttp://www.winthropdc.com\n------------------------------\n\nOriginal Message', 'Hi Steve,\nDo you know if the printer in the local resources is marked on the remote desktop application when they are connecting to your terminal server?  In theory it should take the default printer from the local machine to the terminal server :)\n\n------------------------------\nGerald Buenafe\nOwner/Consultant\nBTP Technologies LLC\nCHANTILLY VA\n------------------------------\n\nOriginal Message', 'Dear @Steve Erbach,\nThere is a very simple explanation to your problem.. and a very simple solution, in fact even two.\nWhen running GP off a TS or Citrix environment, all the users are using the same "GP Client", thus all users are sharing the same DEX.ini file.. which implies that every users is storing his/her own printer settings in the same DEX.ini file.. As a result, with several dozens or hundreds of users, that file can become very long and cluttered with entries about all possible printers in the network.\nRead my blog post from 5 years ago : https://dyngpgeek.wordpress.com/2014/07/25/using-dynamics-gp-in-a-citrix-ts-environment/\nIt explains how to solve the problem.. Either use a personal user profile DEX.ini for each user, or use the \'per-user\' DEX.ini settings, which stores the values no longer in the physical .ini file itself, but rather in the DYNAMICS database for each user.\nSee my comment at the end of the blog post about the pros & cons of each option.\n\n------------------------------\nBeat Bucher\nBusiness Analyst, Dynamics GP SME\nMontreal QC/Canada\n@GP_Beat http://www.gp-geek.com\nMontreal QC GPUG Chapter Leader\nMBS MVP (2015-2018)\n\n------------------------------\n\nOriginal Message']