0

I am looking to pull a table in at http://www.nfl.com/inactives?week=5 in order to process active and inactive players. I am very familiar with rvest and have tried using the code:

library(rvest)

url <- paste0("http://www.nfl.com/inactives?week=5")
Table  <- url %>%
  read_html() %>%
  html_nodes(xpath= '//*[contains(concat( " ", @class, " " ), concat( " ", "yui3-datatable-cell", " " ))]') %>%
  html_table()
TableNew <- Table[[1]]
TableNew

Nothing is coming up correctly though. Ideally, I would like to be able to put all the players and their team name into one single table. I appreciate your insights.

CooperBuckeye05
  • 137
  • 1
  • 12
  • are you sure the page returned is the same you have in your browser? try a simple nodes query like `a`. it's likely the page is generated dynamically. – MichaelChirico Dec 06 '17 at 15:02
  • Sorry, it fully loads once you open the page, but it does take time to pull in all the data. The data does not all come in at once to each of the tables, but does after a few seconds – CooperBuckeye05 Dec 06 '17 at 15:10
  • The page loads trough a lot of Javascript.. So you have at least 3 options: 1. Simulate an html_session that loads and executes the script (it might be tricky). 2. Load the page from a javascript enabled browser and save it to your hard drive, which will undoubtedly modify the html code on the saved copy. 3. Capture the Javascript content with rvest but parse it with another tool. To capture the JS content you can run: JSContent % read_html %>% html_nodes(xpath = '//script[@type="text/javascript"]') %>% as.character() – Nicolás Velásquez Dec 06 '17 at 15:55
  • 1
    Thanks for you help guys, but I found a solution by using Rselenium through which I go to a page scroll down a few times so that it fully loads, then do the scrape. Found at this post: https://stackoverflow.com/questions/29861117/r-rvest-scraping-a-dynamic-ecommerce-page – CooperBuckeye05 Dec 06 '17 at 15:57
  • I posted in the comments above but I found a solution here: https://stackoverflow.com/questions/29861117/r-rvest-scraping-a-dynamic-ecommerce-page – CooperBuckeye05 Dec 06 '17 at 15:58

0 Answers0