0

I want to Scrape a price value on a webiste with the python module BeautifulSoup, I tried multiple things but not found a solution. Also, the price constantly changes. I want to get the 1520.15 on the 7th line between "spans". The html file: 

<div class="tv-category-header__main-price js-scroll-container">
<div class="tv-scroll-wrap tv-scroll-wrap--horizontal js-scroll-scrollable-element">
    <div class="tv-category-header__main-price-content">

        <div class="tv-symbol-price-quote js-last-price-block">
            <div class="tv-symbol-price-quote__row js-last-price-block-value-row">
                <div class="tv-symbol-price-quote__value js-symbol-last"><span>1520.15</span></div>
                <div class="tv-symbol-price-quote__supply">
                    <div class="tv-symbol-price-quote__data-mode tv-data-mode tv-data-mode--size_large js-data-mode tv-data-mode--no-realtime apply-common-tooltip tv-data-mode--realtime tv-data-mode--realtime--no-realtime" title="Real-time">R</div>
                    <div class="tv-symbol-price-quote__currency js-symbol-currency">USD</div>
                </div>
                <div class="js-symbol-change-direction tv-symbol-price-quote__change tv-symbol-price-quote__change--growing">
                    <span class="js-symbol-change tv-symbol-price-quote__change-value">+5.18</span>
                    <span class="js-symbol-change-pt tv-symbol-price-quote__change-value">(+0.34%)</span>
                </div>
            </div>
            <div class="tv-symbol-price-quote__sub-line">
                <span class="js-last-price-block-title tv-symbol-price-quote__market-stat tv-symbol-price-quote__market-stat--open">Market Open</span>
                <span class="js-symbol-lp-time">(Dec 31 14:01 UTC-5)</span>
            </div>
        </div>
    </div>
</div>

This is my code:

import requests
import bs4
from bs4 import BeautifulSoup


r = requests.get('https://samplewebsite.com/pagewithprice')
page_soup = BeautifulSoup(r.text, 'lxml')
price = page_soup.find("div", class_="tv-symbol-price-quote__value js-symbol-last").find("span")
print(price)

I am using python 3.6 and the lastest release of Beautifulsoup

Tecnzz
  • 1
  • 2
    The chance is that the price is injected via Javascript into the document. BeautifulSoup doesn't execute Javascript. But you can use `selenium` for example. Can you share the URL? – Andrej Kesely Jan 01 '20 at 21:57
  • The exact link for that price is www.tradingview.com/symbols/XAUUSD/ – Tecnzz Jan 01 '20 at 22:22
  • The site seems loading the data through websockets, so BeautifulSoup won't help you here. You may try `selenium` or find if there's official API to get information from. – Andrej Kesely Jan 01 '20 at 22:40

0 Answers0