How to select elements within an iframe element in Puppeteer

Question

Since ESPN does not provide an API, I am trying to use Puppeteer to scrape data about my fantasy football league. However, I am having a hard time trying to login using puppeteer due to the login form being nested with an iframe element.

I have gone to http://www.espn.com/login and selected the iframe. I can't seem to select any of the elements within the iframe except for the main section by doing

    frame.$('.main')

This is the code that seems to get the iframe with the login form.

    const browser = await puppeteer.launch({headless:false});
    const page = await browser.newPage();

    await page.goto('http://www.espn.com/login')
    await page.waitForSelector("iframe");

    const elementHandle = await page.$('div#disneyid-wrapper iframe');
    const frame = await elementHandle.contentFrame();
    await browser.close()

I want to be able to access the username field, password field, and the login button within the iframe element. Whenever I try to access these fields, I get a return of null.

hardkoded · Accepted Answer · 2019-06-03T01:33:47.527

22

You can get the iframe using contentFrame as you are doing now, and then call $.

const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();

await page.goto('http://www.espn.com/login')
await page.waitForSelector("iframe");

const elementHandle = await page.$('div#disneyid-wrapper iframe');
const frame = await elementHandle.contentFrame();
await frame.waitForSelector('[ng-model="vm.username"]');
const username = await frame.$('[ng-model="vm.username"]');
await username.type('foo');
await browser.close()

edited Jun 03 '19 at 01:33

answered Jun 03 '19 at 01:07

hardkoded

13,167
3
34
48

frame.$('[ng-model="vm.username"]') seems to return null. – SwapnikK Jun 03 '19 at 01:27
@SwapnikK you could also use a waitForSelector. Answer edited. – hardkoded Jun 03 '19 at 01:34

score 8 · Answer 2 · answered Jan 31 '20 at 07:59

I had an issue with finding stripe elements. The reason for that is the following:

You can't access an with different origin using JavaScript, it would be a huge security flaw if you could do it. For the same-origin policy browsers block scripts trying to access a frame with a different origin. See more detailed answer here

Therefore when I tried to use puppeteer's methods:Page.frames() and Page.mainFrame(). ElementHandle.contentFrame() I did not return any iframe to me. The problem is that it was happening silently and I couldn't figure out why it couldn't find anything.

Adding these arguments to launch options solved the issue: '--disable-web-security', '--disable-features=IsolateOrigins,site-per-process'

many thanks for your answer, this fixed my issue. in case you know can you please share any disadvantages of enabling these flags? — opensource-developer, Aug 20 '20 at 22:22

score -1 · Answer 3 · edited Sep 29 '19 at 14:08

-1

await elementHandle.contentFrame() is null:

const frame = await elementHandle.contentFrame();

edited Sep 29 '19 at 14:08

Brian Tompsett - 汤莱恩

5,195
62
50
120

answered Sep 29 '19 at 13:37

Jared Chacon

1
1

Please elaborate the answer. This looks insufficient – Mohit Mutha Sep 29 '19 at 13:43
const frame = await elementHandle.contentFrame(); – Jared Chacon Sep 29 '19 at 14:05

How to select elements within an iframe element in Puppeteer

3 Answers3

Linked