Automatically download files using selenium Chromedriver 3.11.0 on a Mac

Question

I've read through a bunch of selenium topics on here and I've been going in circles on how the permissions / options should be set for the chromedriver. I've made the following code:

System.setProperty("webdriver.chrome.driver", "/Users/username/chromedriver");
String downloadFilepath = "//User//username//automation-testing//";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("profile.default_content_settings.popups", 0);
chromePrefs.put("download.default_directory", downloadFilepath);
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", chromePrefs);
options.addArguments("disable-popup-blocking");
DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
cap.setCapability(ChromeOptions.CAPABILITY, options);
WebDriver driver = new ChromeDriver(cap);

The call to the download page is made using a simple

driver.get(url);

which redirects to a csv file.

I keep getting a popup prompt asking me if downloading files is okay. It's worth mentioning that the new ChromeDriver(cap) line is deprecated, but I can't seem to find documentation on how to use the method that replaced it that covers this use case.

simply `new ChromeDriver(options);` – Florent B. Apr 20 '18 at 00:06 — Florent B., Apr 20 '18 at 00:06

DebanjanB · Answer 1 · 2018-04-20T14:37:56.317

1

It seems you were almost there. You need to use the method merge() from MutableCapabilities Class to merge the DesiredCapabilities type of object into ChromeOptions type object and initiate the WebDriver and WebClient instance by passing the ChromeOptions object as follows :

System.setProperty("webdriver.chrome.driver", "/Users/username/chromedriver");
String downloadFilepath = "//User//username//automation-testing//";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("profile.default_content_settings.popups", 0);
chromePrefs.put("download.default_directory", downloadFilepath);
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", chromePrefs);
options.addArguments("disable-popup-blocking");
DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
cap.setCapability(ChromeOptions.CAPABILITY, options);
options.merge(cap);
WebDriver driver = new ChromeDriver(options);

PS : As a reference you can have a look at the discussions in mutablecapabilities tag

Update

As per your comment update as you are stuck with the download confirmation window you can look at the discussion Auto-download in firefox browser with java-selenium not working to solve your issue.

edited Apr 20 '18 at 14:37

answered Apr 20 '18 at 06:27

DebanjanB

118,661
30
168
217

Oddly enough, even after these changes I'm still getting a confirmation window when I run this -- do you have any insight as to why? – A_Elric Apr 20 '18 at 14:06
@A_Elric My answer was constructed as per your baseline question `It's worth mentioning that the new ChromeDriver(cap) line is deprecated, but I can't seem to find documentation on how to use the method that replaced it that covers this use case.`. If your requirement have changed feel free to raise a new ticket. Satckoverflow volunteers will be happy to help you out. – DebanjanB Apr 20 '18 at 14:10
the question is about the download not happening due to a popup window- though mentioning that this may be partly because I was not using mutable capabilities tag -- after changing the tags over, I remain blocked on the larger issue. I'm happy to upvote you for your efforts though. – A_Elric Apr 20 '18 at 14:30
@A_Elric Checkout my answer update and let me know the status. – DebanjanB Apr 20 '18 at 14:38
I am uncertain that solution will work with chrome, as the arguments are likely different... – A_Elric Apr 20 '18 at 14:40

score 1 · Accepted Answer · answered Apr 20 '18 at 18:30

Solution that I used was pretty intense full source below to log into jira via google below, then download a filter view to csv (current selection) below:

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.openqa.selenium.By;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeDriverService;
import org.openqa.selenium.chrome.ChromeOptions;

import java.io.File;
import java.io.FileFilter;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.HashMap;
import java.util.Map;

public class Scrape {
    public static void doScrape(String[] urls) {
        try{
            for(String url : urls) {
                //Create new Chromedriver, set file download path, allow the download popup to be automatically accepted,and merge the properties into chromedriver
                System.setProperty("webdriver.chrome.driver", "/Users/damienbell/chromedriver");
                String downloadFilepath = "/Users/damienbell/automation-testing";

                ChromeOptions options = new ChromeOptions();
                options.addArguments("--test-type");
                //options.addArguments("--headless");
                options.addArguments("--disable-extensions");

                //Instantiate above options in driverService
                ChromeDriverService driverService = ChromeDriverService.createDefaultService();
                ChromeDriver driver = new ChromeDriver(driverService, options);


                Map<String, Object> commandParams = new HashMap<>();
                commandParams.put("cmd", "Page.setDownloadBehavior");

                Map<String, Object> params = new HashMap<String, Object>();
                params.put("behavior", "allow");
                params.put("downloadPath", downloadFilepath);
                params.put("cmd", "Page.setDownloadBehavior");


                commandParams.put("params", params);
                ObjectMapper om = new ObjectMapper();
                CloseableHttpClient httpClient = HttpClients.createDefault();
                String command = null;
                try{
                    command = om.writeValueAsString(commandParams);
                }catch(JsonProcessingException jpe){ jpe.printStackTrace(); }
                String postURL = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";
                HttpPost postRequest = new HttpPost(postURL);
                postRequest.addHeader("content-type", "application/json");
                postRequest.addHeader("accept", "*.*");
                try{
                    postRequest.setEntity(new StringEntity(command));
                    httpClient.execute(postRequest);
                }
                catch (UnsupportedEncodingException uee) { uee.printStackTrace(); }
                catch (IOException ioe) { ioe.printStackTrace(); }


                driver.get("https://x.atlassian.net/secure/Dashboard.jspa?selectPageId=11502");
                Thread.sleep(3000);  // Let the user actually see something!
                ((ChromeDriver) driver).findElementById("menu-sign-in").click();
                Thread.sleep(3000);
                ((ChromeDriver) driver).findElementById("google-signin-button").click();
                Thread.sleep(3000);
                ((ChromeDriver) driver).findElementById("identifierId").sendKeys("email@email.com");
                Thread.sleep(700);
                ((ChromeDriver) driver).findElementById("identifierNext").click();
                Thread.sleep(2000);
                driver.findElement(By.cssSelector("input[name=password]")).sendKeys(Secret.getPassword());
                Thread.sleep(600);
                ((ChromeDriver) driver).findElementById("passwordNext").click();
                Thread.sleep(10000);
                driver.get(url);
                Thread.sleep(56000);
                File[] files = new File("/Users/user/automation-testing").listFiles(new FileFilter() {
                    @Override
                    public boolean accept(File path) {
                        if (path.isFile()) {
                            ParseCSV.doParse(path);
                            path.delete();
                            return true;
                        }
                        else{
                            System.out.println("Failure");
                        }
                        return false;
                    }
                });
                driver.quit();
            }
        }catch( java.lang.InterruptedException inter ){ System.err.println("Thread.sleep broke something, wtf"); inter.printStackTrace(); }
    }
}

score 0 · Answer 3 · edited Dec 01 '20 at 10:39

0

On python's selenium library I've added to the webdriver.ChromeOptions() object the "headless" argument so it won't show the chrome window - Means no download prompt.

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('headless')

edited Dec 01 '20 at 10:39

Dharman

21,838
18
57
107

answered Dec 01 '20 at 10:33

Yuval Press

1

Automatically download files using selenium Chromedriver 3.11.0 on a Mac

3 Answers3

Update