0

If I use next function to get google output:

function myFunction() {
  var post_url, result;
  post_url = "http://www.google.com/search?q=stack+overflow";
  result = UrlFetchApp.fetch(post_url);
  Logger.log(result);
}

doesn't work.
P.S. Sorry, I have to eхplore some dependences. I take an example 

function scrapeGoogle() {

  var response = UrlFetchApp.fetch("http://www.google.com/search?q=labnol");

  var myRegexp = /<h3 class=\"r\">([\s\S]*?)<\/h3>/gi;

  var elems = response.getContentText().match(myRegexp);

  for(var i in elems) {

    var title = elems[i].replace(/(^\s+)|(\s+$)/g, "")
                        .replace(/<\/?[^>]+>/gi, "");
    Logger.log(title);

  }

}

and it works, than I begin to do some modifications and noticed that when I have some error in code it gives me an error

Request failed for http://www.google.com/search?q=labnol returned code 503.

So I did some researches without error's and it solution works. But when I began to form it to the function in lib it begans to throw me an error of 503 each time!
I'm very amazing of such behavior...
Here is short video only for fact. https://youtu.be/Lem9eiIVY0I
P.P.S. Oh! I've broke some violations, so the google engine send me to stop list so I run this:

function scrapeGoogle() {
  var options =
     { 
       'muteHttpExceptions': true
     }
  var response = UrlFetchApp.fetch("http://www.google.com/search?q=labnol", options);
  Logger.log(response);
}

and get
About this page

Our systems have detected unusual traffic from your computer network. This page checks to see if it's really you sending the requests, and not a robot. Why did this happen?

As I see I have to use some special google services to get the search output and not to be prohibited?

Karen Fisher
  • 643
  • 1
  • 6
  • 21
  • what doesn't work? are you getting an error? you need to provide more info if anyone is to help you. – DLeh Mar 12 '15 at 18:52
  • Possible duplicate of [How do I get Google search results from urlfetch in google apps script](https://stackoverflow.com/questions/29962902/how-do-i-get-google-search-results-from-urlfetch-in-google-apps-script) – Rubén Feb 26 '18 at 00:01

1 Answers1

0

You can use simple regex to extract Google search results.

  var regex = /<h3 class=\"r\">([\s\S]*?)<\/h3>/gi;  
  var items = response.getContentText().match(regex);

Alternatively, you can use the ImportXML function in sheets.

  =IMPORTXML(GOOGLE_URL, "//h3[@class='r']")

See: Scrape Google Search with Sheets

Amit Agarwal
  • 8,542
  • 1
  • 23
  • 35
  • May i know why your example at your website is not working using urlfetchApp ? i want to get title,url and description but need correct regex . – user1788736 Oct 25 '15 at 23:49