How to make a HTTP request using Ruby on Rails?

Question

I would like to take information from another website. Therefore (maybe) I should make a request to that website (in my case a HTTP GET request) and receive the response.

How can I make this in Ruby on Rails?

If it is possible, is it a correct approach to use in my controllers?

score 341 · Accepted Answer · edited Jun 22 '14 at 12:11

341

You can use Ruby's Net::HTTP class:

require 'net/http'

url = URI.parse('http://www.example.com/index.html')
req = Net::HTTP::Get.new(url.to_s)
res = Net::HTTP.start(url.host, url.port) {|http|
  http.request(req)
}
puts res.body

edited Jun 22 '14 at 12:11

Martin Tournoij

23,567
24
90
127

answered Jan 02 '11 at 23:29

João Silva

81,431
26
144
151

1

what does the 'req' mean here? – sixty4bit Sep 04 '14 at 15:20
1

Looks like this might be a blocking request, would it not? – Scott Eisenberg Apr 17 '15 at 17:02
where to put the api key? – user1735921 Sep 14 '15 at 12:26
@João Silva How can i set a timeout for my request? – Jeff Jun 09 '16 at 19:42
1

Just adding that the www. shouldn't be necessary, it typically isn't. – JackHasaKeyboard Oct 01 '16 at 05:00
What if you get a `Errno::ECONNREFUSED: Failed to open TCP connection to :80` error? – Jwan622 Feb 27 '17 at 16:30
@ScottEisenberg, it is a blocking request, you are right. There are a variety of ways to build an async request. – tschwab Jun 27 '17 at 13:25
This [link](http://www.rubyinside.com/nethttp-cheat-sheet-2940.html) provides several code snippets for use cases of the Net::HTTP module. Very helpful. – Francio Rodrigues Dec 09 '17 at 13:59
I always wondered if _www.example.com_ had anything. Turns out it does! – Joshua Pinter Mar 18 '18 at 02:01
This doesn’t work in all cases; you really should look at the net/http’s manual and look for how you want to do this for your case. – Smar Mar 19 '19 at 08:00

score 114 · Answer 2 · edited Nov 18 '16 at 09:22

114

Net::HTTP is built into Ruby, but let's face it, often it's easier not to use its cumbersome 1980s style and try a higher level alternative:

edited Nov 18 '16 at 09:22

sandstrom

12,776
5
59
60

answered Jan 02 '11 at 23:39

stef

13,499
2
42
66

4

Or ActiveResource, which comes with Rails! – Marnen Laibow-Koser Nov 14 '11 at 22:28
9

I would like to caution against doing so as you will add more dependencies to your rails app. More dependencies means more memory consumption and also potentially larger attack surface. Using `Net::HTTP` is cumbersome but the trade off isn't worth it. – Jason Yeo Apr 12 '16 at 09:34
3

This should be the accepted answer. Why program when you can just install lots of Gems? – omikes Jun 08 '17 at 17:15
Read the comment of Jason Yeo, It's better to avoid a lot dependencies. – Victor Castro Apr 02 '18 at 05:14
5

@JasonYeo Strongly disagree. Introducing dependencies means you don't reinvent the wheel, and you benefit from the hard work others have already done. If a gem exists that makes your life easier, there's generally no good reason not to use it. – Marnen Laibow-Koser Apr 10 '18 at 22:27
@MarnenLaibow-Koser uhuh. Something something leftpad saga. But anyway, I use HTTParty. I love it. It's super easy to use if I am making a request with a form body. But if I am simply making a GET request, I would rather stick to Net::HTTP. In that case, it's not worth it to include 10MB of dependencies to make a small GET request. – Jason Yeo Jun 07 '18 at 08:08
1

@JasonYeo The leftpad saga only happened because NPM ran its repository poorly and let the author delete all his packages. Properly managed package repos don’t do that (and anyway, it’s OSS, so you can easily mirror if you want). That’s is, the leftpad saga is not an argument against introducing dependencies in general, but rather against managing the repo poorly. I do agree with your other point, that a big dependency that does way more than you need can be overkill for the value it provides. – Marnen Laibow-Koser Jun 07 '18 at 13:31

score 96 · Answer 3 · edited Apr 01 '15 at 17:21

96

OpenURI is the best; it's as simple as

require 'open-uri'
response = open('http://example.com').read

edited Apr 01 '15 at 17:21

Andrey Mikhaylov - lolmaus

20,948
5
68
119

answered Jul 14 '13 at 17:52

user2454031

1,269
1
9
7

11

It's important to warn, that `open-uri` won't follow redirects. – yagooar Oct 22 '14 at 10:01
3

@yagooar which is great, prevents malicious redirects like `file:///etc/passwd` – gertas Nov 17 '15 at 15:58
1

Please note, that it will not close connection. Use https://stackoverflow.com/a/4217269/820501 – ShockwaveNN Oct 26 '18 at 14:04

score 83 · Answer 4 · edited Feb 04 '17 at 00:08

83

require 'net/http'
result = Net::HTTP.get(URI.parse('http://www.example.com/about.html'))
# or
result = Net::HTTP.get(URI.parse('http://www.example.com'), '/about.html')

edited Feb 04 '17 at 00:08

kkurian

3,615
3
28
47

answered Jan 02 '11 at 23:23

Andrei Andrushkevich

9,729
4
28
42

Mark Thomas · Answer 5 · 2013-02-11T20:58:45.770

15

I prefer httpclient over Net::HTTP.

client = HTTPClient.new
puts client.get_content('http://www.example.com/index.html')

HTTParty is a good choice if you're making a class that's a client for a service. It's a convenient mixin that gives you 90% of what you need. See how short the Google and Twitter clients are in the examples.

And to answer your second question: no, I wouldn't put this functionality in a controller--I'd use a model instead if possible to encapsulate the particulars (perhaps using HTTParty) and simply call it from the controller.

edited Feb 11 '13 at 20:58

answered Jan 02 '11 at 23:53

Mark Thomas

35,360
9
68
99

And how is it possible to pass safely parameters in the URL? Eg: http ://www.example.com/index.html?param1=test1&param2=test2. Then I need to read from the other website parameters and prepare the responce. But how can I read parameters? – user502052 Jan 03 '11 at 00:01
What do you mean, you need to read the other website's parameters? How would that even be possible? What are you trying to achieve? – Marnen Laibow-Koser Nov 14 '11 at 22:29

score 8 · Answer 6 · edited Dec 17 '16 at 01:25

Here is the code that works if you are making a REST api call behind a proxy:

require "uri"
require 'net/http'

proxy_host = '<proxy addr>'
proxy_port = '<proxy_port>'
proxy_user = '<username>'
proxy_pass = '<password>'

uri = URI.parse("https://saucelabs.com:80/rest/v1/users/<username>")
proxy = Net::HTTP::Proxy(proxy_host, proxy_port, proxy_user, proxy_pass)

req = Net::HTTP::Get.new(uri.path)
req.basic_auth(<sauce_username>,<sauce_password>)

result = proxy.start(uri.host,uri.port) do |http|
http.request(req)
end

puts result.body

the Tin Man · Answer 7 · 2013-06-13T15:50:35.743

8

My favorite two ways to grab the contents of URLs are either OpenURI or Typhoeus.

OpenURI because it's everywhere, and Typhoeus because it's very flexible and powerful.

edited Jun 13 '13 at 15:50

answered Jan 03 '11 at 03:22

the Tin Man

150,910
39
198
279

How to make a HTTP request using Ruby on Rails?

7 Answers7

Linked

Related