Caching DNS queries in Ruby

Question

We are using the rest-client gem in Ruby for automated testing of our REST API. However I noticed that on every single request it makes, it also does a DNS lookup for the host name. On the local environment if "localhost" is used the tests run fast, but if the proper hostname is used they take 2.5x the time, performing a huge number of DNS lookups.

I believe this issue is not related to rest-client in particular, but the base Ruby networking. I tried requiring 'resolv' and 'resolv-replace' but they did not help. 'dig' reports that the DNS query has a TTL of 1 hour.

Is there a way to make Ruby cache DNS requests? I could change the code to explicitly use the IP address, but that's the wrong place to fix the issue.

I'm running Ubuntu 12.04 and Ruby 1.9.3.

I'm interested in caching DNS queries too. Could you get it to work? — Jong Bor Lee, Nov 21 '12 at 22:06
Haven't found a way. We worked around it by using the IP address. — Sampo, Nov 22 '12 at 06:36
Possible solution: http://stackoverflow.com/questions/11020027/dns-caching-in-linux — Kelvin, Jan 17 '14 at 17:52

score 2 · Answer 1 · edited Dec 15 '14 at 02:21

2

You can use the dnsruby gem to resolve the name to an address, and then use the address in your calls.

#! /usr/bin/env ruby

# Gets the IP address of a host.

require 'dnsruby'  # gem install dnsruby first, of course

def hostname_to_ip_addr(host_name)
  query = Dnsruby::Message.new(host_name)
  response = Dnsruby::Resolver.new.send_message(query)
  response.answer[1].address
end

host_name = 'cnn.com'
ip_addr = hostname_to_ip_addr(host_name)
puts("Host name: #{host_name}, IP address: #{ip_addr}")

^{original code from this Gist}

edited Dec 15 '14 at 02:21

William Price

3,677
1
30
46

answered Dec 15 '14 at 01:51

Keith Bennett

21
3

Welcome to Stack Overflow! While this may answer the question, [it would be preferable in future](http://meta.stackoverflow.com/q/8259) to include the essential parts of the answer here, and provide the link for reference. (It looks like someone is probably already editing the code in for you.) – Nathan Tuggy Dec 15 '14 at 02:15

score 2 · Answer 2 · answered Feb 12 '18 at 16:04

I got to this question looking for ruby dns caching and how resolv.rb might use the TTL to cache dns requests.

Discovering the TTL of a dns record is a bit hidden in the api of resolv.rb but it looks a little something like this:

def get_ip(hostname)
  dns = Resolv.new
  redis = Redis.new # storing in redis
  ip = redis.get("ip:#{hostname}") 
  return ip unless ip.nil?
  begin
    resource = dns.getresource(hostname, Resolv::DNS::Resource::IN::A)
  rescue Resolv::ResolvError
    return false
  end
  # storing in redis for only as long as the TTL allows
  redis.setex("ip:#{hostname}", resource.address.ttl, resource.address.to_s)
  resource.address.to_s # IP address as string
end

Gist

Note: uses Redis as the cache.

score 1 · Answer 3 · answered Apr 03 '14 at 22:17

OS usually provides some dns name caching, but it is not always enabled.

From my understanding ruby uses libc to resolve names. So this might cache responses per-process. i'm not entirely sure about libc, though.

A possible workaround that you may be able to try is to manually resolve the hostname to ip address in your ruby program and then use the returned IP address for repeated requests to the same hosts.

This may not work as nicely for connections to SSL targets, though. Certificate validation often depends on matching a hostname.

Maybe there is a nice ruby network socket gem that doesn't use Net::HTTP and will keep it's own DNS cache.

score 0 · Answer 4 · answered Aug 30 '12 at 21:11

I've just taken a look at the rest-client code and it's just using Net::HTTP, which in turn uses Socket.

Then everything disappears into the Ruby interpreter implementation, which is where my knowledge goes a bit weak (and behaviour will perhaps change depending on whether you're using MRI, JRuby, etc.)

I'd expect DNS resolution to defer to the OS eventually, so perhaps your problem is down to some odd resolver configuration on the host?

Caching DNS queries in Ruby

4 Answers4