2

I'm using yii2-elasticsearch library that handles all request to elasticsearch with curl. The problem is that after 30000-40000 requests (aprox. speed is 18000rpm) I beging to receive curl error 7 (CURLE_COULDNT_CONNECT) on some of requests. I set all elasticsearch log levels to DEBUG and TRACE but it doesn't report any problems, so I think that this issue is related to curl itself and may be connected with some of OS settings (Ubuntu 14.04).

What can cause such kind of behavior?

user2864740
  • 54,112
  • 10
  • 112
  • 187
Lisio
  • 1,461
  • 2
  • 14
  • 19
  • 30-40k requests sounds like it's using up all available sockets / address tuples (interesting reading http://stackoverflow.com/questions/14388706.) If this is the case I would expect that it 'starts working 100% again' after pausing for say, 5 minutes. – user2864740 Aug 25 '15 at 19:20
  • Yes, it looks very close to this since only part of these requests fails, the rest of them succeeds. If I stop script and run it again after a minute all of the following 30-40k requests succeed. But curl in php reuses connections by default so I don't see a reason why it fails. – Lisio Aug 25 '15 at 19:35
  • 1
    Also netstat reports 27k connections to 172.17.0.9:9200 (which is elasticsearch port in a docker container) and another 27k connections to 127.0.0.1:49200 (which is linked to container's port 9200). All these connections are in TIME_WAIT state, even if I set CURLOPT_FORBID_REUSE and CURLOPT_FRESH_CONNECT to true. – Lisio Aug 25 '15 at 19:53
  • Move that information up into the main questions where it will be more visible. – user2864740 Aug 25 '15 at 19:54
  • 1
    Looks like "sysctl net.ipv4.tcp_tw_reuse=1" fixed the issue. Now netstat reports 14k-16k connections with total of 28-32k. Not sure that this setting is safe enough for production. – Lisio Aug 25 '15 at 20:05

0 Answers0