17

I am trying to implement hystrix for my application using hystrix-javanica.

I have configured hystrix-configuration.properties as below

hystrix.command.default.execution.isolation.strategy=SEMAPHORE
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=10000 
hystrix.command.default.fallback.enabled=true
hystrix.command.default.circuitBreaker.enabled=true
hystrix.command.default.circuitBreaker.requestVolumeThreshold=3 
hystrix.command.default.circuitBreaker.sleepWindowInMilliseconds=50000
hystrix.command.default.circuitBreaker.errorThresholdPercentage=50

short-circuit pattern is working fine but i have a doubt in this hystrix.command.default.circuitBreaker.requestVolumeThreshold=3

  1. Is it stating open the circuit after 3 failures or
  2. Open the circuit after 3 concurrent failures.

Gone through the documentation link

Can anybody answer?

 

 

Jay
  • 379
  • 2
  • 5
  • 21

2 Answers2

50

How Hystrix Circuit-Breaker operates: Hystrix does not offer a circuit breaker which breaks after a given number of failures. The Hystrix circuit will break if:

within a timespan of duration metrics.rollingStats.timeInMilliseconds, the percentage of actions resulting in a handled exception exceeds errorThresholdPercentage, provided also that the number of actions through the circuit in the timespan is at least requestVolumeThreshold

What is requestVolumeThreshold? requestVolumeThreshold is a minimum threshold for the volume (number) of calls through the circuit that must be met (within the rolling window), before the circuit calculates a percentage failure rate at all. Only when this minimum volume (in each time window) has been met, will the circuit compare the failure proportion of your calls against the errorThresholdPercentage you have configured.

Imagine there was no such minimum-volume-through-the-circuit threshold. Imagine the first call in a time window errors. You would have 1 of 1 calls being an error, = 100% failure rate, which is higher than the 50% threshold you have set. So the circuit would break immediately.

The requestVolumeThreshold exists so that this does not happen. It's effectively saying, the error rate through your circuit isn't statistically significant (and won't be compared against errorThresholdPercentage) until at least requestVolumeThreshold calls have been received in each time window.

mountain traveller
  • 5,180
  • 17
  • 24
  • Bit confused with the rolling window. Can you please explain bit more? According to my configuration circuit should be open after 3 failures? Isn't it? – Jay Jul 22 '16 at 12:30
  • 3
    See https://github.com/Netflix/Hystrix/wiki/How-it-Works#circuit-breaker for more detail on how the circuit breaker works. The 3 you have configured is not a number of failures on which to break. Hystrix breakers break on _percentage_ of errors (the `errorThresholdPercentage=50`% you have configured), considered across calls in a given time window. The `requestVolumeThreshold=3` is (per my original answer) a minimum-volume-threshold of calls, which must be met in the same time window, to make the %age calculations statistically significant. – mountain traveller Jul 22 '16 at 12:47
  • See https://github.com/Netflix/Hystrix/wiki/Configuration#metricsrollingstatstimeinmilliseconds for how the time windows is configured. – mountain traveller Jul 22 '16 at 12:47
  • 1
    For a circuit-breaker operating by the same principles, concisely explained, you can also see here: https://github.com/App-vNext/Polly/wiki/Advanced-Circuit-Breaker#the-advancedcircuitbreaker-v42 Though the configuration terminology used is different (and it's a .NET not Java product), the operating principles are the same. The Polly page also offers some explanations of how the various configuration parameters interact (eg what happens if you set them too high, too low etc) – mountain traveller Jul 22 '16 at 12:52
  • @mountain traveller: My first request error itself is calling my fallback method. I dint configure "requestVolumeThreshold" explicitly. Can you help me? I have just followed https://spring.io/guides/gs/circuit-breaker/ – Guna Nov 09 '17 at 13:58
1

I am rather new to hystrix but I guess I can help you.

In general hystrix.command.default.circuitBreaker.requestVolumeThreshold is a property that sets the minimum number of requests in a rolling window that will trip the circuit and its default value is 20 and its value can be changed in properties file or in our @HystrixCommand annotated method.

For example, if that property value is 20, then if only 19 requests are received in the rolling window (say a window of 10 seconds) the circuit will not trip open even if all 19 failed. If the failed request value reaches 20, then the circuit will be opened and the corresponding calls will be sent to fallback even if the call succeeds, till the sleeping window time period complete.

Sleeping window time period sets the amount of time, after tripping the circuit, to reject requests before allowing attempts again to determine if the circuit should again be closed. Its value is defaulted to 5000 milliseconds. This can be changed by overriding circuitBreaker.sleepWindowInMilliseconds property.

You can find all the properties and its description here.

informatik01
  • 15,174
  • 9
  • 67
  • 100
Sumanth Duvvuru
  • 181
  • 1
  • 6
  • You may want to look at: https://github.com/Netflix/Hystrix/wiki/Configuration#circuitbreakererrorthresholdpercentage. Your answer is incorrect, in taking no account of `errorThresholdPercentage`. Hope this helps. See https://github.com/Netflix/Hystrix/blob/master/hystrix-core/src/main/java/com/netflix/hystrix/HystrixCircuitBreaker.java#L191 and https://github.com/Netflix/Hystrix/blob/master/hystrix-core/src/main/java/com/netflix/hystrix/HystrixCircuitBreaker.java#L196 – mountain traveller Nov 23 '16 at 23:45
  • Is there any way to verify those configuration properties to make sure these properties are really applied to configure HystrixCommand? – pijushcse Feb 23 '18 at 20:54