CODEWITHSUNDEEP
javascalaakkahystrix
Mukiza Andrew
Mukiza Andrew

Reputation: 763

When Hystrix short-circuits and the circuit is OPEN it never closes even after the the offending service is back up

I am having an Issue where when the hystrix circuit breaker trips, it does not close ever again. I have turned logging to debug and I do not see it trying to allow a test request through in which case it seems to me It will never close since its only supposed to close when a test execution goes through successfully indicating that the offending service is now healthy. According to the documentation the Circuit break configuration defaults should be working but I can not seem to tell why the test request is never allowed through.

2016-02-18 09:00:38,782 noodle-soup-service application-akka.actor.default-dispatcher-7 ERROR akka.actor.OneForOneStrategy - CallServiceCommand short-circuited and fallback failed.
com.netflix.hystrix.exception.HystrixRuntimeException: CallServiceCommand short-circuited and fallback failed.
    at com.netflix.hystrix.AbstractCommand$16.call(AbstractCommand.java:816) ~[com.netflix.hystrix.hystrix-core-1.4.23.jar:1.4.23]
    at com.netflix.hystrix.AbstractCommand$16.call(AbstractCommand.java:790) ~[com.netflix.hystrix.hystrix-core-1.4.23.jar:1.4.23]
    at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$1.onError(OperatorOnErrorResumeNextViaFunction.java:99) ~[io.reactivex.rxjava-1.1.0.jar:1.1.0]
    at rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:71) ~[io.reactivex.rxjava-1.1.0.jar:1.1.0]

...

Caused by: java.lang.RuntimeException: Hystrix circuit short-circuited and is OPEN
    at com.netflix.hystrix.AbstractCommand$1.call(AbstractCommand.java:414) ~[com.netflix.hystrix.hystrix-core-1.4.23.jar:1.4.23]
    ... 38 common frames omitted

Upvotes: 5

Views: 23349

Answers (1)

Mukiza Andrew
Mukiza Andrew

Reputation: 763

In the Akka error handling strategy, I needed to schedule a new test call so that eventually when it succeeds, then Hystrix can close the Circuit.

Upvotes: 1

Related Questions