Akka DeathWatch - Find reason for termination

Question

Question: How can I find out if an actor was stopped gracefully (e.g. through its parent stopping) or through an exception?

Context: With the following deathwatch setup I only get the Terminated.class message in the good test, where I explicitly call stop. I expected a Terminated.class message only in the bad case. Using a supervisorStrategy that stops the child that threw an exception would make no difference, as this leads to the behaviour of the good test. And there I can't find a way to decide if it was caused by an exception or not.

My test setup is the following:

DeathWatch

public class DeathWatch extends AbstractActor {

    @Override
    public Receive createReceive() {
        return receiveBuilder()
                .matchAny(this::logTerminated)
                .build();
    }

    private <P> void logTerminated(final P p) {
        log.info("terminated: {}", p);
    }
}

Actor

public class MyActor extends AbstractActor {

    @Override
    public Receive createReceive() {
        return receiveBuilder()
                .matchEquals("good", s -> { getContext().stop(self()); })
                .matchEquals("bad", s -> { throw new Exception("baaaad"); })
                .build();
    }
}

Test

public class Test {

    private TestActorRef<Actor> actor;

    @Before
    public void setUp() throws Exception {
        actor = TestActorRef.create(ActorSystem.create(), Props.create(MyActor.class), "actor");
        TestActorRef.create(ActorSystem.create(), Props.create(DeathWatch.class),"deathwatch").watch(actor);
    }

    @Test
    public void good() throws Exception {
        actor.tell("good", ActorRef.noSender());
    }

    @Test
    public void bad() throws Exception {
        actor.tell("bad", ActorRef.noSender());
    }
}

Update: Adding the following supervisor, leads to a second logging of "terminated", but yields no further context information.

public class Supervisor extends AbstractActor {

    private final ActorRef child;

    @Override
    public Receive createReceive() {
        return receiveBuilder()
                .match(String.class, s -> child.tell(s, getSelf()))
                .build();
    }

    @Override
    public SupervisorStrategy supervisorStrategy() {
        return new OneForOneStrategy(DeciderBuilder.match(Exception.class, e -> stop()).build());
    }
}

Answer 1

The Terminated message is behaving as expected. From the documentation:

In order to be notified when another actor terminates (i.e. stops permanently, not temporary failure and restart), an actor may register itself for reception of the Terminated message dispatched by the other actor upon termination.

And here:

Termination of an actor proceeds in two steps: first the actor suspends its mailbox processing and sends a stop command to all its children, then it keeps processing the internal termination notifications from its children until the last one is gone, finally terminating itself (invoking postStop, dumping mailbox, publishing Terminated on the DeathWatch, telling its supervisor)....

The postStop() hook is invoked after an actor is fully stopped.

The Terminated message isn't reserved for the scenario in which an actor is stopped due to an exception or error; it comes into play whenever an actor is stopped, including scenarios in which the actor is stopped "normally." Let's go through each scenario in your test case:

1. "Good" case without an explicit supervisor: MyActor stops itself, calls postStop (which isn't overridden, so nothing happens in postStop), and sends a Terminated message to the actor that's watching it (your DeathWatch actor).

2. "Good" case with an explict supervisor: same as 1.

3. "Bad" case without an explicit supervisor: The default supervision strategy is used, which is to restart the actor. A restart does not trigger the sending of a Terminated message.

4. "Bad" case with an explicit supervisor: the supervisor handles the Exception, then stops MyActor, again launching the termination chain described above, resulting in a Termination message sent to the watching actor.

So how does one distinguish between the "good" and "bad" cases when an actor is stopped? Look at the logs. The SupervisorStrategy, by default, logs Stop failures at the ERROR level.

When an exception is thrown, if you want to do more than log the exception, consider restarting the actor instead of stopping it. A restart, unlike a stop, always indicates that something went wrong (as mentioned earlier, a restart is the default strategy when an exception is thrown). You could place post-exception logic inside the preRestart or postRestart hook.

Note that when an exception is thrown while an actor is processing a message, that message is lost, as described here. If you want to do something with that message, you have to catch the exception.

If you have an actor that you want to inform whenever an exception is thrown, you can send a message to this monitor actor from within the parent's supervisor strategy (the parent of the actor that can throw an exception). This assumes that the parent actor has a reference to this monitor actor. If the strategy is declared inside the parent and not in the parent's companion object, then the body of the strategy has access to the actor in which the exception was thrown (via sender). ErrorMessage below is a made-up class:

override val supervisorStrategy =
  OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute) {
    case t: Throwable =>
      val problemActor = sender
      monitorActor ! ErrorMessage(t, problemActor)
      Stop
}