Why is it wrong to use Deferred.result to get the result of a Twisted Deferred object?

Question

From what I've read, one should never use the result attribute of a Twisted Deferred object to access the value of the Deferred. I suspect the reasoning for this is that either

1. No result may not be available at the time of access (raises AttributeError)
2. The result may not be final at the time of access (i.e. not all callbacks have run)

Is there ever a situation where it is appropriate to access the value of the result of the Deferred? Is there a better way to access the result to assign it to a variable or use it later without adding additional callbacks to the Deferred?

Answer 1

Your suspicions are essentially correct.

No result may not be available at the time of access (raises AttributeError)

The idea of Deferred is that some work is happening and it won't be finished by some fixed time. It will be finished when it is finished. Fortunately, when it is finished, Deferred can tell you about the result - this is what callbacks are for. This means that, without using callbacks, you can't know when the work is finished. You might check too early and get an AttributeError (or failure of another sort, more on this below). You might check too late and have wasted some time. You can "fix" the too-early case by checking repeatedly and handling the error. This is called "polling". Large parts of Twisted exist to remove the need to perform polling because polling is typically undesirable (it's expensive). The only way to "fix" the too-late case is by checking more frequently - which makes the "too-early" case worse.

The result may not be final at the time of access (i.e. not all callbacks have run)

A feature Deferred offers is that it allows for composition in event-driven programming. You can have unit A which performs some task and returns a Deferred. You can have unit B which performs some task and depends on unit A and returns some other Deferred. Such composition could look like this:

d = Deferred()

d_a = unit_a()
def a_finished(result):
    d_b = unit_b(result)
    d_b.addCallback(d_final.callback)
d_a.addCallback(a_finished)

What values does d.result take on over the course of the execution of this example? First it is an AttributeError and then it is whatever value the unit_b Deferred is called back with. In this case, you have no incomplete or intermediate results being exposed and there's no problem. However, this composition is awkward, verbose, and failure-prone (it misses error propagation and cancellation features, for example).

The easy, idiomatic, encouraged way to compose Deferred APIs is:

d = unit_a()
d.addCallback(unit_b)

Easier to write, easier to read, and you get features like railroad-style error propagation.

What values does d.result take on in this case, though? First it is an AttributeError. Then, after unit_a is done, d.result is the Deferred returned by unit_b. So if you look it up here, not only do you not have the final result but you don't even have a real value, you have a Deferred instead. Only after unit_b is done does d take on the real result of unit_b.

There are other possible sequences as well. They arise from other Deferred usage patterns that aren't as preferred as the one above but they are certainly possible. For example, you might have an implementation which does:

d = unit_a()

and then have d exposed to your code. After that point you might have the implementation proceed to:

d.addCallback(unit_b)

Now d.result proceeds from AttributeError to the result of unit_a to a Deferred to the result of unit_b. The above is not excellent use of Deferred but as you can see, it is entirely possible. If you are polling d.result in this case, you have the complication of having to guess whether a non-Deferred value is the result of unit_a or of unit_b.

Is there ever a situation where it is appropriate to access the value of the result of the Deferred?

It's hard to completely rule it out. Certainly in the test suite for Deferred itself there are some direct accesses and these may be legitimate. It's possible that in some other test-related tools it may be appropriate to access the result attribute this way - but ideally even there it would be avoided or testing libraries (such as Twisted's trial) would provide better tools for writing these kinds of tests (eg trial does provide successResultOf, failureResultOf, and assertNoResult). Beyond those cases, I think you should look very, very carefully at any use of the result attribute and you will probably find that there is a better (more maintainable, less fragile) solution using callbacks.

Is there a better way to access the result to assign it to a variable or use it later without adding additional callbacks to the Deferred?

The better way to access the result is always to add an additional callback. As another answer suggests, inlineCallbacks is a way to use callbacks that doesn't look like using callbacks and is preferred by many. It lets you do a simple assignment to a local to retrieve a result but the implementation is still using Deferred.addCallback and the related APIs.

Answer 2

I think this is what you are looking for. With inlinecallbacks you are able to access the result without the need of writing callbacks.

@inlineCallBacks def thingummy(): thing = yield makeSomeRequestResultingInDeferred() print(thing) # the result! hoorj!

https://twistedmatrix.com/documents/13.2.0/api/twisted.internet.defer.inlineCallbacks.html