CODEWITHSUNDEEP
celerycelery-taskpython-dataclasses
Salim Fadhley
Salim Fadhley

Reputation: 8235

Can I use dataclasses (or similar) as arguments and return values for Celery tasks?

By default, arguments to Celery tasks can accept primitive types, lists and dicts as arguments and return values.

But what if I'd like to send and return more complex values? I'd like to use something like Python 3.7's dataclasses as arguments and return values. Since I'm using Python 3.6 I'm using a backport of the dataclass, but from what I can tell, that's almost the same thing.

When I try to send or return a dataclass instance I get an error that the object is not JSON serializable... which is correct: The JSON serializer doesn't know how to deal with anything like that.

So there is a way to use a custom JSON serializer that can handle dataclass instances as arguments and return values?

Upvotes: 3

Views: 2494

Answers (1)

2ps
2ps

Reputation: 15936

You can do one of two things: (1) You can use the pickle serializer for celery by setting the task_serializer setting or (2) write your own set of serialization/deserialization functions for json. The example at the link above can show you how to write your own serializer.

n.b., using pickle does come with some security concerns, but that should be fine as long as you have only trusted producers.

Upvotes: 3

Related Questions