Deserializing nested firebase firestore classes on Android

Question

Issue Description

I'm trying to deserialize a simple firebase firestore document for an android app.
The issue is with kotlin nested classes deserialization. And it should be a pretty simple one but it somehow just does not seem to work.

I have a class which would like to store as a kotlin data class the contents of a map on the remote document.
Even though the @PropertyName-supplied field names match with the associated class property names, and the non-nested class (called Bucket) receives all of the "primitive" data correctly, its nested classes just seem not to be filled in with the remote data, falling back to their empty-constructor values you must provide, thus creating a garbage object.

Implementation Details

The Firebase Bucket document data structure is as follows:

I created the following class to represent the whole document:

@Parcelize
data class Bucket(
    @PropertyName("id")
    val id: String,

    @PropertyName("name")
    val name: String,
    @PropertyName("description")
    val description: String,

    @PropertyName("bucket_style")
    val style: BucketStyle,

    @PropertyName("bucket_dates")
    val dates: BucketDates
) : Parcelable {
    constructor() : this("mock.id", "mock.name", "mock.desc", BucketStyle(), BucketDates(), listOf())
}

As an example, with the BucketStyle class:

@Parcelize
data class BucketStyle(
    @PropertyName("icon_family")
    val iconFamily: Int,

    @PropertyName("icon_id")
    val iconId: Int,

    @PropertyName("color_id")
    val colorId: Int
) : Parcelable {
    constructor() : this(0, 0, 1)
}

And symmetrically:

@Parcelize
data class BucketDates(
    @PropertyName("creation_date_timestamp")
    val creationDate: Timestamp,

    @PropertyName("last_edit_timestamp")
    val lastEdit: Timestamp
) : Parcelable {
    constructor() : this(Timestamp.now(), Timestamp.now())
}

And I deserialize it (in reality all the documents in a collection using a real-time listener) using the default firebase-provided method (28.4.2 version of firebase-bom)

buckets.addSnapshotListener { data, error -> 
    val update: List<Bucket> = data.toObjects(Bucket::class.java)
}

And the inner classes are not populated, as shown in this debugger, as the server value differs, and they use their default one, while IDs, names and everything else "primitive" works correctly.

I do not get errors, just a logcat warning while executing:

W/Firestore: (23.0.4) [CustomClassMapper]: No setter/field for bucket_style found on class com.[redacted].model.bucket.Bucket
W/Firestore: (23.0.4) [CustomClassMapper]: No setter/field for bucket_dates found on class com.[redacted].model.bucket.Bucket

Additional Analysis

Those logs perplexed me, because I really have created getters and setters since Bucket is a data class. Or have I?

I decompiled the kotlin file to java and I realized that there is no getter named getbucket_style() for our object:

@PropertyName("bucket_style")
@NotNull
private final BucketStyle style;

@NotNull
public final BucketStyle getStyle() {
    return this.style;
}

We only have getStyle().

We can try to do a trick by renaming the kotlin style property to bucket_style (To match firestore document field name) this way in Bucket:

@PropertyName("bucket_style")
val bucket_style: BucketStyle,

The java decompilation becomes:

@PropertyName("bucket_style")
@NotNull
private final BucketStyle bucket_style;

@NotNull
public final BucketStyle getBucket_style() {
    return this.bucket_style;
}

But the warning stays constant. The generated getter has an uppercase "B" (that's the syntax convention for generating getters in kotlin data classes).
However, if I create a custom getter named getbucket_style() in the original Bucket, as in:

@Parcelize
data class Bucket(
    @PropertyName("id")
    val id: String,

    @PropertyName("name")
    val name: String,
    @PropertyName("description")
    val description: String,

    @PropertyName("bucket_style")
    val style: BucketStyle,

    @PropertyName("bucket_dates")
    val dates: BucketDates
) : Parcelable {

    fun getbucket_style() = style

    constructor() : this("mocker.id", "mocker.name", "mocker.desc", BucketStyle(), BucketDates(), listOf())
}

Whose java decompilation looks like:

@PropertyName("bucket_style")
@NotNull
private final BucketStyle style;


@NotNull
public final BucketStyle getbucket_style() {
    return this.style;
}

@NotNull
public final BucketStyle getStyle() {
    return this.style;
}

Then the warning changes!

W/Firestore: (23.0.4) [CustomClassMapper]: No setter/field for style found on class com.[redacted].model.bucket.Bucket (fields/setters are case sensitive!)

I have no idea why. Also note that on the first warning, the property is referred to as bucket_style, and here as style.

If I do the same trick as before, renaming style to bucket_style, we get this:

@Parcelize
data class Bucket(
    @PropertyName("id")
    val id: String,

    @PropertyName("name")
    val name: String,
    @PropertyName("description")
    val description: String,

    @PropertyName("bucket_style")
    val bucket_style: BucketStyle,

    @PropertyName("bucket_dates")
    val dates: BucketDates
) : Parcelable {

    fun getbucket_style() = bucket_style

    constructor() : this("mock.id", "mock.name", "mock.desc", BucketStyle(), BucketDates(), listOf())
}

Whose java decompilation looks like

@NotNull
private final String description;

@PropertyName("bucket_style")
@NotNull
public final BucketStyle getbucket_style() {
    return this.bucket_style;
}

@NotNull
public final BucketStyle getBucket_style() {
    return this.bucket_style;
}

This crashes without warnings, because (I believe) there are two functions with the same, case-insensitive name:

E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.[redacted], PID: 18996
    java.lang.RuntimeException: Found conflicting getters for name getbucket_style on class com.[redacted].model.bucket.Bucket

Concolusions

The alternative would be to use kotlin Maps to model the nested firestore maps, but classes are much nicer to work with.
I also tried to un-parcele all the classes but the problem persists.
Is there a clean solution / something wrong with the implementation?

EDIT

The workaround is changing the class properties names to match the firestore document field names.

My issue is a special case of a more general bug. It isn't true that deserialization doesn't work for subclasses. It does't work for any class field whose name does not match firebases (in my case).

Indeed, all I had to do was to change the map name from "bucket_style" to "style", and the inner fields (in the now "style" map) to use camel casing (to match my app declarations) and voila, data is now going where it's supposed to. Now I'm using the same names on the backend and on my model, so that the deserializer doesn't get confused.

Still, I don't know how this doesn't work. I believe it is something I am triggering (I refuse to believe that up until now nobody had this issue and reported it), even though my app is still small, follow de facto guidelines and uses firebase exclusively for firestore.

Answer 1

TLDR;

1. The CustomClassMapper of Firestore is checking for getters. (Simplified) You need to apply the annotations to the getter methods of your properties OR have the same naming in fields and getters OR public fields.

2. @PropertyName should be used for serializing (client -> cloud).(See docu) In deserializing the mapper looks for setters and fields and naming must match the naming of the map from api.

3. Do not deserialize in a class where the properties are not public NOR writeable, IF the names do not match. You need setters for this. You can use immutable val property: String, but then the names MUST match.

4. A @PropertyName annotation used on an immutable val property with a different name value (in the annotation) than the field itself, will not work and is ignored. Reason: If it wouldn't be ignored, you would end up serializing multiple entries for the same property.

---

Do I think this is intuitive? No. Absolutely not and it should be documented. But anyway, lets go ->

Does not work:

@PropertyName("my_property")
val myProperty: String

Does work (serializing):

@get:PropertyName("my_property")
val myProperty: String

Does work by accident:

@PropertyName("myProperty")
val myProperty: String

I will explain this later, but first lets discuss your case here:

You applied your @PropertyName annotation without a use-site target, so we have to check the @Target of the annotation (PropertyName) itself. This tells us, it's applicable to methods and fields. Hence, without a use-site target, it will be applied to the field, not to the getter.

The mapper will however first check all the getters and their annotations and ONLY check the fields and their annotations, if they are either:

1. public or
2. their resolved property name matches the one of the getter method

Simplified basics

A kotlin data class with immutable properties will create a backing field and a getter method of that field.

If you want to apply an annotation directly on the field or getter, you have to use Annotation use-site targets, so either

@get:AnnotationClassname or @field:AnnotationClassname

while @AnnotationClassname without target, will fallback to the field here (it depends on the annotation).

How does the mapper parse a class (simplified):

Add public getters:

1. Get all getter methods
2. Try to resolve the name by the method.name OR getter @PropertyName value
3. Store this name in a list (it's a map, but this does not matter here)
4. Now check and store all the getter annotations (document id, ServerTimestamp..)

Add public fields:

1. Get all public accessible fields
2. Try to resolve the name by the field.name OR field @PropertyName value
3. Store in list of properties

Add private (backing) fields (Important part):

1. Get all fields (public, private, not inherited)
2. Try to resolve the name by the field.name OR field @PropertyName value
3. Store in list of FIELDS - IF it's ALREADY in list of PROPERTIES

Here you can see how in 3. the fields are ignored if not already known:

Okay. Now we know how it's parsed. If you add an annotation like this, we remember->

Does not work:

@PropertyName("my_property")
val myProperty: String

The reason this does not work

While parsing the mapper will see a public getter, that has no annotation and store a value inside his property list named -> "myProperty".

The mapper then will see the private field and also see the fields PropertyName annotation, resolving the name to "my_property", which IS NOT inside the properties list and hence, the field and also all of it's annotations (PropertyName, ServerTimestamp, DocumentId) are IGNORED.

Why does this work when serializing:

@get:PropertyName("my_property")
val myProperty: String

Because the mapper will only take the getter into account, store "my_property" in properties list and ignore the field. In fact, this will not work when deserializing, because myProperty will not be recognized as a field, and thus not be used. (This will show a "no setter / field" warning).

Why does this work by accident:

@PropertyName("myProperty")
val myProperty: String

Because the mapper actually sees no difference in resolved getter or field property name. The mapper will store a "myProperty" property and serialize it as such. All annotations will be honored. This is only true for serializing though. Deserializing will show a "no setter / field" warning.

Disclaimer:

This is my understanding of the topic. Probably I still have mistakes here and there and for sure I lack knowledge. If you find a mistake add a comment.

I hope this helps someone.

Source: CustomClassMapper

Answer 2

I suspect this is a bug with Firestore's @PropertyName annotation and Kotlin's data classes and you'll have to find a workaround.

The first thing to note is that Kotlin data classes do not have setters. The value with which a property is initialized in its constructor cannot be changed. Many JSON deserializers are used to the "Java way" of doing things: look for a no-argument constructor, invoke it, and then look for setters to set each field found in the JSON document. The error No setter/field for bucket_style found on class com.[redacted].model.bucket.Bucket tells us that Firestore is invoking the no-arg constructor (with your defaults) and then looking for setters.

What's curious is that Firestore is correctly setting fields on the parent object, Bucket, which also has a no-arg constructor.

A few things worth investigating:

• Does Firestore correctly deserialize a BucketStyle object if it is directly stored with Firebase (e.g. if you add an id and store it un-nested in Firestore)?

• Does the problem resolve if you delete your no-arg constructor (with your defaults) on BucketStyle? This may force the Firestore deserializer to use your constructor with named arguments.

• Does the problem resolve if you remove data from your class declaration and change the val fields to var fields? Under the hood, this should create setters for each field.