Gson. Distinguish null-value field and missing field

Question

I have json model:

{"field1":"0", "field2": "1"}

Sometimes field1 can be null or it can be missing:

{"field1": null, "field2": "1"},
{"field2": "1"}

Can I distinguish if field is null or it is missing?

I don't want to write custom deserializer for full model because real json object is complex.

Answer 1

Gson will first call the default/zero-arg constructor, then populate the object's fields with the Json values by calling setters. So if the value set by the default constructor is not null, a null value would indicate the value was explicitly null. However that requires a default value which can never be set.

One solution is to create POJOs where all the fields are java.util.Optional and the fields are initialized to null. Each getter returns an Optional<T> but the setter accepts just T.

public class OptionalPOJO {
  private Optional<SomeType> someValue;

  public Optional<SomeType> getSomeValue() { return someValue; }
  public void setSomeValue(SomeType newVal) {
    this.someValue = Optional.ofNullable(newVal);
  }
}

public class POJO {
  private SomeType someValue;

  //normal getter and setter
}

Thus a null means the key was missing, an empty Optional means null was explicitly provided, and otherwise the Optional would contain the value given.

You would additionally need to provide a TypeAdapterFactory which unwraps Optionals.

This is a disgusting misuse of Optional. To make it a bit more clean one could create a class similar to Optional but which allows null values.

Answer 2

Sort of. The main problem here is how you can obtain the expected object properties. You can try to detect the expected properties yourself, but Gson provides probably a better but a hack way (and I don't know if it will be available in future versions of Gson). The UnsafeAllocator class is responsible for creating objects without calling their constructors, and, since Gson works on Android, it should work for you as well. This is pretty fine here, because we can find use such a temporary object to convert it to a JSON object and then obtain its keys.

private static final UnsafeAllocator unsafeAllocator = UnsafeAllocator.create();

/**
 * @param gson This Gson instance must have be initialized with {@link GsonBuilder#serializeNulls()}
 */
static Set<String> tryLookupKeys(final Gson gson, final Class<?> clazz)
        throws Exception {
    final Object o = unsafeAllocator.newInstance(clazz);
    final JsonElement jsonElement = gson.toJsonTree(o, clazz);
    if ( !jsonElement.isJsonObject() ) {
        throw new IllegalArgumentException(clazz + " cannot be converted to a JSON object");
    }
    return jsonElement.getAsJsonObject().keySet();
}

Note that it's crucial to have the passed Gson instance to serialize nulls. Another note is that the Gson is not private here, but it's supposed to be passed to the method in order to respect your concrete Gson instance key.

Example of use:

final class FooBarBaz {

    final String foo;
    final int bar;
    final String[] baz;

    FooBarBaz(final String foo, final int bar, final String[] baz) {
        this.foo = foo;
        this.bar = bar;
        this.baz = baz;
    }

}

private static final Gson gson = new GsonBuilder()
        .serializeNulls()
        .create();

final Set<String> expectedKeys = JsonProperties.tryLookupKeys(gson, FooBarBaz.class);
System.out.println("keys: " + expectedKeys);
System.out.println(Sets.difference(expectedKeys, gson.fromJson("{\"foo\":\"foo\",\"bar\":1,\"baz\":[]}", JsonObject.class).keySet()));
System.out.println(Sets.difference(expectedKeys, gson.fromJson("{\"foo\":\"foo\",\"bar\":1,\"baz\":null}", JsonObject.class).keySet()));
System.out.println(Sets.difference(expectedKeys, gson.fromJson("{\"foo\":\"foo\",\"bar\":1}", JsonObject.class).keySet()));

Output:

keys: [foo, bar, baz]
[]
[]
[baz]

Part 2

You can use this approach to detect "incomplete" JSON payloads by writing a custom type adapter. For example:

private static final Gson gson = new GsonBuilder()
        .serializeNulls()
        .registerTypeAdapterFactory(AllKeysRequiredTypeAdapterFactory.get())
        .create();

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@interface AllKeysRequired {
}

final class AllKeysRequiredTypeAdapterFactory
        implements TypeAdapterFactory {

    private static final TypeAdapterFactory allKeysRequiredTypeAdapterFactory = new AllKeysRequiredTypeAdapterFactory();

    private AllKeysRequiredTypeAdapterFactory() {
    }

    static TypeAdapterFactory get() {
        return allKeysRequiredTypeAdapterFactory;
    }

    @Override
    public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
        @SuppressWarnings("unchecked")
        final Class<T> rawType = (Class<T>) typeToken.getRawType();
        // Or any other way you would like to determine if the given class is fine to be validated
        if ( !rawType.isAnnotationPresent(AllKeysRequired.class) ) {
            return null;
        }
        final TypeAdapter<T> delegateTypeAdapter = gson.getDelegateAdapter(this, typeToken);
        final TypeAdapter<JsonElement> jsonElementTypeAdapter = gson.getAdapter(JsonElement.class);
        return AllKeysRequiredTypeAdapter.of(gson, rawType, delegateTypeAdapter, jsonElementTypeAdapter);
    }

    private static final class AllKeysRequiredTypeAdapter<T>
            extends TypeAdapter<T> {

        // This is for the cache below
        private final JsonPropertiesCacheKey jsonPropertiesCacheKey;
        private final TypeAdapter<T> delegateTypeAdapter;
        private final TypeAdapter<JsonElement> jsonElementTypeAdapter;

        private AllKeysRequiredTypeAdapter(final JsonPropertiesCacheKey jsonPropertiesCacheKey, final TypeAdapter<T> delegateTypeAdapter,
                final TypeAdapter<JsonElement> jsonElementTypeAdapter) {
            this.jsonPropertiesCacheKey = jsonPropertiesCacheKey;
            this.delegateTypeAdapter = delegateTypeAdapter;
            this.jsonElementTypeAdapter = jsonElementTypeAdapter;
        }

        private static <T> TypeAdapter<T> of(final Gson gson, final Class<?> rawType, final TypeAdapter<T> delegateTypeAdapter,
                final TypeAdapter<JsonElement> jsonElementTypeAdapter) {
            return new AllKeysRequiredTypeAdapter<>(new JsonPropertiesCacheKey(gson, rawType), delegateTypeAdapter, jsonElementTypeAdapter);
        }

        @Override
        public void write(final JsonWriter jsonWriter, final T t)
                throws IOException {
            delegateTypeAdapter.write(jsonWriter, t);
        }

        @Override
        public T read(final JsonReader jsonReader)
                throws IOException {
            try {
                // First, convert it to a tree to obtain its keys
                final JsonElement jsonElement = jsonElementTypeAdapter.read(jsonReader);
                // Then validate
                validate(jsonElement);
                // And if the validation passes, then just convert the tree to the object
                return delegateTypeAdapter.read(new JsonTreeReader(jsonElement));
            } catch ( final ExecutionException ex ) {
                throw new RuntimeException(ex);
            }
        }

        private void validate(final JsonElement jsonElement)
                throws ExecutionException {
            if ( !jsonElement.isJsonObject() ) {
                throw new JsonParseException("The given tree is not a JSON object");
            }
            final JsonObject jsonObject = jsonElement.getAsJsonObject();
            final Set<String> expectedProperties = jsonPropertiesCache.get(jsonPropertiesCacheKey);
            final Set<String> actualProperties = jsonObject.keySet();
            // This method comes from Guava but can be implemented using standard JDK
            final Set<String> difference = Sets.difference(expectedProperties, actualProperties);
            if ( !difference.isEmpty() ) {
                throw new JsonParseException("The given JSON object lacks some properties: " + difference);
            }
        }

    }

    private static final class JsonPropertiesCacheKey {

        private final Gson gson;
        private final Class<?> rawType;

        private JsonPropertiesCacheKey(final Gson gson, final Class<?> rawType) {
            this.gson = gson;
            this.rawType = rawType;
        }

        @Override
        @SuppressWarnings("ObjectEquality")
        public boolean equals(final Object o) {
            if ( this == o ) {
                return true;
            }
            if ( o == null || getClass() != o.getClass() ) {
                return false;
            }
            final JsonPropertiesCacheKey jsonPropertiesCacheKey = (JsonPropertiesCacheKey) o;
            @SuppressWarnings("ObjectEquality")
            final boolean areEqual = gson == jsonPropertiesCacheKey.gson && rawType == jsonPropertiesCacheKey.rawType;
            return areEqual;
        }

        @Override
        public int hashCode() {
            return gson.hashCode() * 31 + rawType.hashCode();
        }

    }

    private static final LoadingCache<JsonPropertiesCacheKey, Set<String>> jsonPropertiesCache = CacheBuilder.newBuilder()
            .maximumSize(50)
            .build(new CacheLoader<JsonPropertiesCacheKey, Set<String>>() {
                @Override
                public Set<String> load(final JsonPropertiesCacheKey jsonPropertiesCacheKey)
                        throws Exception {
                    return JsonProperties.tryLookupKeys(jsonPropertiesCacheKey.gson, jsonPropertiesCacheKey.rawType);
                }
            });

}

Now if we apply the type adapter factory, we can check for the given JSON properties presence:

@AllKeysRequired
final class FooBarBaz {

    final String foo;
    final int bar;
    final String[] baz;

    FooBarBaz(final String foo, final int bar, final String[] baz) {
        this.foo = foo;
        this.bar = bar;
        this.baz = baz;
    }

}

private static final Gson gson = new GsonBuilder()
        .serializeNulls()
        .registerTypeAdapterFactory(AllKeysRequiredTypeAdapterFactory.get())
        .create();

gson.fromJson("{\"foo\":\"foo\",\"bar\":1,\"baz\":[]}", FooBarBaz.class);
gson.fromJson("{\"foo\":\"foo\",\"bar\":1,\"baz\":null}", FooBarBaz.class);
gson.fromJson("{\"foo\":\"foo\",\"bar\":1}", FooBarBaz.class);

The last gson.fromJson call will throw an exception with the following message:

The given JSON object lacks some properties: [baz]

Answer 3

You can identify if a particular key is missing or null, but then you won't be able to parse that into a Java object. You would have to define that key as JsonElement in your Java model.

public class Pojo {
    private JsonElement field1;
    private String field2;
}

Then after Gson is done parsing that json, you can check whether that particular object(field1) is:

1. null - This means that key is missing
2. an instance of JsonNull - This means that key's value is null
3. Otherwise - This means that key has non null value. You can then parse it again separately into the Java object using Gson.

You can wrap all this logic into a helper class.