How to convert a JSON map into a custom Java list, using Gson?

Question

Given a JSON map like this:

{
    "category1": [],
    "category2": ["item1"],
    "category3": ["item1", "item2"]
}

I would like to convert it into a Java ArrayList of Categories (not a Map), like this:

class Category {
  final String categoryName;
  final ArrayList<String> items;

  public Category(String categoryName, ArrayList<String> items) {
    this.categoryName = categoryName;
    this.items = items;
  }
}

ArrayList<Category> data = new ArrayList<>();

So I expect data to look like this:

for (Category c: data) {
  System.out.println(c.categoryName + ", " + c.items);
}
/**
Expected data =
category1, empty list [] or null
category2, [item1]
category2, [item1, item2]
**/

I've tried using the Gson library:

app/build.gradle:

dependencies {
   ...
   implementation 'com.google.code.gson:gson:2.8.6'
}

JsonConverterTest.java

import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonParser;
import com.google.gson.internal.LinkedTreeMap;
import com.google.gson.reflect.TypeToken;

import org.junit.Test;

import java.lang.reflect.Type;
import java.util.ArrayList;
import java.util.Arrays;

import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

public class JsonConverterTest {

  @Test
  public void gsonMapToListTest() {
    String json = "{\"category1\": [],\"category2\": [\"item1\"],\"category3\": [\"item1\", \"item2\"]}";

    class Category {
      final String categoryName;
      final ArrayList<String> items;

      public Category(String categoryName, ArrayList<String> items) {
        this.categoryName = categoryName;
        this.items = items;
      }
    }

    ArrayList<Category> expectedData = new ArrayList<>();
    expectedData.add(new Category("category1", new ArrayList<String>()));
    expectedData.add(new Category("category2", new ArrayList<>(Arrays.asList("item1"))));
    expectedData.add(new Category("category2", new ArrayList<>(Arrays.asList("item1", "item2"))));

    System.out.println("Expected Data:");
    for (Category c: expectedData) {
      System.out.println(c.categoryName + ", " + c.items);
    }

    // This works, but it returns a Map instead of a List
    LinkedTreeMap<String, ArrayList<String>> dataMap = new Gson().fromJson(json, LinkedTreeMap.class);
    System.out.println("\n data as Map = " + dataMap);

    // This causes "IllegalStateException: Expected BEGIN_ARRAY but was BEGIN_OBJECT at line 1 column 2 path $"
    Type listOfCategoriesType = new TypeToken<ArrayList<Category>>() {}.getType();
    ArrayList<Category> data = new Gson().fromJson(json, listOfCategoriesType); // IllegalStateException here
    assertThat(data, is(expectedData));

    // This causes "java.lang.IllegalStateException: Not a JSON Array: {...}"
    JsonArray jsonArray = JsonParser.parseString(json).getAsJsonArray(); // IllegalStateException here
    data = new Gson().fromJson(jsonArray, listOfCategoriesType);
    assertThat(data, is(expectedData));
  }
}

Using the guides https://www.baeldung.com/gson-list and https://futurestud.io/tutorials/gson-mapping-of-maps , I can only convert the JSON map to a Java Map. But I get an IllegalStateException if I try to convert the JSON map to a Java List:

Type listOfCategoriesType = new TypeToken<ArrayList<Category>>() {}.getType();
ArrayList<Category> data = new Gson().fromJson(json, listOfCategoriesType); // IllegalStateException

or

JsonArray jsonArray = JsonParser.parseString(json).getAsJsonArray(); // IllegalStateException
ArrayList<Category> data2 = new Gson().fromJson(jsonArray, listOfCategoriesType);

So what is the correct way in Gson to convert the Json Map to a Java list, as per the unit test gsonMapToListTest() above?

Answer 1

Not the easiest way, but Gson allows implementing a type adapter like this so you can deserialize the JSON in a result object without intermediate collections.

public final class CombinerTypeAdapterFactory<V, C>
        implements TypeAdapterFactory {

    // used to create a collection to populate
    private final TypeToken<? extends Collection<? super C>> collectionTypeToken;
    // represents a type of each map entry value
    private final TypeToken<V> valueTypeToken;
    // a strategy to map an entry to an element
    private final BiFunction<? super String, ? super V, ? extends C> combine;

    private CombinerTypeAdapterFactory(final TypeToken<? extends Collection<? super C>> collectionTypeToken, final TypeToken<V> valueTypeToken,
            final BiFunction<? super String, ? super V, ? extends C> combine) {
        this.collectionTypeToken = collectionTypeToken;
        this.valueTypeToken = valueTypeToken;
        this.combine = combine;
    }

    public static <V, C> TypeAdapterFactory create(final TypeToken<List<C>> collectionTypeToken, final TypeToken<V> valueTypeToken,
            final BiFunction<? super String, ? super V, ? extends C> combine) {
        return new CombinerTypeAdapterFactory<>(collectionTypeToken, valueTypeToken, combine);
    }

    @Override
    @Nullable
    public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
        // check if supported
        if ( !typeToken.equals(collectionTypeToken) ) {
            return null;
        }
        // grab the value type adapter
        final TypeAdapter<V> valueTypeAdapter = gson.getAdapter(valueTypeToken);
        // and the type adapter the collection to be populated
        final TypeAdapter<? extends Collection<? super C>> collectionTypeAdapter = gson.getDelegateAdapter(this, collectionTypeToken);
        return new TypeAdapter<T>() {
            @Override
            public void write(final JsonWriter out, final T value) {
                throw new UnsupportedOperationException(); // TODO
            }

            @Override
            public T read(final JsonReader in)
                    throws IOException {
                // assuming all of those are always { e1, e2, e3, ... }
                in.beginObject();
                // a hack to create an empty collection (it's modifiable right?)
                final Collection<? super C> collection = collectionTypeAdapter.fromJson("[]");
                // for each entry in the map that's being read
                while ( in.hasNext() ) {
                    // get is name
                    final String name = in.nextName();
                    // and its value
                    final V value = valueTypeAdapter.read(in);
                    // combine
                    final C combined = combine.apply(name, value);
                    // and add to the list
                    collection.add(combined);
                }
                in.endObject();
                // we know better than javac
                @SuppressWarnings("unchecked")
                final T result = (T) collection;
                return result;
            }
        }
                .nullSafe();
    }

}

The following test is green then:

public final class CombinerTypeAdapterFactoryTest {

    @AllArgsConstructor
    @EqualsAndHashCode
    @ToString
    private static final class Category {

        @Nullable
        final String categoryName;

        @Nullable
        final List<String> items;

    }

    private static final Gson gson = new GsonBuilder()
            .disableInnerClassSerialization()
            .disableHtmlEscaping()
            .registerTypeAdapterFactory(CombinerTypeAdapterFactory.create(new TypeToken<List<Category>>() {}, new TypeToken<List<String>>() {}, Category::new))
            .create();

    @Test
    public void test()
            throws IOException {
        try ( final JsonReader jsonReader = new JsonReader(new InputStreamReader(CombinerTypeAdapterFactoryTest.class.getResourceAsStream("input.json"))) ) {
            final List<Category> actualCategories = gson.fromJson(jsonReader, new TypeToken<List<Category>>() {}.getType());
            Assertions.assertIterableEquals(
                    ImmutableList.of(
                            new Category("category1", ImmutableList.of()),
                            new Category("category2", ImmutableList.of("item1")),
                            new Category("category3", ImmutableList.of("item1", "item2"))
                    ),
                    actualCategories
            );
        }
    }

}

Answer 2

Easiest is to simply parse into Map, then convert Map to List<Category>.

LinkedTreeMap<String, ArrayList<String>> dataMap = new Gson().fromJson(json,
        new TypeToken<LinkedTreeMap<String, ArrayList<String>>>() {}.getType());

ArrayList<Category> dataList = new ArrayList<>();
for (Entry<String, ArrayList<String>> entry : dataMap.entrySet())
    dataList.add(new Category(entry.getKey(), entry.getValue()));

The alternative is to write your own TypeAdapter, or to use another JSON library.