How to use JAVA filter lambda to group Arraylist elements

Question

I'm trying to group by JAVA array list elements.

I have an array of JSONObjects like the following, I want to group by the fields price and side and then sum the shares with new fileds buyShares and sellShares

    [{"shares":20,"side":"B","orderId":"001","price":"500"}, 
    {"shares":20,"side":"S","orderId":"002","price":"501"}, 
    {"shares":25,"side":"B","orderId":"003","price":"500"}, 
    {"shares":10,"side":"S","orderId":"004","price":"501"}, 
    {"shares":30,"side":"B","orderId":"005","price":"505"}, 
    {"shares":35,"side":"B","orderId":"006","price":"505"}, 
    {"shares":35,"side":"S","orderId":"007","price":"500"}]

and I want to group by price and by side to have something like the following :

[{"price":"500","buyShares":45, "sellShares":35}, {"sellShares":30,"price":"501"}, {"price":"505","buyShares":65}]

I'm using the following java code :

ArrayList<JSONObject> aOrdersArray = new ArrayList<>(aOrders.values());
System.out.println(aOrdersArray);
List<JSONObject> test = aOrdersArray.stream()
        .distinct()
        .collect(Collectors.groupingBy(jsonObject -> jsonObject.getInt("price")))
        .entrySet().stream()
        .map(e -> e.getValue().stream()
        .reduce((f1,f2) -> {
            JSONObject h = new JSONObject();
            h.put("price",f1.get("price"));
            System.out.println(f1);
            if (f1.get("side").equals("B") && f2.get("side").equals("B")) {
                h.put("buyShares", f1.getInt("shares") + f2.getInt("shares"));
            }
            else if (f1.get("side").equals("S") && f2.get("side").equals("S")) {
                h.put("sellShares", f1.getInt("shares") + f2.getInt("shares"));
            }
            else if (f1.get("side").equals("S")) {
                h.put("sellShares", f1.get("shares"));
            }
            else if (f1.get("side").equals("B")) {
                h.put("buyShares",f1.get("shares"));
            }
            else if (f2.get("side").equals("S")) {
                h.put("sellShares", f2.get("shares"));
            }
            else if (f2.get("side").equals("B")) {
                h.put("buyShares",f2.get("shares"));
            }
            return h;
        }))
        .map(f -> f.get())
        .collect(Collectors.toList());
System.out.println(test);

and I get sometimes the following error

Exception in thread "main" org.json.JSONException: JSONObject["side"] not found.

Answer 1

Whilst everybody is suggesting to use object mapped values to simplify your calculations, it is not a must per se. The reasons I can think of are:

• you need to create a mapping for every reasonable type of JSON node (it can be automated to some extent though) -- this is totally fine if your mappings are widely used in your codebase, but probably (!) not worth doing it for ad-hoc one-time-in-use stuff;
• manipulating a JSON tree directly guarantees that bindings (de/serialization) do not affect the original JSON tree structure (i.e. in Gson (and most likely in Jackson) it is possible to implement a type adapter that does not guarantee a perfect round-trip for data mappings: in-JSON -> mappings -> out-JSON => in-JSON != out-JSON; the same for mappings to JSON and back);
• some JSON libraries (just like yours?) do not provide the object mapping/binding feature at all.

@SuppressWarnings("unchecked")
final Iterable<JSONObject> input = (Iterable<JSONObject>) new JSONTokener(inputReader).nextValue();
final JSONArray actual = StreamSupport.stream(input.spliterator(), false)
        .collect(Collectors.groupingBy(jsonObject -> jsonObject.getString("price")))
        .entrySet()
        .stream()
        .map(entry -> entry.getValue()
                .stream()
                .collect(Collector.of(
                        () -> {
                            final JSONObject jsonObject = new JSONObject();
                            jsonObject.put("price", entry.getKey());
                            return jsonObject;
                        },
                        (a, e) -> {
                            final String side = e.getString("side");
                            final String key;
                            final BigDecimal shares;
                            switch ( side ) {
                            case "B":
                                key = "buyShares";
                                shares = e.getBigDecimal("shares");
                                break;
                            case "S":
                                key = "sellShares";
                                shares = e.getBigDecimal("shares");
                                break;
                            default:
                                throw new AssertionError(side);
                            }
                            if ( !a.has(key) ) {
                                a.put(key, shares);
                            } else {
                                a.put(key, a.getBigDecimal(key).add(shares));
                            }
                        },
                        (a1, a2) -> {
                            throw new UnsupportedOperationException();
                        },
                        Function.identity()
                ))
        )
        .collect(Collector.of(
                JSONArray::new,
                JSONArray::put,
                (a1, a2) -> {
                    throw new UnsupportedOperationException();
                },
                Function.identity()
        ));
final JSONArray expected = (JSONArray) new JSONTokener(expectedReader).nextValue();
Assertions.assertEquals(expected.toString(), actual.toString()); // why? because neither JSONArray nor JSONObject implement equals/hashCode

Answer 2

To use gson you can create a class :

public class Share{
    private int shares;
    private String side;
    private String orderId;
    private String price;
    
    //getters and setters
    public int getShares() {
        return shares;
    }

    public void setShares(int shares) {
        this.shares = shares;
    }

    public String getSide() {
        return side;
    }

    public void setSide(String side) {
        this.side = side;
    }

    public String getOrderId() {
        return orderId;
    }

    public void setOrderId(String orderId) {
        this.orderId = orderId;
    }

    public String getPrice() {
        return price;
    }

    public void setPrice(String price) {
        this.price = price;
    }
}

then you can use this one to map object from json:

 String yourJson = "[{\"shares\":20,\"side\":\"B\",\"orderId\":\"001\",\"price\":\"500\"},{\"shares\":35,\"side\":\"S\",\"orderId\":\"007\",\"price\":\"500\"}]";
 Gson gson = new Gson();
 Type shareListType = new TypeToken<ArrayList<Share>>(){}.getType();
 ArrayList<Share> userArray = gson.fromJson(yourJson, shareListType);  

Answer 3

What you're trying to do is compicated and errorprone because you've started on the wrong foot.

You shouldn't be having a JSONObject in the first place. The data you do have in your JSON input is regular, and you want to operate on its contents.

The right way to start is to make a java object that represents such a record, and then turn your JSON input into a proper, idiomatic java version of that. You want:

@Value class Record {
  int shares;
  RecordKind kind;
  int orderId;
  int price;

  public PriceGroup toGroup() {
    return new PriceGroup(price,
      kind == BUY ? shares : 0,
      kind == SELL ? shares : 0);
  }
}

@Value class PriceGroup {
  int price;
  int buyShares;
  int sellShares;

  public PriceGroup merge(PriceGroup other) {
    if (other.price != this.price) throw new IllegalArgumentException();
    return new PriceGroup(price,
      this.buyShares + other.buyShares,
      this.sellShares + other.sellShares);
  }
}

NB: Uses Lombok's @Value. Assume this thing has a constructor, all fields are final, toString and equals and hashCode are in their right place, etc.

Once you have the above two types, then you can convert your input JSON into a List<Record>. Armed with this List<Record>, then and only then should you start down the path of mapping, grouping, etc.

That error will then, of course, never occur, and your map/group code will be significantly easier to read. Any typos you make will result in compile time errors. auto complete will work fine, etcetera: All the advantages.

To turn JSON into a given type, use Jackson. If you somehow just don't want to add that dependency (you should, really), then write a public static Record fromJson(JSONObject) method in your Record class, and use .map(Record::fromJson) to get to a stream of Record objects right away, and never go back to JSONObject.

List<Record> orders = ...;
List<PriceGroup> test = orders.stream()
        .distinct()
        .map(Record::toGroup)
        .collect(Collectors.groupingBy(PriceGroup::getPrice))
        .entrySet().stream()
        .map(e -> e.getValue().stream()
        .reduce((f1, f2) -> f1.merge(f2))
        .collect(Collectors.toList());

Your code is now vastly simpler to read, all methods have proper names (it is getBuyShares(), not .getInt("buyShares")) and are discoverable via e.g. auto-complete, and you can for example individually test your merge functionality.