Group by n fields in Java using stream API

Question

Need to group List<Object> based upon N property fields, This fields are decided at run time. How can I achieve this ?

Group by multiple field names in java 8 Referred this question, it is using fixed number of fields.

For Example:

Person{ int age, String city, Date doj, double salary}
record1: 25, NYC, 02/25/2018, 50000
record2: 25, MEX, 02/25/2017, 70000
record3: 26, MEX, 02/25/2017, 80000

GroupBy(city, doj)

Record1: = MEX, 02/25/2017, 150000
Record2: = NYC, 02/25/2018, 50000

Salary will added.

I am storing result in Map<Object, List<Object>> I have achieved most of it. Only problem I am facing is how to alter key in groupingBy.

Collectors.groupingBy( date ) : second iteration will mess data for all city which is to be grouped by city+date.This will be solved if I can alter the key to be City+Date How can I alter my key in second iteration Collectors.groupingBy( date )

Answer 1

Using JB Nizet suggested solution, I have put together an entire working solution in which you can group by n number of fields.

• Grouping on any number of fields are possible
• Result is independent of grouping field order
• User can define aggregation stragery

This nested property will help us store the key for our grouping.

public class NestedProperty {
    private final Field property;
    private final Object value;
}

Field here is a simple object which will be feed at runtime. We can have better alternative to decide its type.

public class Field{
        String name;
        Class type;
    }

This interface should be implementation by POGO to define what is the aggregation strategy.

public interface Aggregatable<T> {
        public void add(T o);
    }

Then using NestedProperty object we group the records till n-1 fields using streams.groupby function.

Map<List<NestedProperty>, List<T>> aggregatedRecords = objects.stream()
                    .collect(Collectors.groupingBy(r -> createGroupingKey(Nminus1fields, r), Collectors.toList()));


private static List<NestedProperty> createGroupingKey(java.util.List<Field> fields, Object r) {
            return fields.stream().map(p -> p.toValue(r, p)).collect(Collectors.toList());
        }

Then we can run the main aggregation method

List<?> result = objects.stream().filter( r -> r!=null )
                .collect(Collectors.groupingBy(
                        record -> {
                            try {
                                return cast.cast(PropertyUtils.getNestedProperty(record, field.getName()));
                            }catch(Exception e) {
                                System.out.println("Property not found.");
                            }
                            return null;
                        }))
                .entrySet().stream()
                .map( e -> e.getValue().stream()
                        .reduce((f1, f2) -> {
                            try {
                                return (T) add(classDefination, f1, f2);
                            } catch (Exception e1) {
                                System.out.println("Error is method add()");
                            }
                            return null;
                        })
                ).map(f -> f.get())
                .collect(Collectors.toList());

Please refer to answer in below link: http://www.unbounded.in/group-by-n-fields-in-java-like-sql-using-streams-api/

Answer 2

Here is a complete example:

import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.stream.Collectors;

public class Grouping {

    static final class Person {
        private final int age;
        private final String city;
        private final String doj;
        private final double salary;

        public Person(int age, String city, String doj, double salary) {
            this.age = age;
            this.city = city;
            this.doj = doj;
            this.salary = salary;
        }

        public int getAge() {
            return age;
        }

        public String getCity() {
            return city;
        }

        public String getDoj() {
            return doj;
        }

        public double getSalary() {
            return salary;
        }

        @Override
        public String toString() {
            return "Person{" +
                "age=" + age +
                ", city='" + city + '\'' +
                ", doj='" + doj + '\'' +
                ", salary=" + salary +
                '}';
        }
    }

    enum Property {
        AGE {
            @Override
            protected Object extractValue(Person person) {
                return person.getAge();
            }
        },
        CITY {
            @Override
            protected Object extractValue(Person person) {
                return person.getCity();
            }
        },
        DOJ {
            @Override
            protected Object extractValue(Person person) {
                return person.getDoj();
            }
        };

        protected abstract Object extractValue(Person person);

        public PropertyValue toValue(Person person) {
            return new PropertyValue(this, extractValue(person));
        }
    }

    static final class PropertyValue {
        private final Property property;
        private final Object value;

        public PropertyValue(Property property, Object value) {
            this.property = property;
            this.value = value;
        }

        @Override
        public boolean equals(Object o) {
            if (this == o) {
                return true;
            }
            if (o == null || getClass() != o.getClass()) {
                return false;
            }
            PropertyValue that = (PropertyValue) o;
            return property == that.property &&
                Objects.equals(value, that.value);
        }

        @Override
        public int hashCode() {
            return Objects.hash(property, value);
        }

        @Override
        public String toString() {
            return "PropertyValue{" +
                "property=" + property +
                ", value=" + value +
                '}';
        }
    }

    private static List<PropertyValue> createGroupingKey(List<Property> properties, Person person) {
        return properties.stream().map(property -> property.toValue(person)).collect(Collectors.toList());
    }

    public static void main(String[] args) {
        List<Person> persons = Arrays.asList(
            new Person(25, "NYC", "02/25/2018", 50000),
            new Person(25, "MEX", "02/25/2017", 70000),
            new Person(26, "MEX", "02/25/2017", 80000)
        );

        // TODO ask the user, rather than hardcoding
        List<Property> groupingProperties = Arrays.asList(Property.CITY, Property.DOJ);

        Map<List<PropertyValue>, Double> salaryAggregatedByChosenProperties =
            persons.stream()
                   .collect(Collectors.groupingBy(p -> createGroupingKey(groupingProperties, p),
                                                  Collectors.summingDouble(Person::getSalary)));

        System.out.println("salaryAggregatedByChosenProperties = " + salaryAggregatedByChosenProperties);
    }
}

What it does:

1. ask the user which properties should be used for grouping (this is actually not done, but simulated, since that's not the core of your question). You get back a List<Property>, containing (for example) the properties CITY and DOJ
2. You transform each person into a grouping key, of type List<PropertyValue>, so, the first person will be transformed into [NYC, 02/25/2018], whereas the second and third ones will both be be transformed into [MEX, 02/25/2017] (and thus have the same key).
3. You group the persons by their key
4. You sum the salaries of the persons of the same group