Creating POJO/beans dynamically and set values using CGLib

Question

I have a requirement to parse a text file and generate JSON document. The text file has a pattern of text which contains a key which is a name and the value is a huge text of TSV with headers.

I could parse the text file and generate bean classes using the headers and now i want to set the data to this generated bean class. I am using reflection to do this.

Class<?> beanClass = BeanClassGenerator.beanGenerator(k, mapForBeanGeneration);
            try {
                Object beanClassObject = beanClass.newInstance();
                lines.forEach(line -> {
                    if (line != null && !line.isEmpty() && !line.equals("null")) {
                        String[] lineData = line.split("\t");
                        System.out.println("LineData length :: " + lineData.length);
                        Method[] methods = beanClass.getMethods();
                        System.out.println("Methods length :: " + methods.length);
                        int index = 0;
                        for (Method m : methods) {
                            m.setAccessible(true);
                            if (m.getName().startsWith("set")) {
                                try {
                                    if ((lineData.length <= index) && lineData[index] != null) {
                                        m.invoke(beanClassObject, lineData[index]);
                                        index++;
                                    } else {
                                        m.invoke(beanClassObject, " ");
                                    }
                                } catch (IllegalAccessException | InvocationTargetException e) {
                                    e.printStackTrace();
                                }
                            }
                        }
                    }
                });
                ObjectMapper om = new ObjectMapper();
                System.out.println(om.writeValueAsString(beanClassObject));
            } catch (InstantiationException | IllegalAccessException | JsonProcessingException e) {
                e.printStackTrace();
  }});

The problem with the approach is that most of the times all the column values may not have data it can be nulled.

I am wondering if there is an easier way of doing this. Any help is appreciated.

Here is the bean generation method.

public static Class<?> beanGenerator(final String className, final Map<String, Class<?>> properties) {
        BeanGenerator beanGenerator = new BeanGenerator();
        beanGenerator.setNamingPolicy(new NamingPolicy() {
            @Override
            public String getClassName(String prefix, String source, Object key, Predicate names) {
                return className;
            }
        });

        BeanGenerator.addProperties(beanGenerator, properties);
        return (Class<?>) beanGenerator.createClass();
    }

Here is the sample text file which needs to be converted to the JSON output.

<Data1>
Col1  col2 col3 col4 col5
even    sense   met has
root    greatest    spin    mostly
gentle  held    introduced  palace
cold    equator remember    grandmother
slightly    butter  depth   like
distant second  coast   everyone


<Data2>
Col1  col2 col3 col4 col5 col6 col7 col8
greatest    rope    operation   flies   brown   continent   combination read
slightly    diagram he  grandfather where   party   fifty   pour
well    put plastic anyway  refer   careful correct furniture
how since   army    tongue  birthday    been    clock   official
table   command specific    distant cutting hill    movie   experience
national    though  stopped youth   army    underline   five    know

<Data3>
Col1 col2 col3 col4 col5 col6 col7 col8 col9 col9 col10
vessels characteristic  ship    joy than    tomorrow    high    seven   future  trade
try gray    fourth  advice  week    stream  motion  musical whom    tin
limited daughter    large   rice    came    home    chicken wheat   engine  box
easy    city    pair    strange stage   visitor coach   announced   allow   simple
jet therefore   single  during  construction    flag    bigger  muscle  complex pleasure
income  several coat    range   dull    cattle  damage  jump    present shake

JSON output:

[{
    "<Data1>": [{
            "col1": "",
            "col2": "",
            "col3": "",
            "col4": ""
        },
        {
            "col1": "",
            "col2": "",
            "col3": "",
            "col4": ""
        },
        {
            "col1": "",
            "col2": "",
            "col3": "",
            "col4": ""
        }
    ]

}, {
    "<Data2>": [{
            "col1": "",
            "col2": "",
            "col3": "",
            "col4": "",
            "col5": "",
            "col6": "",
            "col7": "",
            "col8": ""
        },
        {
            "col1": "",
            "col2": "",
            "col3": "",
            "col4": "",
            "col5": "",
            "col6": "",
            "col7": "",
            "col8": ""
        },
        {
            "col1": "",
            "col2": "",
            "col3": "",
            "col4": "",
            "col5": "",
            "col6": "",
            "col7": "",
            "col8": ""
        }
    ]

}]

I came up with a solution using the Maps.

Map<String, List<Map<String, String>>> finalMap = new HashMap<>();
        metadataMap.forEach((k, v) -> {

            List<Map<String, String>> datamap = new ArrayList<>();

            String key = k;
            String[] fields = v.getFields();
            List<String> lines = v.getLines();


            lines.forEach(line -> {
                if (line != null && !line.isEmpty() && !line.equals("null")) {
                    String[] fieldData = line.split("\t");
                    Map<String, String> eachLineMap = new HashMap<>();
                    for (int index = 0; index < fields.length; index++) {
                        if (index < fieldData.length && (fieldData[index] != null && !fieldData[index].isEmpty())) {
                            eachLineMap.put(fields[index], fieldData[index]);
                        } else {
                            eachLineMap.put(fields[index], " ");
                        }
                        datamap.add(eachLineMap);
                    }
                }
            });
            finalMap.put(key, datamap);
        });

        try {
            output = new ObjectMapper().writeValueAsString(finalMap);
        }catch(JsonProcessingException e){
            e.printStackTrace();
        }

Answer 1

You don't need to write all that logic, you can just use Apache Commons BeanUtils; which provides a utility method (among MANY other utilities), that takes a Map of field names versus field values and populate a given bean with it:

    BeanUtils.populate(target, fieldNameValueMap);

Then the only thing you need to implement is the logic to create the fieldNameValueMap Map; which you can do with this simple method:

    Map<String, String> createFieldNameValueMap(String headerLine, String valuesLine) {
        String[] fieldNames = headerLine.split("\t");
        String[] fieldValues = valuesLine.split("\t");
        
        return IntStream.range(0, fieldNames.length)
            .mapToObj(Integer::new)
            .collect(Collectors.toMap(idx -> fieldNames[idx], idx -> fieldValues[idx]));
    }

You can test this solution with the following working demo:

import java.lang.reflect.InvocationTargetException;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

import org.apache.commons.beanutils.BeanUtils;

import lombok.Data;

public class DynamicBeanUtils {
    
    static Map<String, String> createFieldNameValueMap(String headerLine, String valuesLine) {
        String[] fieldNames = headerLine.split("\t");
        String[] fieldValues = valuesLine.split("\t");
        
        return IntStream.range(0, fieldNames.length)
            .mapToObj(Integer::new)
            .collect(Collectors.toMap(idx -> fieldNames[idx], idx -> fieldValues[idx]));
    }
    
    public static void main(String[] args) {
        String headerLine = "booleanValue\tintValue\tstringValue\tdoubleValue\totherValue";
        String valuesLine = "true\t12\tthis bean will be populated\t22.44\ttest string!!!";
        
        Object target = new MyBean();
        try {
            BeanUtils.populate(target, createFieldNameValueMap(headerLine, valuesLine));
        } catch (IllegalAccessException | InvocationTargetException e) {
            // HANDLE EXCEPTIONS!
        }

        System.out.println(target);
    }
    
    @Data
    public static class MyBean {
        private String stringValue;
        private double doubleValue;
        private int intValue;
        private boolean booleanValue;
        private String otherValue;
    }
}

This is the maven repository page for this dependency, so you can include it in your build: https://mvnrepository.com/artifact/commons-beanutils/commons-beanutils/1.9.3

I used Lombok in this solution as well, only to save me the pain of writing getter/setters/toString to test this solution; but it is not required for your solution.

Complete code on GitHub

Hope this helps.

Answer 2

You are going way overboard with your solution.
Your data is organized as an array of variable length arrays; and does not require some crazy on-the-fly class generation solution. As a side note, on-the-fly class generation is not inherently crazy; it is crazy to use on-the-fly class generation in this situation.

Do this:

1. Look at your data; it is organized as follows:
   1. first: outer key
   2. second: exactly one line containing a variable number of space separated array of inner keys.
   3. third: some number of lines containing values.
2. Design a solution to fix your problem
   1. Read the outer key. Use that value to create the outer key portion of your JSON.
   2. Read the inner keys. Store these in an array; use LinkedList, not ClownList (ArrayList).
   3. Do this until the next empty line:
      1. Read a line of values.
      2. Write the inner JSON; use the inner keys as the keys for this.
   4. Skip empty lines until one of the following:
      1. If at end of file, write the ending portion of the JSON.
      2. If you read the next outer key, goto to line 2 (Read the inner keys) above.
3. Write the code.

Answer 3

I realized that instead of creating the POJOs with a complex approach. It is better to use the Maps and convert them to JSON using Jackson ObjectMapper. Posting for others who think this might be a useful approach.

public String convert(Map<String, ? extends Metadata> metadataMap) {
        String output = "";
        Map<String, List<Map<String, String>>> finalMap = new HashMap<>();
        metadataMap.forEach((k, v) -> {

            List<Map<String, String>> datamap = new LinkedList<>();

            String key = k;
            String[] fields = v.getFields();
            List<String> lines = v.getLines();


            lines.forEach(line -> {
                if (line != null && !line.isEmpty() && !line.equals("null")) {
                    String[] fieldData = line.split("\t",-1);
                    Map<String, String> eachLineMap = new HashMap<>();
                    for (int index = 0; index < fields.length; index++) {
                        if (index < fieldData.length && (fieldData[index] != null && !fieldData[index].isEmpty())) {
                            eachLineMap.put(fields[index], fieldData[index]);
                        } else {
                            eachLineMap.put(fields[index], " ");
                        }
                        datamap.add(eachLineMap);
                    }
                }
            });
            finalMap.put(key, datamap);
        });

        try {
            output = new ObjectMapper().writeValueAsString(finalMap);
        }catch(JsonProcessingException e){
            e.printStackTrace();
        }

        return output;
    }