CODEWITHSUNDEEP
javaandroidperformancejsoup
dnUVA
dnUVA

Reputation: 43

Using a HashMap to store large amounts of data is slowing down my Android app, are there other options?

I've written an Android application for my school that generates a HashMap which maps a course name to an ArrayList of available sections for that course (Strings). The map is generated using JSoup to connect to the school website and get all the current course information, parse and format it, and create the HashMap>().

It works. However, it literally takes around 5 minutes for the HashMap to generate on an Android device. I'm a relative novice in programming and I was wondering if there are other, more efficient ways of storing and dealing with such a large amount of data (The HashMap maps to around 800 ArrayLists, which in turn contain several Strings each). Ideally the data will be updated each time the application runs, so I'm not sure if writing to the internal storage would be efficient.

Any suggestions?

Thanks

Edit: Here's the method that creates the HashMap. It's a little convoluted but the website I'm pulling the data from wasn't easy to work with.

public HashMap<String, ArrayList<String>> generateCourseSectionMap()
        {
            ArrayList<String> store = new ArrayList<String>();
            CourseLinks courses = new CourseLinks();
            HashMap<String, String> courseLinks = courses.getCourseMap();
            StringUtils util = new StringUtils();

            HashMap<String, ArrayList<String>> map = new HashMap<String, ArrayList<String>>();

            String sectionFormat = "((\\d){5};(\\d)+;(.*?) \\((.*?)\\);(.*?);(\\d)+ \\/ (\\d)+;(.*?);(TBA|Mo|Tu|We|Th|Fr|Sa|Su)+( (\\d){1,2}:(\\d){2}(AM|PM) - (\\d){1,2}:(\\d){2}(AM|PM))*?;(.*?));";

            Document doc;

            try
                {
                    for (Map.Entry<String, String> entry : courseLinks.entrySet())
                        {
                            doc = Jsoup.connect(entry.getValue()).get();

                            Elements links = doc.select("*+tr>*:not(tr[class~=SectionTopic.*]>*):not(tr[class~=SectionTitle.*]>*)");

                            if (!links.isEmpty())
                                links.remove(0);

                            String build = "";

                            for (Element e : links)
                                {
                                    String s = util.trim(e.text());
                                    if (!s.isEmpty())
                                        build = build + s + ";";
                                }

                            String rebuilt = rebuild(build);
                            store = util.toArrayList(rebuilt.split("BREAK"));

                            for (String d : store)
                                {
                                    Pattern p = Pattern.compile(sectionFormat, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
                                    Matcher m = p.matcher(d);

                                    String[] array = d.split(";");

                                    String firstKey = d.substring(0, d.indexOf(";"));

                                    ArrayList<String> sectionList = new ArrayList<String>();

                                    while (m.find())
                                        sectionList.add(array[0] + ";" + array[1] + ";" + m.group());

                                    map.put(firstKey, sectionList);
                                }
                        }
                }
            catch (IOException e)
                {
                    e.printStackTrace();
                }
            return map;
        }

Upvotes: 4

Views: 1179

Answers (1)

SirDarius
SirDarius

Reputation: 42899

First of all, this:

Pattern p = Pattern.compile(sectionFormat, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);

Compiling a pattern for each iteration in a for loop is suboptimal.
Compile it once at the beginning, and use the compiled pattern subsequently.

Also, this:

build = build + s + ";";

As build is a String, repeatedly concatenating it will create new strings in memory at each iteration.
Consider using a StringBuilder's append method instead.

That being said, these issues are not significant enough that they would slow down your process this much.

There isn't enough information right now for me to quickly notice further obvious problems, however it seems that depending on the number of links that are found and then the number of pages downloaded, most of the time might be spent in reading from the network and parsing HTML pages.

You might want to use a tool such as http://developer.android.com/tools/debugging/debugging-tracing.html to see what's happening.

Upvotes: 1

Related Questions