How to properly do infinite loading in listview

Question

Theme: I am downloading a bunch of data from my external database and I would like to display only a few of them (10) at a time. When the user scrolls down I want to automatically download and display another 10 (and display a Loading progress bar below the Listview as its FooterView, while the items are downloading and loaded into the ListView). I would like to know if this is the proper way to do it, because I encounter a few problems:

Problems

1.Sometimes it's lagging, like the last visible item is item #40 and I can't scroll down anymore unless I scroll up first a bit and then down. (the FooterView does not show up)

2.Sometimes the Loading progress bar stays on and I have to scroll up first then down so that it vanishes and more items show up. (I' waiting in vain, the FooterView is not removed even if the items are downloaded)

How it should work: At first 10 items are downloaded, 6 are visible (screen size dependent). When I scroll down and I am at item #8, another 10 items are downloaded so I can keep scrolling down OR if the items are not yet downloaded, a Loading progressbar shows up on the bottom of the screen as the footer of the ListView until the items are downloaded. Once the items are loaded, the FooterView is removed and I can continue scrolling down.

boolean all_items_downloaded = false;

    num_loadstart = 0;
    num_loadfinish = 10;

    new DownloadUsers3().execute(String.valueOf(num_loadstart), String.valueOf(num_loadfinish));

    lv.setOnScrollListener(new OnScrollListener() {

        @Override
        public void onScroll(AbsListView view, int firstVisibleItem, int visibleItemCount, int totalItemCount) {

            int lastInScreen = firstVisibleItem + visibleItemCount;
            Log.i("FOLLOW Onscroll totalItemCount", totalItemCount + "");
            Log.i("FOLLOW Onscroll visibleItemCount", visibleItemCount + "");
            Log.i("FOLLOW Onscroll lastInScreen", lastInScreen + "");

            if (lastInScreen == totalItemCount) {     
                if (all_items_downloaded) {
                    lv.removeFooterView(loadMoreView);
                }
            }

            //when the last visible item is near the last downloaded item then start loading more
            if ((totalItemCount - lastInScreen == 2) && !(loadingMore) && (totalItemCount != 0)) {
                Log.i("FOLLOW LOADMORE", "NOW");
                if (!all_items_downloaded) {
                    new DownloadUsers3().execute(String.valueOf(totalItemCount), "10");
                }
            }

            lastPos = lv.getLastVisiblePosition();
        }

        @Override
        public void onScrollStateChanged(AbsListView view, int scrollState) {


        }

    });

    return view;
}


public class DownloadUsers3 extends AsyncTask {

    @Override
    protected void onPostExecute(Integer result) {
        Log.i("DownloadUsers3 onPost result", result + "");
        pb.setVisibility(View.GONE);
        loading_ll.setVisibility(View.GONE);
        llMain.setVisibility(View.VISIBLE);

        if (result != null) {

            if (result < 1) tv_noone.setVisibility(View.VISIBLE);
            else tv_noone.setVisibility(View.GONE);

            //set adapter only when items are downloaded the first time aka. no scrolling has been made
            if (result < 11) {
                myadapter = new MyAdapter(ctx, arr_users_id, arr_users_username, arr_users_photo, arr_users_followed, arr_users_numadded, arr_users_numcompleted, arr_users_fbuserid, arr_users_imagetype, arr_users_twuserid, arr_users_twphoto);
                lv.setAdapter(myadapter);
            } else {
                myadapter.notifyDataSetChanged();
            }
        }
        loadingMore = false;

    }


    @Override
    protected void onPreExecute() {
        //show progress dialog only the first time when items are being downloaded
//this is just an overlay progress dialog to show the users when they first open the fragment
        if (arr_users_id.size() < 10) {
            pb.setVisibility(View.VISIBLE);
            loading_ll.setVisibility(View.VISIBLE);
            llMain.setVisibility(View.GONE);
        }
    }

    @Override
    protected Integer doInBackground(String... params) {

        loadingMore = true;

        try{
            HttpClient httpclient = new DefaultHttpClient();
            HttpPost httppost = null;
            httppost = new HttpPost(list_all_users);

            List nameValuePairs = new ArrayList(1);
            String start = params[0];
            String finish = params[1];

            Log.i("At session_userid", session_userid + "");
            nameValuePairs.add(new BasicNameValuePair("session_userid", session_userid));
            nameValuePairs.add(new BasicNameValuePair("start", start));
            nameValuePairs.add(new BasicNameValuePair("finish", finish));
            httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
            HttpResponse response = httpclient.execute(httppost);
            HttpEntity entity = response.getEntity();
            is = entity.getContent();
            }catch(Exception e){
                    Log.e("error", "Error in http connection "+e.toString());
            }

            try{
                BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"),8);
                StringBuilder sb = new StringBuilder();
                String line = null;
                while ((line = reader.readLine()) != null) {
                     sb.append(line + "
");
                }
                is.close();
                Bresult=sb.toString();
                Log.i("Bresult", Bresult + "");
            } catch(Exception e){
                    Log.e("error", "Error converting result "+e.toString());
            }

            if (Bresult.length() > 10) {
                try {
                    jArray = new JSONArray(Bresult);
                    for(int i=0;i



The PHP code for the query is something like this:

SELECT ... FROM ... GROUP BY ... ORDER BY ... LIMIT $start, $finish


where $start and $finish are the variables passed from Java.

DONE with Volley

First I added volley as a library project then added the classes in this tutorial. 

private void makeJsonArrayRequest(final String start, final String finish) {

    loadingMore = true;
    StringRequest postReq = new StringRequest(Method.POST, list_all_users2, new Response.Listener() {

        @Override
        public void onResponse(String response) {

            Log.i("VOLLEYY", response);
            Log.i("VOLLEYY.length()", response.length() + "");
            if (response.length() > 10) {
                try {
                    jArray = new JSONArray(response);
                    for(int i=0;i getParams() {
            Map params = new HashMap();
            params.put("session_userid", session_userid);
            params.put("start", start);
            params.put("finish", finish);
            return params;
        }

    };

    postReq.setShouldCache(false);
    AppController.getInstance().addToRequestQueue(postReq);
}


The loading got faster a bit, but the problems were still present. Sometimes the loading progress bar (the FooterView) did not disappear so I needed to scroll up a bit and then down again. I found that this happened when I jumped over the appropriate item somehow (the one that should have initiated the loading of the next package):

01-17 18:15:57.809: I/FOLLOW Onscroll totalItemCount(3230): 11
01-17 18:15:57.809: I/FOLLOW Onscroll lastInScreen(3230): 6
01-17 18:15:57.859: I/FOLLOW Onscroll totalItemCount(3230): 11
01-17 18:15:57.859: I/FOLLOW Onscroll lastInScreen(3230): 7
01-17 18:15:57.869: I/FOLLOW Onscroll totalItemCount(3230): 11
01-17 18:15:57.869: I/FOLLOW Onscroll lastInScreen(3230): 8
01-17 18:15:57.869: I/FOLLOW Onscroll totalItemCount(3230): 11
01-17 18:15:57.869: I/FOLLOW Onscroll lastInScreen(3230): 8
01-17 18:15:57.899: I/FOLLOW Onscroll totalItemCount(3230): 11
01-17 18:15:57.899: I/FOLLOW Onscroll lastInScreen(3230): 10
01-17 18:15:57.929: I/FOLLOW Onscroll totalItemCount(3230): 11
01-17 18:15:57.929: I/FOLLOW Onscroll lastInScreen(3230): 11


According to this block the new loading should start when the last visible item is 9, but somehow I jumped it over during the scrolling.

if ((totalItemCount - lastInScreen == 2) && !(loadingMore) && (totalItemCount != 0)) {
    if (!all_items_downloaded) {
    makeJsonArrayRequest(String.valueOf(totalItemCount), "10");
    }
}


I solved it by modifying the if (lastInScreen == totalItemCount) {   } block

if (lastInScreen == totalItemCount) {     
                    if (all_items_downloaded) {
                        //Log.i("all_items_downloaded", "yes");
                        lv.removeFooterView(loadMoreView);
                    } else {
                        Log.i("FOLLOW loadingMore", loadingMore + "");
                        if (!loadingMore) {
                            Log.i("TAG", "NOW IT STOPS SO I QUERY AGAIN");
                            makeJsonArrayRequest(String.valueOf(totalItemCount), "10");
                        }
                    }
                }


In case anyone needs...

Update
A few weeks have passed since I changed most of my Asynctasks to Volley. I have just made a small experiment on checking the download time of Volley compared to Async. I download a bunch of data from my VPS, here are the results (in ms):

sonictt1 · Accepted Answer

Your first mistake is using an AsyncTask. You should be using Volley. Everybody should be using Volley. If you have no idea what I'm talking about, check out the lecture at I/O 2013 here. Just google it to find the official tutorial. This, by itself, could fix your lagging. It will also fix lagging outside of this section of your app due to the AsyncTask completing in the BG, unless you actually cancel it properly, which I doubt you do (not because I think you're a bad programmer, but because literally nobody I've ever worked with cancels them properly).

Other than that, I think your psudocode is fine. You're approaching it in the right way to save the user data and load times.

It appears that, in your last AsyncTask, you're doing all of the data parsing in doInBackground() which isn't the way it should be handled (if I'm reading your code correctly and it doesn't do something weird with that data afterwards). You should be parsing data in onPostExecute() and then calling notifyDataSetChanged() on your ListView, otherwise it has no idea that the data is new. Also make sure you have a callback so your custom views know when the new data has arrived.

Honestly, if you switched to Volley (you really should), it would make everything easier, including your debugging, because there is a LOT less boilerplate code.

How to properly do infinite loading in listview

Answers (1)

Related Questions