To many AsyncTask with upload at same time

Question

please excuse my bad English I hope my explanations are understandable.

I am working on app, which communicates with my server. It has different tasks. One of them is to upload my pictures from my smartphone to my server. I want, for my app do this in the background, with minimized use of memory and bandwidth.

Basically it works, but when I have to many new pictures, there are to many AsyncTask at the same time and reduce memory and bandwidth perceptible.

First I use a BroadcastReceiver, which start every 30 min my picture scanner. The Scanner first checks, if the External Storage is readable, if WIFI is on and if there is an Internet Connection.

When that is true it queries a list from pictures, which are already uploaded, from the database.

Then it request all pictures from the MediaStore library, checks some minimum values (size, height, width) and if the pic isn't uploaded. When everything is ok, it start an AsyncTask for resize the image and upload it:

public class Scanner {

    private Context context;
    private PicDAO picDAO;
    private Helper helper;

    public Scanner(Context context) {
        this.context = context;
        picDAO = new PicDAO(context);
        helper = new Helper(context);
    }

    public void startScan(){
        if(helper.isExternalStorageReadable()
                && helper.isWifiOn()
                && helper.checkInternet()){
            HashMap<Integer, String> pics = picDAO.picsHashMap();
            Cursor mCursor = context.getContentResolver().query(MediaStore.Images.Media.EXTERNAL_CONTENT_URI,null,null,null,MediaStore.Images.Media.DEFAULT_SORT_ORDER);
            if(mCursor != null){
                mCursor.moveToFirst();
                while(!mCursor.isAfterLast()) {
                    PicClass pic = new PicClass(
                            mCursor.getInt(mCursor.getColumnIndex(MediaStore.Images.Media._ID)),
                            mCursor.getString(mCursor.getColumnIndex(MediaStore.Images.Media.DISPLAY_NAME)),
                            mCursor.getString(mCursor.getColumnIndex(MediaStore.Images.Media.DATA)),
                            mCursor.getInt(mCursor.getColumnIndex(MediaStore.Images.Media.WIDTH)),
                            mCursor.getInt(mCursor.getColumnIndex(MediaStore.Images.Media.HEIGHT)),
                            mCursor.getInt(mCursor.getColumnIndex(MediaStore.Images.Media.SIZE)),
                            context
                    );
                    if(pic.getSize() > 25000
                            && pic.getHeight() > 200
                            && pic.getWidth() > 200
                            && ( pics.get(pic.getIdent()) == null || !pics.get(pic.getIdent()).equals(pic.getDisplay_name()))
                            ){
                        CreateThumb createThumb = new CreateThumb(context);
                        createThumb.execute(new PicClass[]{pic});
                    }
                    mCursor.moveToNext();
                }
                mCursor.close();
            }
        }
    }
}

The creatThumb looks resize the Image and start a upload (using the volley library):

public class CreateThumb extends AsyncTask<PicClass, Void, PicClass> {

    private Context context;

    public CreateThumb(Context context) {
        this.context = context;
    }

    @Override
    protected PicClass doInBackground(PicClass... pics) {
        Helper helper = new Helper(context);
        String encodedString = "";
        if(helper.isWifiOn() && helper.checkInternet()){
            double dWidth = 1000;
            double dHeight = 1000;
            if(pics[0].getWidth() < (int) dWidth && pics[0].getHeight() < (int) dHeight){
                dWidth = pics[0].getWidth();
                dHeight = pics[0].getHeight();
            }else{
                if (pics[0].getWidth() > pics[0].getHeight()){
                    double div = pics[0].getWidth() / dWidth;
                    dHeight = pics[0].getHeight() / div;
                }else{
                    double div = pics[0].getHeight() / dHeight;
                    dWidth = pics[0].getWidth() / div;
                }
            }
            int width = (int) dWidth;
            int height = (int) dHeight;
            BitmapFactory.Options bmOptions = new BitmapFactory.Options();
            Bitmap bitmap = BitmapFactory.decodeFile(pics[0].getPath(),bmOptions);
            Bitmap thumbnail = ThumbnailUtils.extractThumbnail(bitmap,width,height,0);
            ByteArrayOutputStream stream = new ByteArrayOutputStream();
            thumbnail.compress(Bitmap.CompressFormat.JPEG, 90, stream);
            byte[] byte_arr = stream.toByteArray();
            encodedString = Base64.encodeToString(byte_arr, 0);
        }
        pics[0].setThumb_file(encodedString);
        return pics[0];
    }
    @Override
    protected void onPostExecute(final PicClass pic) {
        if(!pic.getThumb_file().equals("")){
            RequestQueue queue = Volley.newRequestQueue(context);
            String url ="http://example.de/upload.php";
            StringRequest postRequest = new StringRequest(Request.Method.POST, url, new Response.Listener<String>(){
                @Override
                public void onResponse(String response) {
                    if(response.equals("OK")){
                        PicDAO picDAO = new PicDAO(context);
                        picDAO.savePic(pic);
                    }
                }
            },
                new Response.ErrorListener() {
                    @Override
                    public void onErrorResponse(VolleyError error) {}
                }
            ){
                @Override
                protected Map<String, String> getParams() {
                    Map<String, String>  params = new HashMap<>();
                    params.put("img",pic.getThumb_file());
                    params.put("filename",pic.getDisplay_name() + ".jpg");
                    return params;
                }
            };
            queue.add(postRequest);
        }
    }
}

The script on my server:

<?php
    $base = $_POST['img'];
    $filename = $_POST['filename'];
    $binary = base64_decode($base);
    $file = fopen('uploadedimages/'.$filename, 'wb');
    fwrite($file, $binary);
    fclose($file);
    echo "OK";

The Problem is, when I have too many new pictures, it slows down the device and the internet connection, and I get these errors:

W/art: Suspending all threads took: 278.260ms D/Volley: [2790] BasicNetwork.logSlowRequests: HTTP response for request=<[ ] http://example.de/upload.php 0x8a9f5792 NORMAL 1> [lifetime=3447], [size=2], [rc=200], [retryCount=1]

How can I optimize my code or prevent that I have to many uploads simultaneously.

EDIT

I tried to rebuild the scanner part and use only one Queue where I add the request. But it seems like it doesn't work. When there is only one picture it works, but when the script add more than on request, it get no response and on the server is just the first picture.

public class Scanner {

    private Context context;
    private PicDAO picDAO;
    private Helper helper;

    public Scanner(Context context) {
        this.context = context;
        picDAO = new PicDAO(context);
        helper = new Helper(context);
    }

    public void startScan(){
        if(helper.isDeviceReady()){
            Cursor mCursor = context.getContentResolver().query(MediaStore.Images.Media.EXTERNAL_CONTENT_URI,null,null,null,MediaStore.Images.Media.DEFAULT_SORT_ORDER);
            if(mCursor != null){
                final RequestQueue mRequestQueue = Volley.newRequestQueue(context);
                mCursor.moveToFirst();
                while(!mCursor.isAfterLast()) {
                    final PicClass pic = new PicClass(mCursor, context);
                    if(pic.checkSize() && !picDAO.picExist(pic)){
                        BitmapFactory.Options bmOptions = new BitmapFactory.Options();
                        Bitmap bitmap = BitmapFactory.decodeFile(pic.getPath(),bmOptions);
                        Bitmap thumbnail = ThumbnailUtils.extractThumbnail(bitmap,pic.getNewSize()[0],pic.getNewSize()[1],0);
                        ByteArrayOutputStream stream = new ByteArrayOutputStream();
                        thumbnail.compress(Bitmap.CompressFormat.JPEG, 90, stream);
                        byte[] byte_arr = stream.toByteArray();
                        pic.setThumb_file(Base64.encodeToString(byte_arr, 0));
                        StringRequest postRequest = new StringRequest(Request.Method.POST, "http://example.de/upload.php", new Response.Listener<String>(){
                            @Override
                            public void onResponse(String response) {
                                Log.d("DEBUG",response);
                                if(response.equals("OK")){
                                    PicDAO picDAO = new PicDAO(context);
                                    picDAO.savePic(pic);
                                }
                            }
                        },
                                new Response.ErrorListener() {
                                    @Override
                                    public void onErrorResponse(VolleyError error) {
                                        VolleyLog.e("Error: ", error.getMessage());
                                    }
                                }
                        ){
                            @Override
                            protected Map<String, String> getParams() {
                                Map<String, String>  params = new HashMap<>();
                                params.put("img",pic.getThumb_file());
                                params.put("filename",pic.getDisplay_name() + ".jpg");
                                return params;
                            }
                        };
                        mRequestQueue.add(postRequest);
                    }
                    mCursor.moveToNext();
                }
                mCursor.close();
            }
        }
    }
}

Answer 1

Well, to start don't make a new request queue for every upload. THe idea of a queue is that you add a bunch of requests to it, then run the queue and let Volley slowly go through the requests a small group of them at a time. You're creating dozens of queues, and I don't even see a call to run the queue.

I'd also be using a Thread rather than an AsyncTask. AsyncTasks should be one offs. In fact by using this many tasks you're starving out all the other things that may need a task, since they share a common Thread.

Answer 2

How can I optimize my code or prevent that I have to many uploads simultaneously

I'd drop AsyncTask in favor of using IntentService, which executes one job at the time and queues all the other. Of course the whole process is not as trivial as it may look so maybe using dedicated libraries like Android Priority Job Queue would be even better.