Waiting for multi-touch input in android

Question

I am trying to do a multi-touch application. The motivation is to write a Braille application, which will be able to read Braille.

I am using this for the braille reference.

Here is the code segment :

@Override
    public boolean onTouchEvent(MotionEvent event) {

        // get pointer index from the event object
        int pointerIndex = event.getActionIndex();

        // get pointer ID
        int pointerId = event.getPointerId(pointerIndex);

        // get masked (not specific to a pointer) action
        int maskedAction = event.getActionMasked();

        switch (maskedAction & MotionEvent.ACTION_MASK) {

            case MotionEvent.ACTION_DOWN:
            case MotionEvent.ACTION_POINTER_DOWN: {
                // We have a new pointer. Lets add it to the list of pointers

                PointF f = new PointF();
                f.x = event.getX(pointerIndex);
                f.y = event.getY(pointerIndex);
                mActivePointers.put(pointerId, f);
                parseBraille();
                break;
            }
            case MotionEvent.ACTION_MOVE: { // a pointer was moved
                for (int size = event.getPointerCount(), i = 0; i < size; i++) {
                    PointF point = mActivePointers.get(event.getPointerId(i));
                    if (point != null) {
                        point.x = event.getX(i);
                        point.y = event.getY(i);
                    }
                }
                break;
            }
            case MotionEvent.ACTION_UP:
            case MotionEvent.ACTION_POINTER_UP:
            case MotionEvent.ACTION_CANCEL: {
                mActivePointers.remove(pointerId);
                break;
            }
        }
        invalidate();
        return true;
    }

    private void showMsg(String msg){
        Toast.makeText(context, msg, Toast.LENGTH_SHORT).show();
    }

   private void parseBraille(){
        if (mActivePointers.size() == 1){
            showMsg("a");
        }
        else if (mActivePointers.size() == 2){
            double x0 = mActivePointers.get(0).x;
            double y0 = mActivePointers.get(0).y;
            double x1 = mActivePointers.get(1).x;
            double y1 = mActivePointers.get(1).y;
            if (Math.abs(mActivePointers.get(0).x - mActivePointers.get(1).x) < 50){
                showMsg("b");
            }
        }
    }

What's happening here, is, when I touch with two fingers, I want to get "b" as the response, but I am getting "a" because of the first finger and then "b".

How to solve this problem ?

Answer 1

You can use two different approach to solve this problem:

1. Machine Learning: more accurate, more time expensive. You ask some people to perform all the Braille alphabets. You record all the performed letters (let's call them gestures) and use them to train a model. That model will be used to perform recognition of new gestures at runtime.

2. Finite State Machine: less accurate, fastest way, the one that you're trying right now. You basically have to:

   1. Record a gesture. From ACTION_DOWN to ACTION_UP records all the MotionEvent that you receive.

   2. Recognize the gesture using all the recorded MotionEvent instances.

The problem in your code is that you are trying to recognize the gesture (method parseBraille) inside the ACTION_POINTER_DOWN event. This event will be fired whenever a user presses a finger on the touch screen, so also at the first finger press. At that moment, you won't be able to recognize the gesture. You have to wait for gesture ending.

Answer 2

You will get touch events one at a time, so you need to modify your approach. I see two solutions: (1) After a touch, wait a given amount of time in case the user presses another finger, or (2) use the first UP event and then collect the total fingers to determine the appropriate letter.