Separate touch from mouse event

Question

I use a touch monitor to control my nodes on the application. But i also want to control the nodes with mouse events, if the application is not running with a touch monitor, so that both events could be handled. I tried it the following way:

private void initDraggable() {
    touchFilter = new EventHandler<TouchEvent>() {
        @Override public void handle(TouchEvent event) {
            System.out.println("Touch dragged");    

            touchTaskViewModel.actualClickPointProperty().set(new ClickPoint(event.getTouchPoint().getX(), event.getTouchPoint().getY()));
            touchTaskViewModel.sceneXProperty().set(event.getTouchPoint().getSceneX());

            touchTaskViewModel.handleTaskMoved();

            event.consume();
        }
    };

    addEventFilter(TouchEvent.TOUCH_MOVED, touchFilter);

    setOnTouchPressed(new EventHandler<TouchEvent>() {
        @Override
        public void handle(TouchEvent event) {
            touchTaskViewModel.entryClickPointProperty().set(new ClickPoint(event.getTouchPoint().getX(), event.getTouchPoint().getY()));

            touchTaskViewModel.taskSelectedProperty().set(true);

            event.consume();
        }
    });

    setOnTouchReleased(new EventHandler<TouchEvent>() {
        @Override
        public void handle(TouchEvent event) {
            touchTaskViewModel.taskSelectedProperty().set(false);

            event.consume();
        }
    });

    clickFilter = new EventHandler<MouseEvent>() {
      @Override public void handle(MouseEvent event) {
          if(!event.isSynthesized()){
            System.out.println("Mouse dragged!");   

            touchTaskViewModel.actualClickPointProperty().set(new ClickPoint(event.getX(), event.getY()));
            touchTaskViewModel.sceneXProperty().set(event.getSceneX());

            touchTaskViewModel.handleTaskMoved();
            event.consume();
          }
      }
    };

    addEventFilter(MouseEvent.MOUSE_DRAGGED, clickFilter);

    setOnMousePressed(new EventHandler<MouseEvent>() {
        @Override
        public void handle(MouseEvent event) {
            if(!event.isSynthesized()){

                touchTaskViewModel.entryClickPointProperty().set(new ClickPoint(event.getX(), event.getY()));

                touchTaskViewModel.taskSelectedProperty().set(true);

                event.consume();
            }
        }
    });

    setOnMouseReleased(new EventHandler<MouseEvent>() {
        @Override
        public void handle(MouseEvent event) {
            if(!event.isSynthesized()){ 
                touchTaskViewModel.taskSelectedProperty().set(false);

                event.consume();
            }
        }
    });
}

So the function isSynthesized() should return true, if the event is already handled by a touch event. But it returns always false, no matter if i use the touch monitor or the mouse to drag my nodes.

The touch events are handled correctly, i see that it prints both "touch dragged" and "mouse dragged".

Answer 1

Mixing touch and mouse events is hard. Even when isSynthesized is reliable (always seems to be for me), it doesn't solve all possible problems. For example, when you have a Button, and you want to be able to fire its action via either the mouse or the touch screen, you can let the touch screen emulate a mouse, and that will work without any extra code beyond what you need for a mouse interface. However, when you click with the mouse, the Button takes the focus and is visibly armed when you hold down the left mouse button. The Button's action doesn't fire, however, until you release the left mouse button.

For a touch interface, the mouse emulation isn't very good. When you press the Button with your finger, it doesn't take the focus, nor is it armed. When you release the Button by taking your finger off the screen, then you get the mouse press event, then the action fires, and then (somewhat inexplicably) you get the mouse released and mouse clicked events. So, the visual feedback isn't as good when you rely on the touch screen's mouse emulation. Further, depending on your preferences, pushing the button via the screen doesn't do what real electromechanical buttons do, which is cause their actions to "fire" when they are pressed not when they are released.

Now, assuming you want your mouse interface to keep working the way it does (with actions happening upon mouse released events), you can add your own "skip" flag to your touch event processing, and have the action handler query that flag. The trick is knowing when to set that flag, and when not to. One would like to just always set that flag when a touch pressed event is processed, because that's the event that is going to take the focus, arm the Button, and fire the action, so you don't want the mouse events to cause a second firing.

But...

There will be no second firing if you drag your finger off the Button before you lift it from the screen. That's because the emulated mouse released event won't be inside the Button, and that wouldn't fire the action with a real mouse, so the emulation doesn't fire it either (notwithstanding that the mouse released and mouse clicked events both get sent to your code after the action handler is called).

So, to avoid setting the skip flag when no second firing will happen, you must not set it in the touch pressed handler. You must conditionally set it in the touch released handler only when the release occurs in an on-screen point contained by the Button.

The code below shows how this can work. (You asked about dragging, not Buttons, I know. But you didn't get any answers for over two years, and I'm thinking the issues involved in making touch interfaces and mouse interfaces play well together, particularly those requiring that touch events not cause double-firings of actions, can be handled generally with something like the technique I'm using here. Hope it helps.)

import javafx.application.Application;
import javafx.event.ActionEvent;
import javafx.event.EventHandler;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.Label;
import javafx.scene.input.TouchEvent;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;

public class ShowNanoTime extends Application
{
    Label msg = new Label();

    public static void main(String... args)
    {
        Application.launch(args);
    }

    @Override
    public void start(Stage stage) throws Exception
    {
        Button btnShowTime = new Button("Show Time");
        VBox root = new VBox();

        Handler handler = new Handler();

        btnShowTime.setOnTouchPressed(handler.new TouchPressed());
        btnShowTime.setOnTouchReleased(handler.new TouchReleased());

        btnShowTime.setOnAction(handler.new Action(this));

        root.getChildren().addAll(msg, btnShowTime);

        Scene scene = new Scene(root, 640, 360);
        stage.setScene(scene);

        stage.show();
    }

    void showTime()
    {
        msg.setText(String.format("%d", System.nanoTime()));
    }
}

class Handler
{
    boolean skip = false;

    class TouchPressed implements EventHandler<TouchEvent>
    {
        @Override
        public void handle(TouchEvent event)
        {
            System.out.println("Touch pressed.");
            ((Button) event.getSource()).requestFocus();
            ((Button) event.getSource()).arm();
            ((Button) event.getSource()).fire();
        }
    }

    class TouchReleased implements EventHandler<TouchEvent>
    {
        @Override
        public void handle(TouchEvent event)
        {
            System.out.println("Touch released");

            Button button = (Button) event.getSource();

            double x = event.getTouchPoint().getX();
            double y = event.getTouchPoint().getY();

            if (button.contains(x, y))
            {
                System.out.println("Setting skip.");
                skip = true;
            }
        }
    }

    class Action implements EventHandler<ActionEvent>
    {
        ShowNanoTime app;

        Action(ShowNanoTime app)
        {
            this.app = app;
        }

        @Override
        public void handle(ActionEvent event)
        {
            if (skip)
            {
                skip = false;
                System.out.println("Action skipped.");
                return;
            }

            System.out.println("Action happens!");

            app.showTime();
        }
    }
}

Note: There's nothing platform-specific in my code, but I've only run the above on a Windows 10 machine.