Threads with Robot freeze in JavaFX application

Question

I have a problem with a JavaFX Robot in my application. Since i read that Platform.runLater() should not be used for a background calculations, so I've tried to use Task, but unfortunately - unsuccessfully.

To demonstrate my problem i created a simple app with just one button. After the button is clicked, it should count all the pixels from a screen that have R equal to 255 and then, print their number on the screen. But what happens is on the line where the Robot instance is supposed to take a screenshot, it freezes, and never goes forward. If i put the calculation code into a Platform.runLater(), everything works. I don't know why it happens. Can anyone help me?

Here are pieces of my code:

package sample;

import javafx.scene.robot.Robot;


public class Controller {
    private Robot robot = new Robot();
    private PixelCounterService service = new PixelCounterService(robot);

    public void initialize(){
        service.setOnSucceeded(workerStateEvent -> {
            System.out.println(workerStateEvent.getSource().getValue().toString());
        });
        service.start();
    }
}

package sample;

import javafx.concurrent.Service;
import javafx.concurrent.Task;
import javafx.scene.image.PixelReader;
import javafx.scene.image.WritableImage;
import javafx.scene.robot.Robot;
import javafx.stage.Screen;

public class PixelCounterService extends Service<Integer> {
    private Robot robot;

    public PixelCounterService(Robot robot){
        this.robot = robot;
    }

    @Override
    protected Task<Integer> createTask() {
        return new Task<>() {
            @Override
            protected Integer call() throws Exception {
                Screen screen = Screen.getPrimary();
                //any code put here is executed
                WritableImage screenshot = robot.getScreenCapture(null, screen.getBounds());
                //lines below this one are never executed
                PixelReader pixelReader  = screenshot.getPixelReader();
                int redPixels = 0;
                for(int i = 0; i < screenshot.getWidth(); i++){
                    for(int j = 0; j < screenshot.getHeight(); j++){
                        if(pixelReader.getColor(i, j).getRed() == 1){
                            redPixels++;
                        }
                    }
                }
                return redPixels;
            }
        };
    }
}

Since I'm using Java and JavaFX 16 for this project, to start it (in4 IntelliJ) I have to add a following line to a VM options
--module-path "PATH_TO_JAVAFX16\javafx-sdk-16\lib" --add-modules javafx.controls,javafx.fxml,javafx.graphics,javafx.web
and in
Project Structure -> Modules
add JavaFX lib directory.

Answer 1

From the Javadocs:

Robot objects must be constructed and used on the JavaFX Application Thread.

So you need to take your screenshot on the JavaFX Application Thread; you can then process it on the background thread:

public class PixelCounterService extends Service<Integer> {
    private Robot robot;

    public PixelCounterService(Robot robot){
        this.robot = robot;
    }

    @Override
    protected Task<Integer> createTask() {
        Screen screen = Screen.getPrimary();
        WritableImage screenshot = robot.getScreenCapture(null, screen.getBounds());
        PixelReader pixelReader  = screenshot.getPixelReader();
        return new Task<>() {
            @Override
            protected Integer call() throws Exception {
                int redPixels = 0;
                for(int i = 0; i < screenshot.getWidth(); i++){
                    for(int j = 0; j < screenshot.getHeight(); j++){
                        if(pixelReader.getColor(i, j).getRed() == 1){
                            redPixels++;
                        }
                    }
                }
                return redPixels;
            }
        };
    }
}