Generating images pixel by pixel in JavaFX with progress visible

Question

I'm writing an JavaFX application which generates abstract pattern images pixel by pixel. The result should look somewhat like this. Here is my main class:

package application;

public class Main extends Application {

  private static final int WIDTH = 800; 
  private static final int HEIGHT = 600; 

    @Override
    public void start(Stage primaryStage) {
        BorderPane root = new BorderPane();
        Scene scene = new Scene(root, WIDTH, HEIGHT);
        scene.getStylesheets().add(getClass().getResource("application.css").toExternalForm());
        primaryStage.setScene(scene);
        primaryStage.show();

    final Canvas canvas = new Canvas(WIDTH, HEIGHT);

        root.getChildren().add(canvas);

        final GraphicsContext gc = canvas.getGraphicsContext2D();
        final PixelWriter pw = gc.getPixelWriter();

        final PixelGenerator generator = new PixelGenerator(WIDTH, HEIGHT, pw);
        final Thread th = new Thread(generator);
        th.setDaemon(true);
        th.start();
    }

    public static void main(String[] args) {
        launch(args);
    }
}

The PixelGenerator class is generating new pixels one by one, filling the canvas using the PixelWriter.setColor() method. It calculates the color for the new pixels based on some random numbers and on previously generated colors.

If I run the PixelGenerator on the application thread, the GUI is blocked until the whole available space is filled and only then I see the complete picture.

To avoid this I made my PixelGenerator class extend the javafx.concurrent.Task and its call() method generates all the pixels at once. Sometimes it works as expected and I can see how the image gets generated step by step, but sometimes the picture remains unfinished, as if the task would not run to the end. Debugging shows that it always runs to the end, but the later PixelWriter.setColor() calls have no effect.

I tried different approaches to fix it. E.g. I added an onSucceeded event handler, and tried to cause the Canvas to "refresh" since I thought that it just somehow 'skips' its last refresh iteration. No success. Strange, even coloring further pixels has no effect inside of the listener.

I also tried using an AnimationTimer instead of utilizing the Task. It works, but my problem is that I can't predict how much pixels I am able to generate between its handle() calls. The generation algorithm complexity and though the CPU time needed for the pixel generation will change as the algorithm will evolve.

My ideal target would be to spend all available CPU time generating pixels, but at the same time to be able to see the generation progress in detail (60 or even 30 FPS would be OK).

Please help me out, what I'm doing wrong and what direction should I go given my target?

Answer 1

It’s true that you can’t make use of PixelWriter in a thread other than the JavaFX application thread. However, your Task can place the pixel data itself in a non-JavaFX value object like an IntBuffer, which the application thread can then pass to setPixels. An example program:

import java.nio.IntBuffer;

import javafx.application.Application;
import javafx.concurrent.Task;
import javafx.stage.Stage;
import javafx.scene.Scene;
import javafx.scene.canvas.Canvas;
import javafx.scene.canvas.GraphicsContext;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelWriter;
import javafx.scene.layout.BorderPane;
import javafx.scene.paint.Color;

public class PixelGeneratorTest
extends Application {
    public static final int WIDTH = Integer.getInteger("width", 500);
    public static final int HEIGHT = Integer.getInteger("height", 500);

    public class PixelGenerator
    extends Task<IntBuffer> {

        private final int width;
        private final int height;

        public PixelGenerator(int width,
                              int height) {
            this.width = width;
            this.height = height;
        }

        @Override
        public IntBuffer call() {
            IntBuffer buffer = IntBuffer.allocate(width * height);

            for (int y = 0; y < height; y++) {
                for (int x = 0; x < width; x++) {
                    Color pixel = Color.hsb(
                        y * 360.0 / HEIGHT, 1, (double) x / WIDTH);

                    int argb = 0xff000000 |
                        ((int) (pixel.getRed() * 255) << 16) |
                        ((int) (pixel.getGreen() * 255) << 8) |
                         (int) (pixel.getBlue() * 255);

                    buffer.put(argb);
                }
            }

            buffer.flip();

            return buffer;
        }
    }

    @Override
    public void start(Stage stage) {
        Canvas canvas = new Canvas(WIDTH, HEIGHT);

        stage.setTitle("PixelGenerator Test");
        stage.setScene(new Scene(new BorderPane(canvas)));
        stage.show();

        GraphicsContext gc = canvas.getGraphicsContext2D();
        PixelWriter pw = gc.getPixelWriter();

        PixelGenerator generator = new PixelGenerator(WIDTH, HEIGHT);

        generator.valueProperty().addListener((o, oldValue, pixels) ->
            pw.setPixels(0, 0, WIDTH, HEIGHT,
                PixelFormat.getIntArgbInstance(), pixels, WIDTH));

        Thread th = new Thread(generator);
        th.setDaemon(true);
        th.start();
    }
}

If you expect your image to be very large, and thus impractical to keep in memory all at once, you can write the Task to accept constructor arguments that allow it to generate only part of the image, then create multiple tasks to handle the pixel generation piece by piece. Another option is having a single Task that repeatedly calls updateValue, but you would then have to create a custom value class that contains both the buffer and the rectangular area in the image to which that buffer should be applied.

Update:

You’ve clarified you want progressive image rendering. A Task won’t work for rapid updates, since changes to a Task value may be coalesced by JavaFX. So it’s back to fundamentals: make a Runnable that invokes setPixels in a Platform.runLater call, to ensure correct thread usage:

import java.nio.IntBuffer;
import java.util.Objects;

import javafx.application.Application;
import javafx.application.Platform;
import javafx.stage.Stage;
import javafx.scene.Scene;
import javafx.scene.canvas.Canvas;
import javafx.scene.canvas.GraphicsContext;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelWriter;
import javafx.scene.layout.BorderPane;
import javafx.scene.paint.Color;

public class PixelGeneratorTest2
extends Application {
    public static final int WIDTH = Integer.getInteger("width", 500);
    public static final int HEIGHT = Integer.getInteger("height", 500);

    private static final PixelFormat<IntBuffer> pixelFormat =
        PixelFormat.getIntArgbInstance();

    public class PixelGenerator
    implements Runnable {

        private final int width;
        private final int height;
        private final PixelWriter writer;

        public PixelGenerator(int width,
                              int height,
                              PixelWriter pw) {
            this.width = width;
            this.height = height;
            this.writer = Objects.requireNonNull(pw, "Writer cannot be null");
        }

        @Override
        public void run() {
            int blockHeight = 4;
            IntBuffer buffer = IntBuffer.allocate(width * blockHeight);

            try {
                for (int y = 0; y < height; y++) {
                    for (int x = 0; x < width; x++) {
                        Color pixel = Color.hsb(
                            y * 360.0 / HEIGHT, 1, (double) x / WIDTH);

                        int argb = 0xff000000 |
                            ((int) (pixel.getRed() * 255) << 16) |
                            ((int) (pixel.getGreen() * 255) << 8) |
                             (int) (pixel.getBlue() * 255);

                        buffer.put(argb);
                    }

                    if (y % blockHeight == blockHeight - 1 || y == height - 1) {
                        buffer.flip();

                        int regionY = y - y % blockHeight;
                        int regionHeight =
                            Math.min(blockHeight, height - regionY);

                        Platform.runLater(() -> 
                            writer.setPixels(0, regionY, width, regionHeight,
                                pixelFormat, buffer, width));

                        buffer.clear();
                    }

                    // Pretend pixel calculation was CPU-intensive.
                    Thread.sleep(25);

                }
            } catch (InterruptedException e) {
                System.err.println("Interrupted, exiting.");
            }
        }
    }

    @Override
    public void start(Stage stage) {
        Canvas canvas = new Canvas(WIDTH, HEIGHT);

        stage.setTitle("PixelGenerator Test");
        stage.setScene(new Scene(new BorderPane(canvas)));
        stage.show();

        GraphicsContext gc = canvas.getGraphicsContext2D();
        PixelWriter pw = gc.getPixelWriter();

        PixelGenerator generator = new PixelGenerator(WIDTH, HEIGHT, pw);

        Thread th = new Thread(generator);
        th.setDaemon(true);
        th.start();
    }
}

You could invoke Platform.runLater for each individual pixel, but I suspect that would overwhelm the JavaFX application thread.