using stockfish.js by nmrugg to build a chess game review site

Question

link : roadtomagnus

I’m building a chess game review application using StockFish.js (an in-browser engine from NNRUG) with multiple web workers to speed up the evaluation process.

To reproduce the issue:

Navigate to the project folder: bash Copy code

cd frontendTs  
npm run dev

In the app: Click Games. Click on the document icon next to movescount to review any game. Project Structure: /frontengTS/src/scripts/_GameReviewManager.ts: Handles multiple web workers reviewing the same game.

import { EngineLine } from '../types/Game';
import { getFenArr } from './convert';

class GameReviewManager {
  private current_index: number;
  private un_evaluated_sanmoves: string[];
  private engine_responses: { engineLines: EngineLine[]; move_num: number }[];
  private fen_arr: string[];
  private num_of_workers: Number;
  private done_workers_count = 0;
  get_next_move() {
    const current_fen = this.fen_arr[this.current_index];
    const sanmove = this.un_evaluated_sanmoves[this.current_index];
    const move_num = this.current_index;
    this.current_index++;
    return { sanmove, move_num, current_fen };
  }
  // to-do --> test
  done_evaluating() {
    return (
      this.done_workers_count == this.num_of_workers 
      // &&this.engine_responses.length == this.un_evaluated_sanmoves.length
    );
  }

  add_enginelines(engineLines: EngineLine[], move_num: number) {
    this.engine_responses.push({ engineLines, move_num });
  }

  get_engine_response() {
    return this.engine_responses;
  }

  worker_done_evaluating() {
    this.done_workers_count += 1;
  }

  constructor(un_evaluated_sanmoves: string[], number_of_workers: Number) {
    this.current_index = 0;
    this.un_evaluated_sanmoves = un_evaluated_sanmoves;
    this.engine_responses = [];
    this.fen_arr = getFenArr(un_evaluated_sanmoves);
    this.num_of_workers = number_of_workers;
  }
}

export default GameReviewManager;

/frontengTS/src/scripts/_StockfishWorker.ts: Interface for interacting with StockFish.js using UCI PROTOCOL commands.

import type { EngineLine, Evaluation, Lan } from '../types/Game';
import type GameReviewManager from './_GameReviewManager';

class StockfishWorker {
  private workerUrl: URL;
  private stockfishWorker: Worker;
  private verbose: boolean;
  private multipv: number;
  private targetDepth: number;
  readonly game_review_manager: GameReviewManager;

  private waitFor(response: string, errormsg = 'error') {
    return new Promise((resolve, reject) => {
      const listener = <K extends keyof WorkerEventMap>(
        e: WorkerEventMap[K],
      ) => {
        if (this.verbose) console.debug(e);
        if (e instanceof MessageEvent && e.data.includes(response)) {
          this.stockfishWorker.removeEventListener('message', listener);
          resolve(true);
        }
      };
      this.stockfishWorker.addEventListener('message', listener);
      // Add a timeout for error handling (optional)
      setTimeout(() => {
        this.stockfishWorker.removeEventListener('message', listener);
        reject(`delay time exceeded: ${errormsg}`);
      }, 5000);
    });
  }

  _init(restart = false) {
    return new Promise(async (resolve) => {
      this.stockfishWorker.onmessageerror = (e) => console.debug(e);
      if (!restart) {
        this.stockfishWorker.postMessage('uci');
        await this.waitFor('uciok', 'uci setup error');
      } else console.debug('Restarting engine...');
      this.stockfishWorker.postMessage(`ucinewgame`);
      this.stockfishWorker.postMessage('isready');
      this.stockfishWorker.postMessage(
        `setoption name MultiPV value ${this.multipv}`,
      );
      this.waitFor('readyok', 'this.stockfishWorker not ready after timeout')
        .then(() => {
          resolve(true);
        })
        .catch((err) => {
          throw new Error(err);
        });
    });
  }

  evaluateMove(fen: string = 'startpos'): Promise<EngineLine[]> {
    console.log('evaluate move');
    !fen || fen == 'startpos'
      ? this.stockfishWorker.postMessage(`position startpos`)
      : this.stockfishWorker.postMessage(`position fen ${fen}`);
    this.stockfishWorker.postMessage(`go depth ${this.targetDepth}`);

    let messages = [];
    let lines: EngineLine[] = [];
    return new Promise((resolve, reject) => {
      const listener = <K extends keyof WorkerEventMap>(
        e: WorkerEventMap[K],
      ) => {
        if (e instanceof MessageEvent) {
          messages.unshift(e.data);
          if (e.data.includes('depth 0')) {
            if (this.verbose) console.warn(`${e}`);
          }
          if (e.data.startsWith('bestmove') || e.data.includes('depth 0')) {
            this.stockfishWorker.removeEventListener('message', listener);
            let searchMessages = messages.filter((msg) =>
              msg.startsWith('info depth'),
            );
            for (let searchMessage of searchMessages) {
              // Extract depth, MultiPV line ID and evaluation from search message
              let idString = searchMessage.match(/(?:multipv )(\d+)/)?.[1];
              let depthString = searchMessage.match(/(?:depth )(\d+)/)?.[1];

              let bestMove: Lan =
                searchMessage.match(/(?: pv )(.+?)(?= |$)/)?.[1];

              let evaluation: Evaluation = {
                type: searchMessage.includes(' cp ') ? 'cp' : 'mate',
                value: parseInt(
                  searchMessage.match(/(?:(?:cp )|(?:mate ))([\d-]+)/)?.[1] ||
                    '0',
                ),
              };
              // Invert evaluation if black to play since scores are from black perspective
              // and we want them always from the perspective of white
              if (fen && fen.includes(' b ')) {
                evaluation.value *= -1;
              }
              // If any piece of data from message is missing, discard message
              if (!idString || !depthString || !bestMove) {
                lines.push(
                  {
                    id: parseInt(idString),
                    depth: parseInt(depthString),
                    evaluation: { type: 'mate', value: 0 },
                    bestMove: 'a1a1',
                  },
                  {
                    id: parseInt(idString),
                    depth: parseInt(depthString),
                    evaluation: { type: 'mate', value: 0 },
                    bestMove: 'a1a1',
                  },
                );
                resolve(lines);
                return;
              }

              let id = parseInt(idString);
              let depth = parseInt(depthString);

              // Discard if target depth not reached or lineID already present
              if (
                depth != this.targetDepth ||
                lines.some((line) => line.id == id)
              )
                continue;

              lines.push({
                id,
                depth,
                evaluation,
                bestMove,
              });
            }
            clearTimeout(timeoutId); // Clear the timeout if message is received
            console.debug(lines);
            console.debug('cleared time out id');
            resolve(lines);
          }
        }
      };
      this.stockfishWorker.addEventListener('message', listener);
      const timeoutId = setTimeout(() => {
        this.stockfishWorker.removeEventListener('message', listener);
        reject(new Error('takes alot of time'));
      }, 20000); // Adjust timeout as needed
    });
  }

  private restartWorker(
    ui_update_callback: () => void,
    move: {
      current_fen: string;
      move_num: number;
      sanmove: string;
    },
  ) {
    console.log('restarting worker....');
    this.stockfishWorker.terminate();
    this.stockfishWorker = new Worker(this.workerUrl, { type: 'classic' });
    this._init().then(() => {
      this.evaluateStuckMove(ui_update_callback, move).then((res) => {
        if (res.success) {
          console.warn('continue with work');
          this.evaluatePosition(ui_update_callback);
        } else {
          console.error('restarting worker again');
          this.restartWorker(ui_update_callback, move);
        }
      });
    });
  }

  private async evaluateStuckMove(
    ui_update_callback: () => void,
    move: {
      current_fen: string;
      move_num: number;
      sanmove: string;
    },
  ) {
    try {
      console.warn('evaluating halt move', move);
      const engineLines = await this.evaluateMove(move.current_fen);
      ui_update_callback();
      this.game_review_manager.add_enginelines(engineLines, move.move_num);
      return { success: true };
    } catch (error) {
      return { success: false };
    }
  }

  async evaluatePosition(ui_update_callback: () => void) {
    console.log('evaluating position');
    while (!this.game_review_manager.done_evaluating()) {
      const currentMove = this.game_review_manager.get_next_move();
      const { current_fen, move_num, sanmove } = currentMove;
      console.log({ current_fen, move_num, sanmove });
      try {
        const engineLines = await this.evaluateMove(current_fen);
        console.log(`engine lines: ${engineLines}`);  
        // ui_update_callback();
        // this.game_review_manager.add_enginelines(engineLines, move_num);
      } catch (error) {
        console.error('restarting');
        // this.restartWorker(ui_update_callback, currentMove);
      }
    }
    console.log('terminate_worker');
    this.stockfishWorker.terminate();
  }

  /**
   * @param startingpos : fen format
   * @param multipv : number of lines to return
   * @param verbose : testing
   */
  constructor(
    game_review_manager: GameReviewManager,
    multipv = 2,
    targetDepth = 15,
    verbose = false,
  ) {
    var wasmSupported =
      typeof WebAssembly === 'object' &&
      WebAssembly.validate(
        Uint8Array.of(0x0, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00),
      );

    this.workerUrl = wasmSupported
      ? new URL('./stockfish/stockfish.js', import.meta.url)
      : new URL('./stockfish/stockfish.wasm.js', import.meta.url);
    this.stockfishWorker = new Worker(this.workerUrl, { type: 'classic' });
    this.multipv = multipv;
    //this.evaluateMove = this.evaluateMove.bind(this);
    this.targetDepth = targetDepth;
    this.verbose = verbose;
    //this.waitFor = this.waitFor.bind(this);
    this.game_review_manager = game_review_manager;
  }
}
export default StockfishWorker;

/frontengTS/src/routes/ReviewGame.tsx: React component that initiates the game review process.

import Loading_Review_LargeScreen from '../components/Labtop_loading';
import Loading_Review_SmallScreen from '../components/Phone_loading';
import ReviewResult from '../components/ReviewResult';
import { useContext, useEffect, useRef, useState } from 'react';
import { ReviewGameContext } from '../contexts/ReviewGameContext';
import styles from '../styles/ReviewGame.module.css';
import reviewResultStyles from '../styles/ReviewResult.module.css';
import { UserContext } from '../contexts/UserContext';
import { useParams, useLocation } from 'react-router-dom';
import { Classify } from '../scripts/_Classify';
import { EngineLine, Game } from '../types/Game';
import { ClassificationScores, ClassName } from '../types/Review';
import { constructPgn, getMoves, parsePgn } from '../scripts/pgn';
import { GameContext } from '../contexts/GamesContext';
import ChessBoard_Eval from '../components/ChessBoard_Eval';
import { ChessBoardContextProvider } from '../contexts/GameBoardContext';
import GameReviewManager from '../scripts/_GameReviewManager';
import StockfishWorker from '../scripts/_StockfishWorker';
 useEffect(() => {
let number_of_workers = 3;
      const game_review_manager = new GameReviewManager(
        parsedGameData.moves,
        number_of_workers,
      );
      for (let i = 0; i < number_of_workers; i++) {
        const stockfish_worker = new StockfishWorker(
          game_review_manager,
          1,
          18,
        );
        stockfish_worker._init().then(() => {
          console.log('done initializing');
          //stockfish_worker.evaluateMove().then((res) => console.log(res));
          stockfish_worker
            .evaluatePosition(() => {
              console.log(`worker ${i + 1}`);
              setCurrentPerc((old) => old + 1);
            })
            .then(() => {
              game_review_manager.worker_done_evaluating();
              if (game_review_manager.done_evaluating()) {
                /* let moveClassification =
                await ClassificationHelper.getMoveClassification({
                  engineResponse: param.lines,
                  moveNum: param.moveNum,
                  plColor: param.moveNum % 2 ? 1 : -1,
                  gameInfo: gameData,
                  initial_Evaluation: initalEvaluation,
                }); */
                console.log('done evaluating');
                console.log(game_review_manager.get_engine_response());
                /* setMovesClassifications(
                  getClassificationScore(
                    parsedGameData.classifi.map((c) => c.name),
                  ),
                ); */
                console.log('all workers are done');
              } else {
                console.log('all workers are not done');
              }
            });
        });
      }
    }
  }, [location]);

How It Works: StockFish Worker Interface: Provides a wrapper to interact with StockFish.js. Includes functions for: Initializing the worker. Sending moves to the engine. Changing engine settings (e.g., adjusting MultiPV for multiple evaluation lines). Game Review Manager: Manages the game review process by: Distributing moves to multiple web workers. Tracking progress to ensure all moves are evaluated. Aggregating results when all evaluations are complete.

Problems: Delayed Response with MultiPV:

When MultiPV > 1, StockFish often takes a long time to respond. I implemented a timeout mechanism that terminates the worker if it doesn’t respond in time, but this causes frequent exits. The issue doesn’t occur consistently when MultiPV = 1. game review manager seem to not exit correctly after each webworker finished.

The Game Review Manager distributes moves incrementally to workers and ensures no two workers process the same move. However, workers occasionally get stuck, leading to incomplete evaluations.

What I’ve Tried: Debugging Worker Communication:

Verified that moves are correctly sent to workers. Confirmed that no worker is idle when moves are pending. Timeout Adjustments:

Increased the timeout duration for workers, but the issue persists (just delayed). StockFish.js Settings:

Checked for misconfigurations in engine settings related to MultiPV.