NeoChess

NeoChess is a self-learning chess engine written in Python. It uses PyTorch to build a neural network that evaluates chess positions, and it learns by playing games against the Stockfish engine and itself. The core learning mechanism is based on reinforcement learning principles, where the model is rewarded for winning games and penalized for losing. Note that this game was played with the parameter epsilon being set to inf

How It Works

The training process is orchestrated by the selfchess.py script, which follows these steps:

1. Game Simulation: The engine plays a large number of chess games. The games are divided into three categories:

   • NeoChess (as White) vs. Stockfish (as Black)
   • NeoChess (as Black) vs. Stockfish (as White)
   • NeoChess vs. NeoChess (self-play)

2. Parallel Processing: To speed up data generation, games are simulated in parallel using Python's multiprocessing library, utilizing available CPU cores.

3. Move Selection:

   • NeoChess: Uses a negamax search algorithm (search) to explore future moves. The evaluation of terminal positions in the search is provided by its neural network.
   • Stockfish: A standard, powerful chess engine provides the moves for the opponent.

4. Data Collection: During each game, every board position (FEN string) where it is NeoChess's turn to move is stored.

5. Training: After a game concludes, a reward is assigned: +10 for a win, -10 for a loss, and 0 for a draw. The neural network is then trained on the collected board positions from that game. The training target for each position is weighted by the final game outcome, encouraging the model to value positions that lead to wins.

6. Model Saving: The model's state (chessy_model.pth) is saved after each game. A backup (chessy_modelt-1.pth) is also kept and updated periodically.

Model Architecture

The brain of NeoChess is a neural network (NN1 class) with the following structure:

• Embedding Layer: Converts the board's piece representation into a 64-dimensional vector space.
• Multi-Head Attention: An attention mechanism allows the model to weigh the importance of different pieces and their relationships on the board.
• Feed-Forward Network: A deep series of linear layers and ReLU activation functions process the features from the attention layer to produce a final evaluation score for the position.

Requirements

• Python 3.x
• PyTorch
• python-chess library
• A UCI-compatible chess engine binary (e.g., Stockfish)

You can install the Python dependencies using pip:

pip install torch python-chess

Setup and Usage

1. Download Stockfish: Download the appropriate Stockfish binary for your system from the official website.

2. Configure the Script: Open selfchess.py and edit the CONFIG dictionary at the top of the file:

   • stockfish_path: Set this to the absolute path of your downloaded Stockfish executable.
   • model_path: The name of the file to save the primary model.
   • backup_model_path: The name of the file for the backup model.
   • Adjust other parameters like num_games, learning_rate, etc., as needed.

3. Run the Training: Execute the script from your terminal:

   python selfchess.py
   

The script will then begin the training process, printing the status of each game and the training loss.