I Built an AI Arena and Trained AlphaZero to Play Gomoku: Here’s How

Building a board game AI is a fantastic way to dive into Reinforcement Learning and search algorithms. But once you've built your AI, a new problem arises: How do you actually test it against other algorithms? If you write a classic Minimax agent in Java and an AlphaZero model in Python, how do you make them fight?

To solve this, I built a two-part ecosystem:

1. 
   : A lightweight, readable AlphaZero implementation trained to dominate the arena.

Here is a deep dive into the architecture of both projects and how they work together.

🏟️ Part 1: The Arena (Gomoku Battle)

GitHub:

To beat the Alpha-Beta baseline, I implemented the algorithm that conquered Go and Chess: AlphaZero.

Many AlphaZero repositories out there are either overly complex, tied heavily to one specific game, or require massive compute just to see a working demo. I built alphazero-board-games to be clean, modular, and instantly playable.

The Implementation Details

The project is built with Python 3.12+ and uses a modular architecture:

1. 
   Shared Core (alphazero/): This is the heart of the engine. It contains the abstract Game API, the Monte Carlo Tree Search (MCTS) implementation, the Neural Network definitions (Residual Policy/Value networks), and the self-play Reinforcement Learning loop.


2. 
   Game Presets: I implemented the specific rules for gomoku_9_9, gomoku_15_15, and connect4.

Instead of traditional rollouts, the MCTS in this project traverses the tree until it reaches a leaf node, then queries the Residual Network. The network outputs two things:

• 
  Policy ($p$): A probability distribution over possible moves (where to look).


• 
  Value ($v$): An evaluation of the current board state from , write a quick script in your favorite language, and see if it can beat my baseline.


• Want to learn how AlphaZero actually works under the hood, or train an AI to play Connect4? Check out AlphaZero Board Games.

If you find the projects interesting or helpful for learning AI and system design, I'd love it if you gave them a ⭐️ on GitHub! Let me know in the comments if you have any questions about the MCTS implementation or the JavaFX integration! Happy coding! 👨‍💻♟️