Description
When analyzing a classic ancient Chinese Go game using the latest KaTrain with KataGo TRT backend, I encountered a significant and reproducible miscalculation in a relatively ordinary life-and-death situation.
As shown in Figure 1, Black plays a hane on the right side. KataGo suggests two responses. Although the correct solution is among them, the point difference between the wrong response and the correct one is much larger than the 0.5 or 0.4 displayed by KataGo.
Strangely, when playing moves quickly, KataGo can find the correct solution. However, when calculating move by move, it fails to detect the problem with the wrong response. For five consecutive moves, KataGo consistently gives incorrect point difference evaluations with very high confidence (see Figure 2).
On the 6th move, when Black plays a normal tsuke (尖), the software suddenly realizes the miscalculation, and the point difference changes dramatically (Figure 3). After that, KataGo starts frantically searching empty points across the entire board, causing large negative winrate swings (red indicators) to appear all over the board (Figure 4).
This variation is quite ordinary and not an obscure "magic move". The correct move is a clever "ugly shape" tesuji, but the issue occurs in another branch. Theoretically, such an error should have been discovered within 6 moves, yet KataGo failed to notice it in the first 5 moves with high confidence.
Even if I restart the analysis on the same position, KataGo does not remember the previous miscalculation and repeats the same error. This issue can be consistently reproduced.
Additional Information
After first discovering this bug in the early morning, I was able to reproduce it multiple times. Unfortunately, because it was very late at night, I did not take screenshots at the time.
The next day, when I tried to report the bug, I could no longer fully reproduce the exact same behavior. However, the issue can still be partially observed during analysis:
Black’s point difference increases sharply and unexpectedly at several steps. Sudden jumps can be seen in the positions shown in Figure 2 and Figure 5.
When the bug originally occurred:
White’s evaluation in Figure 3 was only -0.5 (instead of the current -5.9).
Black’s evaluation in Figure 4 suddenly surged to +15.7, immediately triggering widespread red indicators as KataGo desperately searched every empty point on the board in an attempt to find a response.
Environment
UI: Latest KaTrain version
Engine: Latest KataGo TRT version 1.16.4
Model: Latest strongest weight [kata1-b28c512nbt-s12674021632-d5782420041]
Analysis settings:
Single move analysis: 40,000 visits
Quick play: 8,000 visits per move
Wide Root Noise: 0.04
numAnalysisThreads = 6
numSearchThreadsPerAnalysisThread = 32
nnMaxBatchSize = 96
Backend: TensorRT
OS: Windows 11
GPU：RTX 3080
Steps to Reproduce
Load the ancient Chinese game SGF in KaTrain.
Navigate to the problem position (Black's hane on the right side).
Perform move-by-move analysis using 40,000 visits per move.
Follow the hane sequence and observe the evaluation for the first 5–6 moves.
Expected Behavior
KataGo should correctly evaluate the point difference and detect the bad variation within a few moves at 40,000 visits.
Actual Behavior
KataGo shows confident but clearly wrong point difference for 5 consecutive moves, then suddenly corrects on the 6th move with dramatic winrate swings and widespread search across the board.
Attachments
Screenshots: Figure 1, Figure 2, Figure 3, Figure 4
Engine log excerpt (optional)
05.zip
Description
When analyzing a classic ancient Chinese Go game using the latest KaTrain with KataGo TRT backend, I encountered a significant and reproducible miscalculation in a relatively ordinary life-and-death situation.
As shown in Figure 1, Black plays a hane on the right side. KataGo suggests two responses. Although the correct solution is among them, the point difference between the wrong response and the correct one is much larger than the 0.5 or 0.4 displayed by KataGo.
Strangely, when playing moves quickly, KataGo can find the correct solution. However, when calculating move by move, it fails to detect the problem with the wrong response. For five consecutive moves, KataGo consistently gives incorrect point difference evaluations with very high confidence (see Figure 2).
On the 6th move, when Black plays a normal tsuke (尖), the software suddenly realizes the miscalculation, and the point difference changes dramatically (Figure 3). After that, KataGo starts frantically searching empty points across the entire board, causing large negative winrate swings (red indicators) to appear all over the board (Figure 4).
This variation is quite ordinary and not an obscure "magic move". The correct move is a clever "ugly shape" tesuji, but the issue occurs in another branch. Theoretically, such an error should have been discovered within 6 moves, yet KataGo failed to notice it in the first 5 moves with high confidence.
Even if I restart the analysis on the same position, KataGo does not remember the previous miscalculation and repeats the same error. This issue can be consistently reproduced.
Additional Information
After first discovering this bug in the early morning, I was able to reproduce it multiple times. Unfortunately, because it was very late at night, I did not take screenshots at the time.
The next day, when I tried to report the bug, I could no longer fully reproduce the exact same behavior. However, the issue can still be partially observed during analysis:
Black’s point difference increases sharply and unexpectedly at several steps. Sudden jumps can be seen in the positions shown in Figure 2 and Figure 5.
When the bug originally occurred:
White’s evaluation in Figure 3 was only -0.5 (instead of the current -5.9).
Black’s evaluation in Figure 4 suddenly surged to +15.7, immediately triggering widespread red indicators as KataGo desperately searched every empty point on the board in an attempt to find a response.
Environment
UI: Latest KaTrain version
Engine: Latest KataGo TRT version 1.16.4
Model: Latest strongest weight [kata1-b28c512nbt-s12674021632-d5782420041]
Analysis settings:
Single move analysis: 40,000 visits
Quick play: 8,000 visits per move
Wide Root Noise: 0.04
numAnalysisThreads = 6
numSearchThreadsPerAnalysisThread = 32
nnMaxBatchSize = 96
Backend: TensorRT
OS: Windows 11
GPU：RTX 3080
Steps to Reproduce
Load the ancient Chinese game SGF in KaTrain.
Navigate to the problem position (Black's hane on the right side).
Perform move-by-move analysis using 40,000 visits per move.
Follow the hane sequence and observe the evaluation for the first 5–6 moves.
Expected Behavior
KataGo should correctly evaluate the point difference and detect the bad variation within a few moves at 40,000 visits.
Actual Behavior
KataGo shows confident but clearly wrong point difference for 5 consecutive moves, then suddenly corrects on the 6th move with dramatic winrate swings and widespread search across the board.
Attachments
Screenshots: Figure 1, Figure 2, Figure 3, Figure 4
Engine log excerpt (optional)
05.zip