Skip to content

feat: Add Python 3.10-3.14 syntax support #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
phodal merged 2 commits into from
Jan 23, 2026
+884 −4
Merged

feat: Add Python 3.10-3.14 syntax support #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3387692
feat: Add Python 3.10-3.14 syntax support
phodal Jan 23, 2026
59fba90
test: Improve test coverage for PythonVersion and add edge case tests
phodal Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion chapi-ast-python/src/main/antlr/PythonLexer.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,13 @@ EXEC : 'exec';
TRUE : 'True';
FALSE : 'False';

// Python 3.10+ soft keywords (can also be used as identifiers)
MATCH : 'match';
CASE : 'case';

// Python 3.12+ soft keywords
TYPE : 'type';

// Operators

DOT : '.';
Expand Down Expand Up @@ -118,7 +125,8 @@ RIGHT_SHIFT_ASSIGN : '>>=';
POWER_ASSIGN : '**=';
IDIV_ASSIGN : '//=';

STRING : ([uU] | [fF] [rR]? | [rR] [fF]?)? (SHORT_STRING | LONG_STRING)
// String literals including Python 3.14 t-strings
STRING : ([uU] | [fF] [rR]? | [rR] [fF]? | [tT] [rR]? | [rR] [tT])? (SHORT_STRING | LONG_STRING)
| ([bB] [rR]? | [rR] [bB]) (SHORT_BYTES | LONG_BYTES)
;

Expand Down
187 changes: 185 additions & 2 deletions chapi-ast-python/src/main/antlr/PythonParser.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ compound_stmt
| TRY COLON suite (except_clause+ else_clause? finally_clause? | finally_clause) #try_stmt
| ASYNC? WITH with_item (COMMA with_item)* COLON suite #with_stmt
| decorator* (classdef | funcdef) #class_or_func_def_stmt
| match_stmt #match_stmt_wrapper
;

suite
Expand All @@ -87,23 +88,199 @@ finally_clause
: FINALLY COLON suite
;

// Python 3.10+ match statement (PEP 634)
match_stmt
: MATCH subject_expr COLON LINE_BREAK INDENT case_block+ DEDENT
;

subject_expr
: star_named_expression COMMA star_named_expressions?
| named_expression
;

star_named_expressions
: (COMMA star_named_expression)+ COMMA?
;

star_named_expression
: STAR expr
| named_expression
;

named_expression
: name ASSIGN test
Copy link

@augmentcode augmentcode bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

named_expression is defined as name ASSIGN test, but ASSIGN is = in the lexer, so valid walrus subjects (:=) in match statements won’t parse and invalid name = expr could be accepted in expression position.

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎

| test
;

case_block
: CASE pattern guard? COLON suite
;

guard
: IF test
;

// Pattern matching patterns
pattern
: or_pattern
| as_pattern
;
Comment on lines +124 to +127
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pattern matching grammar has a structural issue. The pattern rule includes both or_pattern and as_pattern, but as_pattern is defined as or_pattern AS pattern_capture_target. This means:

  1. pattern can be as_pattern
  2. as_pattern includes or_pattern
  3. or_pattern includes closed_pattern
  4. closed_pattern can be many things including group_pattern
  5. group_pattern is OPEN_PAREN pattern CLOSE_PAREN

However, looking at PEP 634, the as_pattern should be at the same level as or_pattern, not nested within pattern. The current structure is:

pattern: or_pattern | as_pattern
as_pattern: or_pattern AS pattern_capture_target

This is actually correct per PEP 634, but the rule should ensure that as_pattern takes precedence in the alternatives. Consider reordering to put as_pattern first to ensure it's matched before falling back to or_pattern.

Copilot uses AI. Check for mistakes.

as_pattern
: or_pattern AS pattern_capture_target
;

or_pattern
: closed_pattern (OR_OP closed_pattern)*
;

closed_pattern
: literal_pattern
| capture_pattern
| wildcard_pattern
| class_pattern
| sequence_pattern
| mapping_pattern
| group_pattern
;
Comment on lines +137 to +145
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an ambiguity between capture_pattern and wildcard_pattern in the closed_pattern alternatives. Both rules will match any NAME token:

  • capture_pattern expands to pattern_capture_target which is name which includes NAME
  • wildcard_pattern is directly NAME

When the parser encounters a NAME token in a pattern context, ANTLR won't know whether to treat it as a capture_pattern or wildcard_pattern. This ambiguity should be resolved by:

  1. Removing one of the alternatives (since wildcard is meant to only match '_', it should be handled differently)
  2. Using a semantic predicate to distinguish '_' from other names
  3. Restructuring to have a single 'name_pattern' that's semantically interpreted

This is related to the wildcard_pattern issue noted separately, but the ambiguity in closed_pattern makes it a more serious parsing concern.

Copilot uses AI. Check for mistakes.

literal_pattern
: MINUS? number
| STRING+
| NONE
| TRUE
| FALSE
;

capture_pattern
: pattern_capture_target
;

pattern_capture_target
: name
;

wildcard_pattern
: NAME // Matches '_' specifically, handled semantically
Copy link

@augmentcode augmentcode bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wildcard_pattern : NAME overlaps with capture_pattern/pattern_capture_target (which also start from an identifier), so _ vs capture can become ambiguous and wildcard_pattern may never appear in the parse tree as intended.

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎

Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment states this matches '_' specifically and is "handled semantically", but there's no semantic action or validation in the grammar rule itself. This could be misleading to future maintainers.

Either add a semantic predicate to actually check for '_' at parse time, or update the comment to clarify that this semantic check must be performed in the listener/visitor code that processes the AST. The comment should reference where this semantic validation occurs.

Suggested change
: NAME // Matches '_' specifically, handled semantically
: NAME // Matches a NAME; treating '_' as the wildcard pattern is done in later semantic analysis (e.g. in listeners/visitors processing match patterns)

Copilot uses AI. Check for mistakes.
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wildcard_pattern rule is defined as just NAME, with a comment stating it "Matches '_' specifically, handled semantically". This implementation is incorrect because it will match ANY name token, not just the underscore wildcard.

This means patterns like case foo: where foo is intended to be a capture pattern would incorrectly be parsed as a wildcard pattern. The wildcard pattern should either:

  1. Be checked semantically to ensure the NAME is specifically '_', or
  2. Use a more specific lexer token or parser predicate to match only '_'

The current implementation could lead to incorrect AST generation where any capture pattern is treated as a wildcard.

Suggested change
: NAME // Matches '_' specifically, handled semantically
: {getCurrentToken().getText().equals("_")}? NAME // Matches '_' specifically

Copilot uses AI. Check for mistakes.
;
Comment on lines +137 to +165
Copy link

@coderabbitai coderabbitai bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat -n chapi-ast-python/src/main/antlr/PythonParser.g4 | sed -n '137,165p'

Repository: phodal/chapi

Length of output: 701

🏁 Script executed:

# Check for wildcard and capture pattern usage in visitors and analyzers
rg -n "wildcard_pattern|capture_pattern" -A 3 -B 1 --type java --type python

Repository: phodal/chapi

Length of output: 142

🏁 Script executed:

# Check if there's semantic handling for _ in the codebase
rg -n "equals.*_|_.*equals" --type java -A 2 -B 1 | head -50

Repository: phodal/chapi

Length of output: 1795

🏁 Script executed:

rg -n "^\s*name\s*:" chapi-ast-python/src/main/antlr/PythonParser.g4 -A 5

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

fd -t f "PythonParserBase\|PythonVisitor" chapi-ast-python/

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

fd -t f "\.g4" chapi-ast-python/src/main/antlr/

Repository: phodal/chapi

Length of output: 150

🏁 Script executed:

# Search for any semantic predicate or special handling of underscore in Python parser
rg -n "TEXT\|semantic" chapi-ast-python/src/main/antlr/PythonParser.g4 -i | head -20

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

# Search in entire PythonParser.g4 for name rule definition
rg -n "^name\s*:" chapi-ast-python/src/main/antlr/PythonParser.g4 -A 5

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

# Also search with whitespace variations
grep -n "^\s*name\s*:" chapi-ast-python/src/main/antlr/PythonParser.g4 -A 5

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

# Get entire PythonParser.g4 to understand structure
wc -l chapi-ast-python/src/main/antlr/PythonParser.g4

Repository: phodal/chapi

Length of output: 107

🏁 Script executed:

# Search for pattern_capture_target definition and what it actually calls
grep -n "pattern_capture_target\|NAME\|IDENTIFIER" chapi-ast-python/src/main/antlr/PythonParser.g4 | head -40

Repository: phodal/chapi

Length of output: 341

🏁 Script executed:

# Get lines around pattern definitions
cat -n chapi-ast-python/src/main/antlr/PythonParser.g4 | sed -n '155,170p'

Repository: phodal/chapi

Length of output: 427

🏁 Script executed:

# Get the name rule definition
cat -n chapi-ast-python/src/main/antlr/PythonParser.g4 | sed -n '488,495p'

Repository: phodal/chapi

Length of output: 232

🏁 Script executed:

# Check if there are any visitor classes generated and how they handle wildcard_pattern
find chapi-ast-python -name "*Visitor*.java" -o -name "*BaseVisitor*.java" | head -5

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

# Check for any Python files in the codebase that might handle AST traversal
find chapi-ast-python -type f -name "*.py" | head -10

Repository: phodal/chapi

Length of output: 646

🏁 Script executed:

# Search for wildcard_pattern references in Java source code
find chapi-ast-python/src -name "*.java" -type f -exec grep -l "wildcard_pattern" {} \;

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

# Search more broadly for any references to wildcard in the codebase
rg "wildcard" chapi-ast-python/src --type java -i

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

# Check if there's a base listener/visitor pattern in chapi for Python
find chapi-ast-python/src/main -type f -name "*.java" | head -20

Repository: phodal/chapi

Length of output: 258

🏁 Script executed:

cat chapi-ast-python/src/main/java/chapi/ast/antlr/PythonParserBase.java

Repository: phodal/chapi

Length of output: 741

🏁 Script executed:

# Search for any pattern handling in the entire module
rg "pattern" chapi-ast-python/src/main/java -i -l

Repository: phodal/chapi

Length of output: 121

Wildcard pattern is unreachable due to capture_pattern precedence.

Both capture_pattern and wildcard_pattern ultimately match the NAME token (capture_pattern via pattern_capture_target → name → NAME), and since capture_pattern appears first in the ordered choice on line 139, it will always match before wildcard_pattern gets evaluated. This prevents wildcard patterns (specifically _) from ever being parsed as wildcard_pattern nodes. Python 3.10+ match statements require proper wildcard pattern support, so downstream code expecting wildcard_pattern AST nodes will never receive them.

Reorder wildcard_pattern before capture_pattern in the closed_pattern rule and add a semantic predicate to wildcard_pattern to restrict it to the _ identifier:

Proposed fix 
-closed_pattern
-    : literal_pattern
-    | capture_pattern
-    | wildcard_pattern
+closed_pattern
+    : literal_pattern
+    | wildcard_pattern
+    | capture_pattern
     | class_pattern
     | sequence_pattern
     | mapping_pattern
     | group_pattern
     ;

-wildcard_pattern
-    : NAME  // Matches '_' specifically, handled semantically
+wildcard_pattern
+    : name=NAME { $name.text.equals("_") }?
     ;
🤖 Prompt for AI Agents 
In `@chapi-ast-python/src/main/antlr/PythonParser.g4` around lines 137 - 165, The
closed_pattern rule currently places capture_pattern before wildcard_pattern
causing NAME-based captures to shadow the '_' wildcard; fix this by moving
wildcard_pattern above capture_pattern in closed_pattern and restrict
wildcard_pattern to only match the underscore identifier by adding a semantic
predicate or explicit text check (e.g., in wildcard_pattern ensure the token
matched by NAME equals "_" instead of any NAME). Update the rules referenced:
closed_pattern, wildcard_pattern, capture_pattern (and its components
pattern_capture_target → name → NAME) so wildcard_pattern is evaluated first and
only produces a wildcard_pattern node when the NAME token is "_" .


class_pattern
: dotted_name OPEN_PAREN pattern_arguments? CLOSE_PAREN
;

pattern_arguments
: positional_patterns COMMA keyword_patterns COMMA?
| positional_patterns COMMA?
| keyword_patterns COMMA?
;

positional_patterns
: pattern (COMMA pattern)*
;

keyword_patterns
: keyword_pattern (COMMA keyword_pattern)*
;

keyword_pattern
: name ASSIGN pattern
;

sequence_pattern
: OPEN_BRACKET maybe_sequence_pattern? CLOSE_BRACKET
| OPEN_PAREN open_sequence_pattern? CLOSE_PAREN
;

open_sequence_pattern
: maybe_star_pattern COMMA maybe_sequence_pattern?
;

maybe_sequence_pattern
: maybe_star_pattern (COMMA maybe_star_pattern)* COMMA?
;

maybe_star_pattern
: star_pattern
| pattern
;

star_pattern
: STAR (pattern_capture_target | wildcard_pattern)
;

mapping_pattern
: OPEN_BRACE items_pattern? CLOSE_BRACE
;

items_pattern
: key_value_pattern (COMMA key_value_pattern)* COMMA?
;

key_value_pattern
: (literal_pattern | attr) COLON pattern
| double_star_pattern
;

double_star_pattern
: POWER pattern_capture_target
;

attr
: dotted_name
;

group_pattern
: OPEN_PAREN pattern CLOSE_PAREN
;

with_item
// NB compile.c makes sure that the default except clause is last
: test (AS expr)?
;

// Python 2 : EXCEPT test COMMA name
// Python 3 : EXCEPT test AS name
// Python 3.14+ (PEP 758): EXCEPT test, test, test without parentheses (only WITHOUT AS clause)
except_clause
: EXCEPT (test ({CheckVersion(2)}? COMMA name {SetVersion(2);} | {CheckVersion(3)}? AS name {SetVersion(3);})?)? COLON suite
| EXCEPT except_types COLON suite // Python 3.14+ without parentheses (no AS clause allowed)
Copy link

@augmentcode augmentcode bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Autodetect mode, CheckVersion(2) succeeds, so except ValueError, TypeError: is likely to match the Python 2 COMMA name branch (and call SetVersion(2)) instead of the new except_types alternative. That would misclassify Python 3.14 bracketless-except as Python 2 and change its meaning.

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎

Comment on lines 98 to +246
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The except_clause grammar has a potential ambiguity issue. Both alternatives start with EXCEPT test which could cause parsing conflicts:

  1. First alternative: EXCEPT (test ...)? COLON suite - allows zero or one test
  2. Second alternative: EXCEPT except_types COLON suite - requires test (COMMA test)+

When the parser sees EXCEPT test COMMA test COLON, it could match either:

  • The first alternative with the Python 2 path: EXCEPT test COMMA name (if test is followed by COMMA name)
  • The second alternative: EXCEPT except_types (if test is followed by COMMA test)

This ambiguity could lead to incorrect parsing. Consider reordering the alternatives or making the second alternative more specific to avoid conflicts. The more specific alternative (except_types requiring at least two exceptions) should typically come before the more general one.

Suggested change
: EXCEPT (test ({CheckVersion(2)}? COMMA name {SetVersion(2);} | {CheckVersion(3)}? AS name {SetVersion(3);})?)? COLON suite
| EXCEPT except_types COLON suite // Python 3.14+ without parentheses (no AS clause allowed)
: EXCEPT except_types COLON suite // Python 3.14+ without parentheses (no AS clause allowed)
| EXCEPT (test ({CheckVersion(2)}? COMMA name {SetVersion(2);} | {CheckVersion(3)}? AS name {SetVersion(3);})?)? COLON suite

Copilot uses AI. Check for mistakes.
;

// Python 3.14+ allows multiple exception types without parentheses
except_types
: test (COMMA test)+
;
Comment on lines 241 to 252
Copy link

@coderabbitai coderabbitai bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# Search for version gating patterns in the grammar file
rg -n "CheckVersion|SetVersion|supportsPython314|PythonVersion" chapi-ast-python/src/main/antlr/PythonParser.g4 | head -50

Repository: phodal/chapi

Length of output: 704

🏁 Script executed:

# Get context around the except_clause and except_types rules
sed -n '230,260p' chapi-ast-python/src/main/antlr/PythonParser.g4

Repository: phodal/chapi

Length of output: 963

🏁 Script executed:

# Check if there are any lexer/semantic action files
find chapi-ast-python -name "*.java" -o -name "*.py" -o -name "*.g4" | grep -i "listener\|visitor\|action\|semantic" | head -20

Repository: phodal/chapi

Length of output: 38

🏁 Script executed:

# Search for CheckVersion implementation and how it's used
find chapi-ast-python -name "*.java" | head -20

Repository: phodal/chapi

Length of output: 258

🏁 Script executed:

# Look for PythonParser listener or base classes
find chapi-ast-python -name "*Listener*" -o -name "*Visitor*" -o -name "Python*" | grep -E "\.(java|py)$" | head -20

Repository: phodal/chapi

Length of output: 258

🏁 Script executed:

# Search for other Python 3.14+ specific rules to see if they have version guards
rg -n "3\.14|PEP 758|type_params" chapi-ast-python/src/main/antlr/PythonParser.g4 | head -30

Repository: phodal/chapi

Length of output: 574

🏁 Script executed:

# Check if there are any other rules with semantic actions but no CheckVersion
rg -n "EXCEPT|except_types" chapi-ast-python/src/main/antlr/PythonParser.g4 -A 3 -B 3

Repository: phodal/chapi

Length of output: 676

🏁 Script executed:

# Examine CheckVersion implementation in PythonParserBase
cat chapi-ast-python/src/main/java/chapi/ast/antlr/PythonParserBase.java

Repository: phodal/chapi

Length of output: 741

🏁 Script executed:

# Check if type_params rule has any version guards
rg -n "type_params" chapi-ast-python/src/main/antlr/PythonParser.g4 -B 2 -A 5 | head -40

Repository: phodal/chapi

Length of output: 730

🏁 Script executed:

# Check if there are other unguarded Python 3.12+ or 3.14+ features
rg -n "Python 3\.(12|13|14)\+|PEP (695|758)" chapi-ast-python/src/main/antlr/PythonParser.g4 | head -20

Repository: phodal/chapi

Length of output: 591

🏁 Script executed:

# Look for how type_params, type_alias_stmt are used and if they're version-guarded at call sites
rg -n "type_params|type_alias_stmt" chapi-ast-python/src/main/antlr/PythonParser.g4 -B 1 | head -40

Repository: phodal/chapi

Length of output: 698

🏁 Script executed:

# Check if there's any validation or version checking outside the grammar
rg -n "supportsPython|checkVersion|validateVersion" chapi-ast-python/src/main/java -i

Repository: phodal/chapi

Length of output: 295

🏁 Script executed:

# Check PythonVersion.java to see the version enum values
cat chapi-ast-python/src/main/java/chapi/ast/antlr/PythonVersion.java

Repository: phodal/chapi

Length of output: 1374

🏁 Script executed:

# Look at how the grammar file is structured and check for other rules without CheckVersion that should be version-specific
rg -n "EXCEPT except_types|except_types" chapi-ast-python/src/main/antlr/PythonParser.g4 -B 5 -A 5

Repository: phodal/chapi

Length of output: 729

Add version guard to except_types alternative.
The except_types rule (lines 250–252) lacks a CheckVersion predicate, allowing Python 3.14+ bracketless exception syntax to parse in Python 2 and 3 modes. Add {CheckVersion(314)}? before the second except_clause alternative to prevent unintended matches in earlier versions, consistent with the version-gating pattern used in the first alternative (line 245).

🤖 Prompt for AI Agents 
In `@chapi-ast-python/src/main/antlr/PythonParser.g4` around lines 241 - 252, The
grammar allows the Python 3.14+ bracketless exception form to match in older
modes because the second alternative of except_clause lacks a version predicate;
add the version guard by prepending the predicate {CheckVersion(314)}? to the
alternative that references except_types so that except_types is only used when
CheckVersion(314) is true, keeping the existing CheckVersion/SetVersion pattern
used in the first except_clause alternative (refer to except_clause,
except_types, CheckVersion and SetVersion).


classdef
: CLASS name (OPEN_PAREN arglist? CLOSE_PAREN)? COLON suite
: CLASS name type_params? (OPEN_PAREN arglist? CLOSE_PAREN)? COLON suite
;

funcdef
: ASYNC? DEF name OPEN_PAREN typedargslist? CLOSE_PAREN (ARROW test)? COLON suite
: ASYNC? DEF name type_params? OPEN_PAREN typedargslist? CLOSE_PAREN (ARROW test)? COLON suite
;

// Python 3.12+ type parameter syntax (PEP 695)
type_params
: OPEN_BRACKET type_param_list CLOSE_BRACKET
;

type_param_list
: type_param (COMMA type_param)* COMMA?
;

type_param
: name type_param_bound? // TypeVar: T, T: int
| STAR name // TypeVarTuple: *Ts
| POWER name // ParamSpec: **P
;

type_param_bound
: COLON test // T: SomeType or T: (Type1, Type2)
;

// Python 3.12+ type alias statement (PEP 695)
type_alias_stmt
: TYPE name type_params? ASSIGN test
;

// python 3 paramters
Expand Down Expand Up @@ -159,6 +336,7 @@ small_stmt
| {CheckVersion(2)}? EXEC expr (IN test (COMMA test)?)? {SetVersion(2);} #exec_stmt // Python 2
| ASSERT test (COMMA test)? #assert_stmt
| {CheckVersion(3)}? NONLOCAL name (COMMA name)* {SetVersion(3);} #nonlocal_stmt // Python 3
| type_alias_stmt #type_stmt // Python 3.12+
;

import_dot_ellipsis : (DOT | ELLIPSIS) ;
Expand Down Expand Up @@ -313,6 +491,11 @@ name
: NAME
| TRUE
| FALSE
// Python 3.10+ soft keywords (can be used as identifiers)
| MATCH
| CASE
// Python 3.12+ soft keywords
| TYPE
;

number
Expand Down
34 changes: 33 additions & 1 deletion chapi-ast-python/src/main/java/chapi/ast/antlr/PythonVersion.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,20 @@
package chapi.ast.antlr;

/**
* Python version enumeration for syntax compatibility.
*
* Version history:
* - Python 3.10: Pattern matching (match/case)
* - Python 3.12: Type parameter syntax (PEP 695), type statement
* - Python 3.14: Template strings (t-strings), bracketless except (PEP 758)
*/
public enum PythonVersion {
Autodetect(0),
Python2(2),
Python3(3);
Python3(3),
Python310(310), // Pattern matching
Copy link

@augmentcode augmentcode bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds Python310/Python312/Python314, but the parser’s SetVersion(...) currently only maps 2 and 3, and PythonAnalyser constructs the parser in Autodetect without setting a newer version. As a result, these newer enum values (and the feature-check helpers) may not take effect during normal parsing unless something sets parser.Version externally.

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎

Python312(312), // Type parameter syntax
Python314(314); // Template strings, bracketless except
Comment on lines 3 to +17
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version numbering scheme is inconsistent across enum values. Python2 uses value 2, Python3 uses value 3, but Python 3.10+ uses values like 310, 312, 314.

This inconsistency creates confusion in the support methods:

  • Line 33: value >= 310 || this == Python3 - this special case is needed because Python3(3) < 310
  • This means Python3.getValue() returns 3, which is less than 310, making version comparisons counterintuitive

Consider using a consistent scheme:

  • Option 1: Use full version numbers for all (e.g., Python2=200, Python3=300, Python310=310)
  • Option 2: Use a separate major/minor version structure
  • Option 3: Document this inconsistency clearly and why Python3 is treated specially

The current approach where Python3(3) is considered to support Python 3.10 features (line 33) suggests Python3 means "latest 3.x", which should be documented.

Copilot uses AI. Check for mistakes.

private final int value;

Expand All @@ -14,4 +25,25 @@ public enum PythonVersion {
public int getValue() {
return value;
}

/**
* Check if this version supports Python 3.10+ features (pattern matching)
*/
public boolean supportsPatternMatching() {
return this == Autodetect || value >= 310 || this == Python3;
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic for supportsPatternMatching() is inconsistent with supportsTypeParameters().

The method includes || this == Python3 which means generic Python3 (value=3) would support pattern matching, but pattern matching was only introduced in Python 3.10. This is inconsistent because:

  1. supportsTypeParameters() correctly only allows version >= 312 or Autodetect
  2. Pattern matching requires Python 3.10+, not just Python 3

This should be: return this == Autodetect || value >= 310; (removing the || this == Python3 part) to match the pattern used in the other support methods.

Suggested change
return this == Autodetect || value >= 310 || this == Python3;
return this == Autodetect || value >= 310;

Copilot uses AI. Check for mistakes.
}

/**
* Check if this version supports Python 3.12+ features (type parameters)
*/
public boolean supportsTypeParameters() {
return this == Autodetect || value >= 312;
}

/**
* Check if this version supports Python 3.14+ features (t-strings, bracketless except)
*/
public boolean supportsPython314Features() {
return this == Autodetect || value >= 314;
Comment on lines +40 to +47
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The supportsTypeParameters() method doesn't include || this == Python3 unlike supportsPatternMatching(). This means:

  • Python3 version will return false for type parameters support (since value=3 < 312)
  • Python3 version will return true for pattern matching support (due to special case on line 33)

This inconsistency is confusing. If Python3 is meant to represent "generic Python 3.x" that should support all features (as suggested by line 33), then this method should also include || this == Python3. If Python3 is meant to represent specifically Python 3.0-3.9, then line 33's special case for pattern matching is incorrect (as noted in a separate comment).

The behavior should be consistent across all feature support methods.

Suggested change
return this == Autodetect || value >= 312;
}
/**
* Check if this version supports Python 3.14+ features (t-strings, bracketless except)
*/
public boolean supportsPython314Features() {
return this == Autodetect || value >= 314;
return this == Autodetect || value >= 312 || this == Python3;
}
/**
* Check if this version supports Python 3.14+ features (t-strings, bracketless except)
*/
public boolean supportsPython314Features() {
return this == Autodetect || value >= 314 || this == Python3;

Copilot uses AI. Check for mistakes.
}
Comment on lines +29 to +48
Copy link

@coderabbitai coderabbitai bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Search for PythonVersion.Python3 usage and feature gate method calls
rg -n "PythonVersion\.Python3\b|supportsPatternMatching|supportsTypeParameters|supportsPython314Features" -A 2 -B 2

Repository: phodal/chapi

Length of output: 2213

🏁 Script executed:

#!/bin/bash
# Find and examine the PythonVersion enum definition
fd -e java | xargs rg -l "enum PythonVersion|class PythonVersion" | head -5

Repository: phodal/chapi

Length of output: 121

🏁 Script executed:

#!/bin/bash
# Read the PythonVersion.java file
cat -n chapi-ast-python/src/main/java/chapi/ast/antlr/PythonVersion.java

Repository: phodal/chapi

Length of output: 1717

Align Python3 semantics across feature gates to avoid inconsistent feature availability.

supportsPatternMatching() treats Python3 (value=3) as supporting 3.10+ features with an explicit check, but supportsTypeParameters() and supportsPython314Features() exclude it entirely. Since Python3 represents a generic/unknown Python 3.x version (as seen in PythonParserBase.java line 22), this creates an inconsistency: either all three methods should include Python3, or none should. Decide on the intended semantics—does generic Python 3 assume modern feature support, or should version checks be strict?—and align all three methods accordingly.

🤖 Prompt for AI Agents 
In `@chapi-ast-python/src/main/java/chapi/ast/antlr/PythonVersion.java` around
lines 29 - 48, The three feature-gate methods (supportsPatternMatching,
supportsTypeParameters, supportsPython314Features) are inconsistent about
treating the enum constant Python3 as “generic Python 3.x”; align their
semantics by either always treating Python3 as supporting modern 3.x features or
never doing so—pick one. Concretely, update the implementations in
PythonVersion: if you choose to treat Python3 as generic modern 3.x, add the
same check (this == Python3) to supportsTypeParameters() and
supportsPython314Features(); if you choose strict numeric checks, remove the
this == Python3 clause from supportsPatternMatching(). Keep the Autodetect
handling (this == Autodetect) unchanged and use the existing value field for
numeric comparisons.

}
Loading
Loading