-
Notifications
You must be signed in to change notification settings - Fork 450
Description
Apache Iceberg version
None
Please describe the bug 🐞
Issue Description:
Describe the bug
When attempting to initialize a RestCatalog pointing to an Apache Polaris REST server, the initialization fails immediately. Apache Polaris returns valid REST endpoints that include the PUT HTTP method (e.g., for updating namespaces/tables). However, PyIceberg's Endpoint parser and Pydantic validation strictly expect only GET, HEAD, POST, or DELETE.
This causes two cascaded errors depending on the PyIceberg version:
- A
TypeError: 'str' object is not callablewhen trying to instantiate theHttpMethod. - A Pydantic
ValidationErrorif the type casting is bypassed, explicitly stating thatPUTis not an allowed enum value.
To Reproduce
Attempt to connect to an Apache Polaris catalog using RestCatalog:
from pyiceberg.catalog.rest import RestCatalog
catalog = RestCatalog(
"iceberg",
**{
"uri": "http://<polaris-host>:8181/api/catalog",
"credential": "<credential>",
"warehouse": "iceberg",
}
)Expected behavior
The RestCatalog should initialize successfully, parse the valid PUT endpoints returned by the Polaris configuration, or at least gracefully ignore unsupported HTTP methods without crashing the entire initialization process.
Error Logs
Error 1: Type Error during Endpoint parsing
File /opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:116, in Endpoint.from_string(cls, endpoint)
114 if len(elements) != 2:
115 raise ValueError(f"Invalid endpoint: {endpoint}")
--> 116 return cls(http_method=HttpMethod(elements[0].upper()), path=elements[1])
TypeError: 'str' object is not callableError 2: Pydantic Validation Error (Underlying issue)
ValidationError: 1 validation error for ConfigResponse
endpoints.http_method
Input should be 'GET', 'HEAD', 'POST' or 'DELETE' [type=enum, input_value='PUT', input_type=str]Environment
- PyIceberg version:
0.11.0 - Apache Polaris version:
1.3.0-incubating - Python version:
3.11
Proposed Solution
- Update
HttpMethod(Enum/Literal) inpyiceberg/catalog/rest/__init__.pyto includePUT,PATCH, andOPTIONS. - Ensure that unknown/future HTTP methods returned by a REST Catalog's
/v1/configendpoint do not hard-crash the client.
Willingness to contribute
- I can contribute a fix for this bug independently
- I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- I cannot contribute a fix for this bug at this time