fix(sqla): coerce ARRAY element type in conv_ARRAY (closes #1724)

[jbbqqf]

Summary

AdminModelConverter.conv_ARRAY returned a Select2TagsField with no
coerce callable, so every submitted value was cast via the default
text_type and the resulting Python list was always list[str]. For a
Postgres ARRAY(Integer) column that round-trip fails on save with

column "x" is of type integer[] but expression is of type text[]

(originally reported in #1724 in 2018 and still reproducible on
master; the long-standing workaround in that thread is to subclass
Select2TagsField and override process_formdata to call int() on
each entry).

Fixes #1724.

Context

• The buggy converter is at
  flask_admin/contrib/sqla/form.py:620-626 (pre-change):

  @converts(
      "sqlalchemy.dialects.postgresql.base.ARRAY", "sqlalchemy.sql.sqltypes.ARRAY"
  )
  def conv_ARRAY(self, field_args, **extra):
      return form.Select2TagsField(save_as_list=True, **field_args)

• Select2TagsField already supports a coerce parameter
  (flask_admin/form/fields.py:228) that is applied to each tag in
  process_formdata (flask_admin/form/fields.py:253-256).

• SQLAlchemy's ARRAY exposes column.type.item_type, and standard
  inner types implement .python_type. Detect that and use it as
  coerce.

Changes

• flask_admin/contrib/sqla/form.py: in conv_ARRAY, look up the
  column's item_type.python_type. When that returns a non-str type
  (int, float, bool, ...) pass it through to Select2TagsField as
  coerce. Fall back silently when python_type is missing
  (AttributeError / NotImplementedError) or the column object can't
  be introspected -- the previous behavior for ARRAY(String) is
  preserved unchanged.
• flask_admin/tests/sqla/test_form.py: four new tests in
  TestArrayConverter:
  • test_conv_ARRAY_integer_coerces_each_item_to_int
  • test_conv_ARRAY_float_coerces_each_item_to_float
  • test_conv_ARRAY_string_keeps_string_default
  • test_conv_ARRAY_missing_item_type_falls_back_to_text
• doc/changelog.rst: [unreleased] bugfix entry.

Reproduce BEFORE/AFTER yourself (copy-paste)

# --- one-time setup ---
git clone https://github.com/pallets-eco/flask-admin.git /tmp/fa-1724 && cd /tmp/fa-1724
uv venv .venv --python 3.12
source .venv/bin/activate
uv pip install -e ".[sqlalchemy]" pytest

# --- BEFORE: origin/master, integer ARRAY tags stay as strings ---
git checkout origin/master
git fetch https://github.com/jbbqqf/flask-admin.git fix/1724-array-integer-coerce
git checkout FETCH_HEAD -- flask_admin/tests/sqla/test_form.py
pytest flask_admin/tests/sqla/test_form.py::TestArrayConverter -v
# Expected: 2 failed, 2 passed
#   - test_conv_ARRAY_integer_coerces_each_item_to_int  -> ['1', '2', '3'] != [1, 2, 3]
#   - test_conv_ARRAY_float_coerces_each_item_to_float -> ['1.5', '2.0'] != [1.5, 2.0]
git checkout origin/master -- flask_admin/tests/sqla/test_form.py

# --- AFTER: this branch, element types are coerced from item_type.python_type ---
git checkout FETCH_HEAD
pytest flask_admin/tests/sqla/test_form.py::TestArrayConverter -v
# Expected: 4 passed

What I ran locally

• New tests, focused:

  pytest flask_admin/tests/sqla/test_form.py::TestArrayConverter -v
  # 4 passed
  

• Whole test_form.py file (covers all converters, including the
  parametrized "can_override_widget" matrix where conv_ARRAY is
  already exercised):

  pytest flask_admin/tests/sqla/test_form.py -v
  # 27 passed
  

• Broader regression sweep on the SQLA + generic model code paths:

  pytest flask_admin/tests/sqla/test_basic.py flask_admin/tests/sqla/test_form.py \
         flask_admin/tests/test_model.py -q --tb=no
  # 229 passed, 56 skipped, 11 xfailed
  

• Lint:

  ruff check  flask_admin/contrib/sqla/form.py flask_admin/tests/sqla/test_form.py
  ruff format --check flask_admin/contrib/sqla/form.py flask_admin/tests/sqla/test_form.py
  # All checks passed! / 2 files already formatted
  

Edge cases

#	Scenario	Inner type	Submitted	Expected field.data	Verified by
1	ARRAY(Integer)	Integer -> int	"1,2,3"	[1, 2, 3]	test_conv_ARRAY_integer_coerces_each_item_to_int
2	ARRAY(Float)	Float -> float	"1.5, 2.0"	[1.5, 2.0]	test_conv_ARRAY_float_coerces_each_item_to_float
3	ARRAY(String)	String -> str	"alpha,beta,gamma"	["alpha", "beta", "gamma"] (unchanged)	test_conv_ARRAY_string_keeps_string_default
4	column.type without item_type (mocked/legacy)	n/a	"x,y"	["x", "y"] (fallback)	test_conv_ARRAY_missing_item_type_falls_back_to_text
5	Inner type without python_type (e.g. JSON)	raises NotImplementedError	unchanged	no coerce injected, no crash	covered by the try/except branch in the new code path

Risk / blast radius

• Only conv_ARRAY is modified, plus its tests.
• For ARRAY(String) -- the only inner type that has historically
  worked end-to-end -- the new code path explicitly skips injecting a
  coerce (it's already the default), so existing apps using string
  arrays see no behavior change.
• Users who had subclassed Select2TagsField to add their own
  per-element int() cast (per the workaround in the issue thread)
  keep working as before, because:
  • the converter only sets coerce when the inner type's
    python_type is not str;
  • it does it via setdefault, so an explicit user-provided
    field_args["coerce"] wins.
• No template/JS/static touched.

Release note

SQLAlchemy backend: scaffolded forms for Postgres `ARRAY(Integer)` /
`ARRAY(Float)` columns now save correctly. The `Select2TagsField` field
generated by `conv_ARRAY` infers the inner type's `python_type` and
uses it as the per-tag coerce callable, so saved values match the
column's element type instead of being sent back as `text[]`.

---

PR drafted with assistance from Claude Code (Anthropic). The change was
reviewed manually against flask-admin's source -- file:line references in
the body were checked against the cloned tree, and the copy-paste
BEFORE/AFTER reproducer block is the one I used during development;
reviewers can paste it verbatim.

[ElLorans]

Thank you for your PR!
I simplified the comment and removed the cast by introducing a more specific field args type.
LGTM. I’ll merge this in the next few days to allow time for additional feedback.

[samialfattani]

Thanks for the greate job.
There was another try (PR #2784) to resolve similar issue, it is better to have a look on the and make sure that all datatypes are covered includeing str and Enum types. anyway i suggest to include the following test-code in this PR. This code tests the both explicit-and-auto coerce, for all datatypes in all covered databases (9 types x 2 coerce types (explicit+auto) x 4 DB Providers = 72 test cases).

import enum
import inspect
import typing as t
from unittest.mock import MagicMock

import pytest
import sqlalchemy as sa
import sqlalchemy_utils as sau
import wtforms
from wtforms.fields.simple import StringField
from wtforms.validators import Length
from wtforms.validators import NumberRange

from flask_admin.contrib.sqla.form import AdminModelConverter
from flask_admin.tests.conftest import skip_or_return_session_or_db
from flask_admin.tests.sqla.test_basic import CustomModelView



class EnumChoices(enum.Enum):
    First = 101
    Second = 150


class StrEnumChoices(enum.Enum):
    First = "101"
    Second = "150"


def create_models(sqla_db_ext):
    class Model1(sqla_db_ext.Base):  # type: ignore[name-defined, misc]
        __tablename__ = "model1"

        def __init__(self, test1, int_field):
            self.test1 = test1
            self.int_field = int_field

        id = sa.Column(sa.Integer, primary_key=True, autoincrement=True)
        test1 = sa.Column(sa.String(20))
        int_field = sa.Column(sa.Integer)
        float_field = sa.Column(sa.Float)
        choice_field = sa.Column(sa.String, nullable=True)
        enum_field = sa.Column(sa.Enum("101", "150"), nullable=True)  # type: ignore[var-annotated]
        enum_type_field = sa.Column(sa.Enum(EnumChoices), nullable=True)  # type: ignore[var-annotated]
        sau_choicetype = sa.Column(
            sau.ChoiceType(
                [("101", "First"), ("150", "Second")]
            )  # default impl=sa.String()
        )
        sau_choicetype_impl_int = sa.Column(
            sau.ChoiceType([(101, "First"), (150, "Second")], impl=sa.Integer())
        )
        sau_choicetype_with_enum = sa.Column(
            sau.ChoiceType(EnumChoices, impl=sa.Integer())
        )
        sau_choicetype_with_strenum = sa.Column(
            sau.ChoiceType(StrEnumChoices, impl=sa.String())
        )

        def __str__(self):
            return self.test1

    sqla_db_ext.create_all()

    return Model1


def prepare_kwargs(
    expected_coerce: type[t.Any],
    use_coerce_explicitly: bool,
    field_name: str,
) -> dict[str, t.Any]:
    validators: list[t.Any] = []
    kwargs: dict[str, t.Any] = dict()

    if expected_coerce in [int, float]:
        f_choices = [(expected_coerce(101), "First"), (expected_coerce(150), "Second")]
        validators = [NumberRange(min=100, max=199)]
        kwargs["form_choices"] = {field_name: f_choices}

    elif expected_coerce in [
        str,
    ]:
        f_choices = [(expected_coerce(101), "First"), (expected_coerce(150), "Second")]
        validators = [Length(min=1, max=3)]
        kwargs["form_choices"] = {field_name: f_choices}

    elif expected_coerce in [EnumChoices, StrEnumChoices]:
        pass

    kwargs["form_columns"] = [field_name]

    if use_coerce_explicitly:
        kwargs["form_args"] = dict()
        kwargs["form_args"][field_name] = {"validators": validators}
        kwargs["form_args"][field_name]["coerce"] = expected_coerce

    return kwargs


@pytest.mark.parametrize("use_coerce_explicitly", [False, True])
@pytest.mark.parametrize(
    "field_name, expected_coerce, coerced_value, model_value",
    [
        ("int_field", int, None, 101),
        ("float_field", float, None, 101.0),
        ("choice_field", str, None, "101"),
        ("enum_field", str, None, "101"),
        ("enum_type_field", EnumChoices, "First", EnumChoices.First),
        ("sau_choicetype", str, None, sau.Choice("101", "First")),
        ("sau_choicetype_impl_int", int, None, sau.Choice(101, "First")),
        ("sau_choicetype_with_enum", EnumChoices, "101", EnumChoices.First),
        ("sau_choicetype_with_strenum", StrEnumChoices, "101", StrEnumChoices.First),
    ],
)
def test_coerce(
    app,
    admin,
    sqla_db_ext,
    session_or_db,
    field_name,
    expected_coerce,
    use_coerce_explicitly,
    coerced_value,
    model_value,
):
    with app.app_context():
        Model1 = create_models(sqla_db_ext)
        sqla_db_ext.db.session.add_all(
            [
                Model1("101", int_field=101),
                Model1("102", int_field=102),
            ]
        )
        sqla_db_ext.db.session.commit()

        kwargs = prepare_kwargs(expected_coerce, use_coerce_explicitly, field_name)

        param = skip_or_return_session_or_db(sqla_db_ext, session_or_db)

        view1 = CustomModelView(Model1, param, name="My Model1", **kwargs)
        admin.add_view(view1)

    value = expected_coerce(101) if coerced_value is None else coerced_value

    client = app.test_client()

    rv = client.get("/admin/model1/new/")
    data = rv.data.decode("utf-8")
    assert f'value="{value}"' in data
    assert ">First</option>" in data

    rv = client.post(
        "/admin/model1/new/",
        data={field_name: f"{value}"},
        follow_redirects=True,
    )
    data = rv.data.decode("utf-8")
    assert "Record was successfully created" in data

    iserted = sqla_db_ext.db.session.query(Model1).order_by(Model1.id.desc()).first()
    assert iserted is not None
    assert getattr(iserted, field_name) == model_value

[ElLorans]

Thanks! Does this PR pass this test case?
Why do you need to test explicitly the type with expected_coerce? Can't we let the ORM raise an error and check for that?

[samialfattani]

Thanks! Does this PR pass this test case?

Yes,

flask_admin/tests/sqla/test_form.py .................................... [ 50%]
ssssssssssssssssss..................                                     [100%]

======================= 54 passed, 18 skipped in 16.37s ========================
Finished running tests!

Why do you need to test explicitly the type with expected_coerce? Can't we let the ORM raise an error and check for that?

this for those people who spcified coerce casting explicitly as a workarround before merging this PR. If this PR has the right logic then both explicit and auto coerce should lead to the same results.

Note: the above code is modified

[ElLorans]

But the test we are talking about is not testing coerce casting explicitly, so I would just check the orm accepted the types

[samialfattani]

But the test we are talking about is not testing coerce casting explicitly, so I would just check the orm accepted the types

correct me if i'm wrong, in the next upcomming version, won't you recommend the users to eliminate the explicit coerce for all selectable fields and leave to Flask-Admin to decide the right one ? i'm expecting that user's code would be like:

class UserAdmin(ModelView):

    form_choices = [(2006, "2006"), (2007, "2007"), (2008, "2008")]
    form_args = {
      "join_in": {
         "validators" : [Length(min=2005, max=2030)] ,
         "corece" : int
       }
   }

and after this PR it should be like:

class UserAdmin(ModelView):

    form_choices = [(2006, "2006"), (2007, "2007"), (2008, "2008")]
    form_args = {
      "join_in": {
         "validators" : [Length(min=2005, max=2030)] ,
       }
   }

my test code make sure that user explicitly define the coerce=int or it is auto determined by Flask-Admin, then this won't change data in the model and even in the database.