SelectField backward compatibility and minor QoL improvements

[azmeuk]

I addressed most of the points raised in #922

• Add a post_process step which happens right after process
• Add optional form and field args to SelectField and DataList choices callbacks. Callbacks are resolved during the post_process step, so it can access the other fields .data
• Choice inherit from NamedTuple instead of Dataclass. An additional private _Choice class is needed to pass default values to the tuple members since NamedTuple forbids overriding new
• Restore has_groups and iter_groups
• Backward compatibility for iter_choices returning tuples, for render_options taking value, label, selected instead of choice.
• Support choices as dict as suggested in Support dict for SelectForm.choices #886

The compatibility layer makes the code bigger and harder to read, but hopefully things goes better when we delete the deprecations in 3.4/4.0.

I have tested those modification in downstream projects (wtf-peewee, wtforms-alchemy, starlette-wtf, flask-appbuilder) and it solves all the issues related to SelectField.

/cc @Daverball

[ElLorans]

Couldn't double check everything, but if we were extending a Field now we have to change from iter_choices to _iter_choices_normalized?

[Daverball]

Thanks for working on this, this looks pretty good and alleviates my concerns. I will try to take another deep dive later, to see if there is anything else you or I missed.

I did spot one small issue with the new callback, but that's an easy fix, it just adds a little bit more code.

[azmeuk]

Couldn't double check everything, but if we were extending a Field now we have to change from iter_choices to _iter_choices_normalized?

No, you keep overriding iter_choices. _iter_choices_normalized is just a private helper that normalizes the iter_choices output to keep backward compatibility (and raise deprecation warnings).

[Daverball]

Upon closer review I found a few additional subtle issues, but other than that I think we're good.

[Daverball]

I stand corrected, found a couple more things in the widget code.

[azmeuk]

I think I adressed most of your feedback. I went back to dataclasses with iter for the tuple compatibility, because this is hard to handle default values and inheritance with them. I tested on several dowstream projects and the unit test suites pass.

[Daverball]

I think I adressed most of your feedback. I went back to dataclasses with iter for the tuple compatibility, because this is hard to handle default values and inheritance with them. I tested on several dowstream projects and the unit test suites pass.

Thanks so much, I'll take another closer look later, but there's one thing I can already say now.

Using dataclasses for SelectChoice/DataListChoice seems fine to me, however for Choice I still think a NamedTuple is the way to go. Choice doesn't need any defaults, you can just make all four components mandatory, since it's for internal use only, so you also don't need to use any subclassing tricks, you can just directly use a plain NamedTuple.

class Choice(NamedTuple):
    """
    A rendered option yielded by
    :meth:`SelectFieldBase.iter_choices` and
    :meth:`SelectFieldBase.iter_groups`.

    ``selected`` is computed against the field's current data. To
    declare options on a :class:`SelectField`, use
    :class:`SelectChoice` instead.

    :param value:
        The value that will be sent in the request.
    :param label:
        The label of the option.
    :param selected:
        Whether the option is currently selected. Set by ``iter_choices``;
        you rarely set this yourself.
    :param render_kw:
        A dict containing HTML attributes that will be rendered
        with the option.
    """

    value: str
    label: str
    selected: bool
    render_kw: dict

The main reason to use a NamedTuple for Choice is so that type checkers better understand them as structured types for the purposes of unpacking, you can't unpack a dataclass that has mixed field types and have type checkers know the type of each component. This preserves exact typing information and ergonomics for people that decide to keep their implementations the same, instead of switching to named attribute access, which can make the code longer.

I.e. compare

for value, label, selected, render_kw in field.iter_choices():
    do_something_with(value, label, selected)

to

for choice in field.iter_choices():
    do_something_with(choice.value, choice.label, choice.selected)

Neither code is obviously better than the other. Forcing people to switch their code to the latter, just to preserve good type checking experience seems like unnecessary downstream churn for no real benefit to anyone, when a NamedTuple would've worked just as well. It also avoids any potential risks for breakage in downstream code that expects a tuple, since they will still get one, since NamedTuple classes are subclasses of tuple.