Move setup.py metadata to pyproject.toml

[gforcada]

Closes #315

The easy part is done: adapt the zope.meta script to be usable within plone.meta.

We still need to fix what happens after we run again config-package on a repository 😅

Now we have a legal problem 🙃:

On setup.py we used both:

license="GPL",

but also classifiers:

"License :: OSI Approved :: GNU General Public License v2 (GPLv2)",

On pyproject.toml though, we are pressed to use license expressions which mostly boils down to either put a single string from the SPDX license list or a combination of a few of them (though not so much our case).

On license= in setup.py we have a mix of:

BSD
gpl
GPL
GPL v 2
GPL version 2
GPL version 2 or later
GPLv2
LGPL
ZPL 2.1

The last (ZPL 2.1) one seems easy: license = "ZPL-2.1".

What about the GPL or BSD ones? the few ones that mention v2 is GPL-2.0-only or GPL-2.0-or-later ?

Same for BSD, there are so many variations... 😵‍💫 😅

[gforcada]

@mauritsvanrees who could the our legal hero for this PR? 🤔 😅 see the first message for the details.

[mauritsvanrees]

Note: I did not look at the code yet.

@gforcada We could make a mapping from our most used license classifiers to the new license expressions. I see at least these:

License :: OSI Approved :: GNU General Public License v2 (GPLv2)
License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)
License :: OSI Approved :: Zope Public License
License :: OSI Approved :: BSD License

• If there is more than one license classifier: stop with an error.
• If there is a license classifier, but it is not in the list: probably also stop with an error. If the user wants to keep the license, they can always temporarily remove the classifier, run the script, and put their own chosen license expression in pyproject.toml.
• If there is a license classifier and a license key, and they don't match, stop with an error. If the classifier says GPL v2 (plus maybe 'or later') then I would say it is fine if the license key has "GPL" in it, in whatever spelling.

Maybe @polyester has thoughts here?

You could also ask in the sprint standup tomorrow if anyone there has knowledge about this.

[mauritsvanrees]

I am trying it on collective.recipe.backup. See collective/collective.recipe.backup#99

Eyeballing the changes, what stands out is this:

+[project.urls]
+Source = "https://github.com/collective/collective.recipe.backup"
+Issues = "https://github.com/plone/Products.CMFPlone/issues"
+Changelog = "https://github.com/plone/collective.recipe.backup/blob/315-move-setup-py-to-pyproject-toml/CHANGES.rst"
+# END-MARKER-MANUAL-CONFIG

• Pointing to the CMFPlone issue tracker makes sense for a lot of our packages, but of course not for collective packages. It would be nice to be able to override this with an option. Something like --issues own to use the repo url + /issues. Maybe also --issues https://somewhere.else to pass a literal url.
• The Changelog url should use the main/master branch (or whatever the branch was that the repo started on).

At the end of the file I see this change:

+
+[tool.setuptools.dynamic]
+readme = {file = ["README.rst", "CHANGES.rst"]}
+

• It would be nice to avoid the extra line at the end.
• The setup.py still calculates a long_description as well. This should be either in setup.py or in pyproject.toml, not both. I thought that in tool.setuptools.dynamic it did not quite work to specify more than one file. As far as I know, this is why in zope.meta they decided to only use the readme, and no longer include the CHANGES.rst. Instead, the changelog is linked in project.urls.

Another one:

• There is a separate commit 'Add news entry'. It would be good to have this in the main commit.
• The news snippet says "Update configuration files." This should be something like "Move package configuration from setup.py to pyproject.toml.

[mauritsvanrees]

Rather more importantly, the extras_require is lost. The GitHub Actions give errors like this:

collective.recipe.backup 6.0.0.dev0 does not provide the extra 'test'

[mauritsvanrees]

• Pointing to the CMFPlone issue tracker makes sense for a lot of our packages, but of course not for collective packages. It would be nice to be able to override this with an option. Something like --issues own to use the repo url + /issues. Maybe also --issues https://somewhere.else to pass a literal url.

Easier: if the repo url starts with github.com/plone use the CMFPlone tracker, otherwise use the one from the repo itself.
That should do the right thing 99% of the time, and we don't need an option. The user can always edit the url manually later.

[gforcada]

@mauritsvanrees given your feedback:

• there is an --issues command line option (also documented 📚 ) that behaves like you said (by default Products.CMFPlone, if a URL is provided that is used verbatim, if own is provided the repository URL + /issues)
• the Changelog URL gets the current branch the repository is in, and if it is not main nor master it prints a warning
• the extra line at the end of pyproject.toml I have to investigate it, but anyway, that part should be moved to the section within the comment guards, otherwise at next config-package update it will be removed
• the setup.py long description being added verbatim is a problem of using f-strings I assume, as some code runs setup.py to get the final values of variables, and thus being able to extract them. I will check if we can actually drop it directly
• as for multiple files in long_description: I've seen examples that is indeed possible, my personal taste is that if it is only README + CHANGES we could drop CHANGES as it is added on project.urls, but there are some distributions (plone.testing) that puts plenty of files together. But although it would be nice to create +100 PRs within a few minutes, I would expect that we want to eyeball and fine tune some projects, so I expect that this long_description will be a bit of it and there is not too much point of trying to make it perfect, as each distribution is slightly different

We could go on and on here, as metadata is very intricate... like do we want to standardize the maintainers to:

    maintainers_table = {
        "name": "Plone Foundation and contributors",
        "email": "plone-developers@lists.sourceforge.net",
    }

That's what will be doing this script. Is that email still valid? sourceforge.net ❗

Should we do something similar to the authors field? At least for the github.com/plone packages.

[mauritsvanrees]

In collective.recipe.backup, check-python-versions does not like the result. Seems strange:

$ pre-commit run -a check-python-versions
check-python-versions....................................................Failed
- hook id: check-python-versions
- exit code: 1

Traceback (most recent call last):
  File "/Users/maurits/community/collective.recipe.backup/setup.py", line 5, in <module>
    from setuptools import setup
ModuleNotFoundError: No module named 'setuptools'
setup.py says:          (empty)
pyproject.toml says:    3.10, 3.11, 3.12, 3.13, 3.14
- python_requires says: 3.10, 3.11, 3.12, 3.13, 3.14

mismatch!

I see the same in plone.restapi (if I first fix an error by moving TEST_REQUIRES into the setup() call).

It seems to me an error in check-python-versions. But maybe if we basically empty pyproject.toml we can change the pre-commit config: instead of args: ['--only', 'setup.py,pyproject.toml'] it could be args: ['--only', 'pyproject.toml']. Maybe we could change config_package.py to set this correctly if the START-MARKER-MANUAL-CONFIG is in pyproject.toml, or the project table is there.

[gforcada]

The check-python-versions mismatch could be fixed by adjusting the config files to be checked, adding tox.ini and pyproject.toml should be enough