1.0.1: pytest fails in `tests/test_server.py::test_redirect_location_is_https_for_secure_server` unit

[kloczek]

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-1.0.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-1.0.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -p no:relaxed
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-httpbin-1.0.1
plugins: httpbin-1.0.1, forked-1.4.0, xdist-2.5.0
collected 20 items

tests/test_httpbin.py .............                                                                                                                                  [ 65%]
tests/test_server.py ......F                                                                                                                                         [100%]

================================================================================= FAILURES =================================================================================
____________________________________________________________ test_redirect_location_is_https_for_secure_server _____________________________________________________________

httpbin_secure = <pytest_httpbin.serve.SecureServer object at 0x7f352851acd0>

    def test_redirect_location_is_https_for_secure_server(httpbin_secure):
        assert httpbin_secure.url.startswith('https://')
        response = requests.get(
            httpbin_secure + "/redirect-to?url=/html",
            allow_redirects=False
        )
        assert response.status_code == 302
        assert response.headers.get('Location')
>       assert response.headers['Location'].startswith('https://')
E       AssertionError: assert False
E        +  where False = <built-in method startswith of str object at 0x7f3528501cf0>('https://')
E        +    where <built-in method startswith of str object at 0x7f3528501cf0> = '/html'.startswith

tests/test_server.py:105: AssertionError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
127.0.0.1 - - [31/Dec/2021 05:00:36] "GET /redirect-to?url=/html HTTP/1.1" 302 0
========================================================================= short test summary info ==========================================================================
FAILED tests/test_server.py::test_redirect_location_is_https_for_secure_server - AssertionError: assert False
======================================================================= 1 failed, 19 passed in 2.56s =======================================================================

[bnavigator]

FYI, (irrespective of the reporters earlier fail) this now generally fails due to pallets/flask#4496 and pallets/werkzeug#2352

[kloczek]

Thx. I've sibscribed to those tickets 😄

[kloczek]

No, one sec .. these are PRs.
OK will try to update my package with flask module to check that 😄
Will bask ASP with results.

[kloczek]

Just tested In build env with installed only below list of modules.

Package            Version
------------------ --------------
attrs              21.4.0
blinker            1.4
Brotli             1.0.9
build              0.7.0
charset-normalizer 2.0.12
click              8.1.2
codespell          2.1.0
cycler             0.11.0
decorator          5.1.1
distro             1.7.0
extras             1.0.0
fixtures           3.0.0
Flask              2.1.1
fonttools          4.32.0
gpg                1.17.1-unknown
httpbin            0.7.0
idna               3.3
importlib-metadata 4.11.3
iniconfig          1.1.1
itsdangerous       2.1.2
Jinja2             3.1.1
kiwisolver         1.3.2
libcomps           0.1.18
MarkupSafe         2.1.1
matplotlib         3.5.1
numpy              1.22.3
olefile            0.46
packaging          21.3
pbr                5.8.1
pep517             0.12.0
Pillow             9.1.0
pip                22.0.4
pluggy             1.0.0
py                 1.11.0
PyGObject          3.42.0
pyparsing          3.0.8
pytest             7.1.1
python-dateutil    2.8.2
raven              6.10.0
requests           2.27.1
rpm                4.17.0
setuptools         62.0.0
six                1.16.0
testtools          2.5.0
tomli              2.0.1
urllib3            1.26.9
Werkzeug           2.1.1
wheel              0.37.1
zipp               3.8.0

Cc: @pallets
Looks like pytest is failing because httpbin is failing in importing BaseResponse from werkzeug.wrappers (2.1.1) module. I'm not 100% sure does it mean that httpbin needs to be updated or it is lates werkzeug` module issue

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-1.0.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-1.0.2-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
Traceback (most recent call last):
  File "/usr/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 187, in console_main
    code = main()
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 145, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 324, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/usr/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/helpconfig.py", line 102, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1016, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1304, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1187, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/usr/lib64/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
    exec(co, module.__dict__)
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-1.0.2-2.fc35.x86_64/usr/lib/python3.8/site-packages/pytest_httpbin/plugin.py", line 3, in <module>
    from httpbin import app as httpbin_app
  File "/usr/lib/python3.8/site-packages/httpbin/__init__.py", line 3, in <module>
    from .core import *
  File "/usr/lib/python3.8/site-packages/httpbin/core.py", line 22, in <module>
    from werkzeug.wrappers import BaseResponse
ImportError: cannot import name 'BaseResponse' from 'werkzeug.wrappers' (/usr/lib/python3.8/site-packages/werkzeug/wrappers/__init__.py)

---

PS. I cannot report issues in werkzeug repo because @pallets blocked me more than year ago when I've been only started learning about package python modules 😞
@pallets could you please unblock me? Because you've blocked me I'm not receiving notifications about your new releases 😞
Since than I've learned a lot about packaging, testing and reporting issues of python modules (I've packaged since then almost 900 python modules .. you can check my activity https://github.com/kloczek)

[kloczek]

OK .. looks like postmanlabs/httpbin#674 may be solution of that issue.
Going to check that PR.

[kloczek]

After fixing httpbin issue with below patch

Based on https://github.com/postmanlabs/httpbin/pull/674

--- a/httpbin/core.py~  2022-04-13 14:14:23.000000000 +0000
+++ b/httpbin/core.py   2022-04-13 14:15:17.159804065 +0000
@@ -19,7 +19,7 @@
 from six.moves import range as xrange
 from werkzeug.datastructures import WWWAuthenticate, MultiDict
 from werkzeug.http import http_date
-from werkzeug.wrappers import BaseResponse
+from werkzeug.wrappers import Response
 from werkzeug.http import parse_authorization_header
 from raven.contrib.flask import Sentry

@@ -50,7 +50,6 @@
 # Prevent WSGI from correcting the casing of the Location header
 # and forcing it to be absolute. This moved from BaseResponse to
 # Response in werkzeug 2.0.0, so we set both to be safe.
-BaseResponse.autocorrect_location_header = False
 Response.autocorrect_location_header = False

 # Find the correct template folder when running from a different location

looks like pytest still is failing

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-1.0.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-1.0.2-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-httpbin-1.0.2
plugins: httpbin-1.0.2
collected 20 items

tests/test_httpbin.py .............                                                                                                                                  [ 65%]
tests/test_server.py ......F                                                                                                                                         [100%]

================================================================================= FAILURES =================================================================================
____________________________________________________________ test_redirect_location_is_https_for_secure_server _____________________________________________________________

httpbin_secure = <pytest_httpbin.serve.SecureServer object at 0x7fca25d1c160>

    def test_redirect_location_is_https_for_secure_server(httpbin_secure):
        assert httpbin_secure.url.startswith('https://')
        response = requests.get(
            httpbin_secure + "/redirect-to?url=/html",
            allow_redirects=False
        )
        assert response.status_code == 302
        assert response.headers.get('Location')
>       assert response.headers['Location'].startswith('https://')
E       AssertionError: assert False
E        +  where False = <built-in method startswith of str object at 0x7fca25d79b30>('https://')
E        +    where <built-in method startswith of str object at 0x7fca25d79b30> = '/html'.startswith

tests/test_server.py:105: AssertionError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
127.0.0.1 - - [13/Apr/2022 14:22:09] "GET /redirect-to?url=/html HTTP/1.1" 302 0
========================================================================= short test summary info ==========================================================================
FAILED tests/test_server.py::test_redirect_location_is_https_for_secure_server - AssertionError: assert False
======================================================================= 1 failed, 19 passed in 2.63s =======================================================================

[AdamWill]

Yeah, I've been seeing this in Fedora builds too. That's why the package is stuck at 1.0.0. I didn't want to update it with this test failing, which I assumed was indicating some kind of real problem.

But looking at it again now, I suspect what's really going on is that this test was written with an httpbin+werkzeug combination which suffers from postmanlabs/httpbin#647 . i.e., the expected result of the test is basically wrong. httpbin intends to tell werkzeug not to rewrite the Location responses, by setting autocorrect_location_header = False. But with current unpatched httpbin and recent werkzeug, that doesn't work: this is what postmanlabs/httpbin#647 is all about.

So I need to go back and look at what the pytest-httpbin change that added this test was actually trying to do, and if it's a thing that makes sense at all if httpbin is actually behaving as intended, and if so, how we can test it properly.

[graingert]

I updated the test in #76

[kloczek]

Just tested 2.0.0 and now all looks good

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-2.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-httpbin-2.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network' -p no:relaxed
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-httpbin-2.0.0
configfile: setup.cfg
plugins: httpbin-2.0.0, asyncio-0.21.0, timeout-2.1.0, anyio-3.6.2, aiohttp-1.0.4, mock-3.10.0
asyncio: mode=strict
collected 21 items

tests/test_httpbin.py .............                                                                                                                                                   [ 61%]
tests/test_server.py ........                                                                                                                                                         [100%]

==================================================================================== 21 passed in 3.29s =====================================================================================

Thank you 👍