building portable-exe: Cannot import 'hatchling.build'

[Kapral67]

Describe the bug

pip._vendor.pyproject_hooks._impl.BackendUnavailable: Cannot import 'hatchling.build'

when building aws-cli portable-exe on platforms that do not have precompiled booloaders for pyinstaller

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

make to succeed

Current Behavior

PYTHONDONTWRITEBYTECODE=1 "/dev/shm/venv/bin/python" "./backends/build_system" \
        build \
        --artifact "portable-exe" \
        --build-dir "./build" --download-deps
Collecting flit-core==3.12.0 (from -r /dev/shm/aws-cli/requirements/download-deps/bootstrap-lock.txt (line 9))
  Using cached flit_core-3.12.0-py3-none-any.whl (45 kB)
Requirement already satisfied: pip==25.3 in ./build/venv/lib/python3.13/site-packages (from -r /dev/shm/aws-cli/requirements/download-deps/bootstrap-lock.txt (line 13)) (25.3)
Collecting setuptools==78.1.1 (from -r /dev/shm/aws-cli/requirements/download-deps/bootstrap-lock.txt (line 17))
  Using cached setuptools-78.1.1-py3-none-any.whl (1.3 MB)
Collecting wheel==0.38.4 (from -r /dev/shm/aws-cli/requirements/download-deps/bootstrap-lock.txt (line 21))
  Using cached wheel-0.38.4-py3-none-any.whl (36 kB)
Installing collected packages: wheel, setuptools, flit-core
Successfully installed flit-core-3.12.0 setuptools-78.1.1 wheel-0.38.4

[notice] A new release of pip is available: 25.3 -> 26.0.1
[notice] To update, run: /dev/shm/aws-cli/build/venv/bin/python -m pip install --upgrade pip
Collecting altgraph==0.17.5 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 7))
  Using cached altgraph-0.17.5-py2.py3-none-any.whl (21 kB)
Collecting awscrt==0.31.2 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 13))
  Using cached awscrt-0.31.2-cp313-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.2 MB)
Collecting colorama==0.4.6 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 55))
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting distro==1.8.0 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 59))
  Using cached distro-1.8.0-py3-none-any.whl (20 kB)
Collecting docutils==0.19 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 63))
  Using cached docutils-0.19-py3-none-any.whl (570 kB)
Collecting importlib-metadata==7.1.0 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 67))
  Using cached importlib_metadata-7.1.0-py3-none-any.whl (24 kB)
Collecting jmespath==1.0.1 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 71))
  Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting macholib==1.16.4 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 75))
  Using cached macholib-1.16.4-py2.py3-none-any.whl (38 kB)
Collecting packaging==26.0 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 79))
  Using cached packaging-26.0-py3-none-any.whl (74 kB)
Collecting prompt-toolkit==3.0.51 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 85))
  Using cached prompt_toolkit-3.0.51-py3-none-any.whl (387 kB)
Collecting pyinstaller==6.19.0 (from -r /dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt (line 89))
  Using cached pyinstaller-6.19.0.tar.gz (4.0 MB)
  Preparing metadata (pyproject.toml) ... done

[notice] A new release of pip is available: 25.3 -> 26.0.1
[notice] To update, run: /dev/shm/aws-cli/build/venv/bin/python -m pip install --upgrade pip
ERROR: Exception:
Traceback (most recent call last):
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/cli/base_command.py", line 107, in _run_wrapper
    status = _inner_run()
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/cli/base_command.py", line 98, in _inner_run
    return self.run(options, args)
           ~~~~~~~~^^^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/cli/req_command.py", line 85, in wrapper
    return func(self, options, args)
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/commands/install.py", line 388, in run
    requirement_set = resolver.resolve(
        reqs, check_supported_wheels=not options.target_dir
    )
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 99, in resolve
    result = self._result = resolver.resolve(
                            ~~~~~~~~~~~~~~~~^
        collected.requirements, max_rounds=limit_how_complex_resolution_can_be
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_vendor/resolvelib/resolvers/resolution.py", line 601, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_vendor/resolvelib/resolvers/resolution.py", line 434, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_vendor/resolvelib/resolvers/resolution.py", line 150, in _add_to_criteria
    if not criterion.candidates:
           ^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_vendor/resolvelib/structs.py", line 194, in __bool__
    return bool(self._sequence)
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 165, in __bool__
    self._bool = any(self)
                 ~~~^^^^^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 149, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
                       ^^^^^^^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 39, in _iter_built
    candidate = func()
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 180, in _make_candidate_from_link
    base: BaseCandidate | None = self._make_base_candidate_from_link(
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        link, template, name, version
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 226, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ~~~~~~~~~~~~~^
        link,
        ^^^^^
    ...<3 lines>...
        version=version,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 318, in __init__
    super().__init__(
    ~~~~~~~~~~~~~~~~^
        link=link,
        ^^^^^^^^^^
    ...<4 lines>...
        version=version,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__
    self.dist = self._prepare()
                ~~~~~~~~~~~~~^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 238, in _prepare
    dist = self._prepare_distribution()
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 329, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/operations/prepare.py", line 543, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/operations/prepare.py", line 658, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
        req,
    ...<3 lines>...
        self.check_build_deps,
    )
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/operations/prepare.py", line 77, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        build_env_installer, build_isolation, check_build_deps
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/distributions/sdist.py", line 71, in prepare_distribution_metadata
    self.req.prepare_metadata()
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/req/req_install.py", line 530, in prepare_metadata
    self.metadata_directory = generate_metadata(
                              ~~~~~~~~~~~~~~~~~^
        build_env=self.build_env,
        ^^^^^^^^^^^^^^^^^^^^^^^^^
        backend=self.pep517_backend,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        details=details,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/operations/build/metadata.py", line 34, in generate_metadata
    distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_internal/utils/misc.py", line 715, in prepare_metadata_for_build_wheel
    return super().prepare_metadata_for_build_wheel(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        metadata_directory=metadata_directory,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        config_settings=cs,
        ^^^^^^^^^^^^^^^^^^^
        _allow_fallback=_allow_fallback,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 224, in prepare_metadata_for_build_wheel
    return self._call_hook(
           ~~~~~~~~~~~~~~~^
        "prepare_metadata_for_build_wheel",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        },
        ^^
    )
    ^
  File "/dev/shm/aws-cli/build/venv/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 402, in _call_hook
    raise BackendUnavailable(
    ...<4 lines>...
    )
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Cannot import 'hatchling.build'
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/dev/shm/aws-cli/./backends/build_system/__main__.py", line 127, in <module>
    main()
    ~~~~^^
  File "/dev/shm/aws-cli/./backends/build_system/__main__.py", line 123, in main
    parsed_args.func(parsed_args)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/./backends/build_system/__main__.py", line 41, in build
    aws_venv = _bootstap_venv(
        parsed_args.build_dir,
        parsed_args.artifact,
        parsed_args.download_deps,
    )
  File "/dev/shm/aws-cli/./backends/build_system/__main__.py", line 76, in _bootstap_venv
    aws_venv.bootstrap(artifact_type, download_deps)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/./backends/build_system/awscli_venv.py", line 56, in bootstrap
    self._install_requirements(PORTABLE_EXE_REQUIREMENTS_LOCK)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/./backends/build_system/awscli_venv.py", line 79, in _install_requirements
    self._pip_install(
    ~~~~~~~~~~~~~~~~~^
        ["--no-build-isolation", "-r", requirements_file],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        cwd=cwd,
        ^^^^^^^^
    )
    ^
  File "/dev/shm/aws-cli/./backends/build_system/awscli_venv.py", line 135, in _pip_install
    self._utils.run(args, **run_kwargs)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/aws-cli/./backends/build_system/utils.py", line 235, in run
    return subprocess.run(args, **kwargs)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 577, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/dev/shm/aws-cli/build/venv/bin/python', '-m', 'pip', 'install', '--no-build-isolation', '-r', PosixPath('/dev/shm/aws-cli/requirements/download-deps/portable-exe-lock.txt')]' returned non-zero exit status 2.
make: *** [Makefile:20: build] Error 1

Reproduction Steps

export PIP_NO_BINARY=pyinstaller
export PYINSTALLER_COMPILE_BOOTLOADER=1
./configure --with-download-deps --with-install-type=portable-exe
make

Possible Solution

hatchling not included in requirements/download-deps/portable-exe-lock.txt?

Additional Information/Context

No response

CLI version used

2.34.13

Environment details (OS name and version, etc.)

6.19.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 14 Mar 2026 01:07:43 +0000 x86_64 GNU/Linux

[hssyoo]

Thanks for raising this with us @Kapral67. PyInstaller 6.16.0 started using hatchling for its build backend, which is why the source build is expecting hatchling: pyinstaller/pyinstaller@5df0f47

Specifically, this behavior doesn't pair well with the fact that the AWS CLI's source build uses pip's --no-build-isolation flag when downloading build/runtime dependencies to ensure reproducibility:

aws-cli/backends/build_system/awscli_venv.py

Lines 78 to 82 in d2283bb

	def _install_requirements(self, requirements_file, cwd=None):
	self._pip_install(
	["--no-build-isolation", "-r", requirements_file],
	cwd=cwd,
	)

1. export PIP_NO_BINARY=pyinstaller forces PyInstaller to be built from source
2. PyInstaller's pyproject.toml requires hatchling to build
3. --no-build-isolation forces pip to use the current virtual environment and does not create an isolated one where it can auto-install hatchling
4. PyInstaller fails to build

The build dependencies declared in requirements/ are designed to build the AWS CLI v2 from source and not its own dependencies. Mainly, it helps us avoid whack-a-mole every time a dependency changes its build backend and keeps the dependency graph manageable.

There are good reasons to build AWS CLI's dependencies from source, like if a pre-built wheel isn't available on PyPI. But I'm not sure if declaring build dependencies for all packages is the right direction. I'm curious how you'd feel about a flag like --with-build-isolation that exposes an option to omit the --no-build-isolation flag when installing dependencies.

[Kapral67]

yes, exposing with-build-isolation would work for me.
thanks @hssyoo

[hssyoo]

Gotcha, I'll talk with the team about exposing configuration. In the meantime, could you verify that removing the --no-build-isolation arg in:

aws-cli/backends/build_system/awscli_venv.py

Lines 78 to 82 in d2283bb

	def _install_requirements(self, requirements_file, cwd=None):
	self._pip_install(
	["--no-build-isolation", "-r", requirements_file],
	cwd=cwd,
	)

fixes your issue?

[Kapral67]

yes make succeeds after enabling build isolation