Installing: "subprocess exited with error"

[hughjonesd]

Hi,

Here's output from trying to install the requirements.txt file.

My python version is 3.11.5. numpy seems to install on its own successfully.

Platform: M2 Mac with Ventura 13.4.1.

/usr/local/bin/python3 -m pip install -r ~/pgi_correct/requirements.txt 
Collecting attrs==20.1.0 (from -r /Users/davidhugh-jones/pgi_correct/requirements.txt (line 1))
  Downloading attrs-20.1.0-py2.py3-none-any.whl (49 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.0/49.0 kB 194.6 kB/s eta 0:00:00
Collecting coverage==5.2.1 (from -r /Users/davidhugh-jones/pgi_correct/requirements.txt (line 2))
  Downloading coverage-5.2.1.tar.gz (694 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 694.1/694.1 kB 233.9 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting importlib-metadata==1.7.0 (from -r /Users/davidhugh-jones/pgi_correct/requirements.txt (line 3))
  Downloading importlib_metadata-1.7.0-py2.py3-none-any.whl (31 kB)
Collecting iniconfig==1.0.1 (from -r /Users/davidhugh-jones/pgi_correct/requirements.txt (line 4))
  Downloading iniconfig-1.0.1-py3-none-any.whl (4.2 kB)
Collecting more-itertools==8.5.0 (from -r /Users/davidhugh-jones/pgi_correct/requirements.txt (line 5))
  Downloading more_itertools-8.5.0-py3-none-any.whl (44 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.9/44.9 kB 292.9 kB/s eta 0:00:00
Collecting numpy==1.19.1 (from -r /Users/davidhugh-jones/pgi_correct/requirements.txt (line 6))
  Downloading numpy-1.19.1.zip (7.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.3/7.3 MB 258.5 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [58 lines of output]
      Running from numpy source directory.
      <string>:470: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
      /private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-install-b2ear7da/numpy_14218c892c13452d8f61775c1f1e2c90/tools/cythonize.py:73: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        required_version = LooseVersion('0.29.21')
      /private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-install-b2ear7da/numpy_14218c892c13452d8f61775c1f1e2c90/tools/cythonize.py:75: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        if LooseVersion(cython_version) < required_version:
      warning: _philox.pyx:19:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: /private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-install-b2ear7da/numpy_14218c892c13452d8f61775c1f1e2c90/numpy/__init__.pxd:17:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
                  self.rng_state.ctr.v[i] = counter[i]
      
              self._reset_state_variables()
      
              self._bitgen.state = <void *>&self.rng_state
              self._bitgen.next_uint64 = &philox_uint64
                                         ^
      ------------------------------------------------------------
      
      _philox.pyx:195:35: Cannot assign type 'uint64_t (*)(void *) except? -1 nogil' to 'uint64_t (*)(void *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to type 'uint64_t (void *) except? -1 nogil'.
      Processing numpy/random/_bounded_integers.pxd.in
      Processing numpy/random/_philox.pyx
      Traceback (most recent call last):
        File "/private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-install-b2ear7da/numpy_14218c892c13452d8f61775c1f1e2c90/tools/cythonize.py", line 235, in <module>
          main()
        File "/private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-install-b2ear7da/numpy_14218c892c13452d8f61775c1f1e2c90/tools/cythonize.py", line 231, in main
          find_process_files(root_dir)
        File "/private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-install-b2ear7da/numpy_14218c892c13452d8f61775c1f1e2c90/tools/cythonize.py", line 222, in find_process_files
          process(root_dir, fromfile, tofile, function, hash_db)
        File "/private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-install-b2ear7da/numpy_14218c892c13452d8f61775c1f1e2c90/tools/cythonize.py", line 188, in process
          processor_function(fromfile, tofile)
        File "/private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-install-b2ear7da/numpy_14218c892c13452d8f61775c1f1e2c90/tools/cythonize.py", line 77, in process_pyx
          subprocess.check_call(
        File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/usr/local/bin/python3', '-m', 'cython', '-3', '--fast-fail', '-o', '_philox.c', '_philox.pyx']' returned non-zero exit status 1.
      Cythonizing sources
      Traceback (most recent call last):
        File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-build-env-4orxtoff/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 396, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-build-env-4orxtoff/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 507, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "/private/var/folders/mr/5c9516m5013_s0fjflvfx6f00000gn/T/pip-build-env-4orxtoff/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 499, in <module>
        File "<string>", line 479, in setup_package
        File "<string>", line 274, in generate_cython
      RuntimeError: Running cythonize failed!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[DiasPereira]

I have the exact same issue. Any hints are highly appreciated.

[JonJala]

Judging from numpy/numpy#24377 and numpy/numpy#24330, it looks like it could be an issue with your version of Cython? There is mention made of using the "--no-build-isolation" flag with pip, so you could try installing Numpy directly using that and see if it helps?

…

On Mon, Nov 6, 2023 at 2:20 PM DiasPereira ***@***.***> wrote: I have the exact same issue. Any hints are highly appreciated. — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APIOF52Q5REKMECAMWX6ERDYDEZ7JAVCNFSM6AAAAAA5JNI2FOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJWGEYDQNJSGM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[DiasPereira]

Hello,

I finally was able to complete the installation. The issue was that I could not install the version of the modules specified in the "requirements.txt". I installed them sequentially, and without the version specification when I encountered an error. I'm pasting my code below in case it is helpful for someone else.

#load python
module purge
module load 2022
module spider python
module load Python/3.10.4-GCCcore-11.3.0

python -h

#install virtual environment
python -m pip install --user virtualenv
python -m virtualenv --help

#use virtual environment
virtualenv -p $(which python3) pgic_env
source pgic_env/bin/activate

#install required modules
pip install attrs==20.1.0
pip install coverage==5.2.1
pip install importlib-metadata==1.7.0
pip install iniconfig==1.0.1
pip install more-itertools==8.5.0
pip install numpy
#successfully install numpy 1.26.1
#pip install numpy==1.19.1
pip install packaging==20.4
#pip install pandas==1.1.1
pip install pandas
#Successfully installed pandas-2.1.2
pip install patsy==0.5.1
pip install pluggy==0.13.1
pip install py==1.9.0
pip install pyparsing==2.4.7
pip install pytest==6.0.1
pip install pytest-cov==2.10.1
#pip install python-dateutil==2.8.1
pip install python-dateutil
#pip install pytz==2020.1
pip install pytz
#pip install scipy==1.5.2
pip install spicy
pip install six==1.15.0
#pip install statsmodels==0.12.0
pip install statsmodels
pip install toml==0.10.1
pip install wget==3.2
pip install zipp==3.1.0

#clone repository
git clone https://github.com/JonJala/pgi_correct.git
cd pgi_correct

#test if correctly installed
python ./pgic.py -h

[hughjonesd]

I updated my requirements.txt and it seems to work for me. Submitted a pull request.