Pip 10 can't upgrade distutils installed packages without help

[espoelstra]

Would it be possible to rescue this type of failure and perform one of the workarounds, either pip install --ignore-installed package or rm -rf $PYTHON_PATH/site-packages/package and then attempting the install via pip again to pull in the removed dependencies correctly?

pypa/pip#5247 (comment)

We would like to stop pinning pip 9, but this pip 10 behavior has broken a number of our cookbooks sadly enough when trying to install the awscli via `python_package 'awscli', which is a pretty important package for everything else we do.

[espoelstra]

According to the comments here, --ignore-installed is the safe way to do this as it takes more time but will forcibly reinstall the required packages via pip and ensure the correct metadata is in place.

https://stackoverflow.com/q/49932926/3794873

[coderanger]

I mean you can set install_options '--ignore-installed' yourself. Is that not enough?

[espoelstra]

@coderanger You are right, if you encounter this error outside of the python_runtime setup it is probably on the user of python_package and not the cookbook itself to add the option to rescue the install. The tricky part is this error could occur on any package that is a dependency of the one you are trying to install, or you could have them all installed correctly and never hit it. I'm not sure how much of a time "waste" it is reinstalling all the dependencies in the case that your desired package isn't already installed or the correct version, but I suppose you should only really hit it once in a converge on a node because all the dependencies for that package that were installed using distutil will get reinstalled via pip and avoid the warning in the future.

My thought with having it in the cookbook as a rescue is then the user wouldn't have to worry about the workaround on the slim chance that they hit it (Ubuntu 14.04 seems particularly prone to this when installing awscli via the cloud-cli cookbook or via pip in general). At least having the workaround documented here should help others that run into this issue, I'm not sure if any of the other open issues have hit this and just didn't track down the root cause, but I'll see about testing the install_options route and I'd imagine that will fix us up nicely.