Build pyenv Python versions on macOS Catalina 10.15
19 February 2020 9 comments Python, MacOSX
https://github.com/pyenv/pyenv/issues/1348#issuecomment-588001367
I'm still working on getting pyenv in my bloodstream. It seems like totally the right tool for having different versions of Python available on macOS that don't suddenly break when you run brew upgrade periodically. But every thing I tried failed with an error similar to this:
python-build: use openssl from homebrew
python-build: use readline from homebrew
Installing Python-3.7.0...
python-build: use readline from homebrew
BUILD FAILED (OS X 10.15.x using python-build 20XXXXXX)
Inspect or clean up the working tree at /var/folders/mw/0ddksqyn4x18lbwftnc5dg0w0000gn/T/python-build.20190528163135.60751
Results logged to /var/folders/mw/0ddksqyn4x18lbwftnc5dg0w0000gn/T/python-build.20190528163135.60751.log
Last 10 log lines:
./Modules/posixmodule.c:5924:9: warning: this function declaration is not a prototype [-Wstrict-prototypes]
if (openpty(&master_fd, &slave_fd, NULL, NULL, NULL) != 0)
^
./Modules/posixmodule.c:6018:11: error: implicit declaration of function 'forkpty' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
pid = forkpty(&master_fd, NULL, NULL, NULL);
^
./Modules/posixmodule.c:6018:11: warning: this function declaration is not a prototype [-Wstrict-prototypes]
2 warnings and 2 errors generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
I read through the Troubleshooting FAQ and the "Common build problems" documentation. xcode was up to date and I had all the related brew packages upgraded. Nothing seemed to work.
Until I saw this comment on an open pyenv issue: "Unable to install any Python version on MacOS"
All I had to do was replace the 10.14 for 10.15 and now it finally worked here on Catalina 10.15. So, the magical line was this:
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk \ MACOSX_DEPLOYMENT_TARGET=10.15 \ PYTHON_CONFIGURE_OPTS="--enable-framework" \ pyenv install -v 3.7.6
Hopefully, by blogging about it you'll find this from Googling and I'll remember the next time I need it because it did eat 2 hours of precious evening coding time.
You are awesome! I was getting nowhere, and that was exactly the solution I needed.
Seconding Peter's comment!
Thank you for this post! It saved me a lot of time!
Thanks for the contribution!
On the new MacOS version (11.0 aka Big Sur), the command line becomes:
SKDROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk \
MACOSX_DEPLOYMENT_TARGET=11.0 \
PYTHON_CONFIGURE_OPTS="--enable-framework" \
pyenv install 3.7.6
But then another error comes up:
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.5.10.tar.xz...
-> https://www.python.org/ftp/python/3.5.10/Python-3.5.10.tar.xz
Installing Python-3.5.10...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
BUILD FAILED (OS X 11.0.1 using python-build 1.2.21-1-g943015eb)
Inspect or clean up the working tree at /var/folders/hg/fcmc5nwj43dg2p_cp6gc4rmr0000gn/T/python-build.20201116231208.85510
Results logged to /var/folders/hg/fcmc5nwj43dg2p_cp6gc4rmr0000gn/T/python-build.20201116231208.85510.log
Last 10 log lines:
posix_error();
^~~~~~~~~~~
4 warnings generated.
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -Werror=declaration-after-statement -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/darnaud/.pyenv/versions/3.5.10/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/darnaud/.pyenv/versions/3.5.10/include -DPy_BUILD_CORE -c ./Modules/_sre.c -o Modules/_sre.o
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -Werror=declaration-after-statement -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/darnaud/.pyenv/versions/3.5.10/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/darnaud/.pyenv/versions/3.5.10/include -DPy_BUILD_CORE -c ./Modules/_codecsmodule.c -o Modules/_codecsmodule.o
11 warnings and 1 error generated.
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -Werror=declaration-after-statement -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/darnaud/.pyenv/versions/3.5.10/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/darnaud/.pyenv/versions/3.5.10/include -DPy_BUILD_CORE -c ./Modules/_weakref.c -o Modules/_weakref.o
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
1 warning generated.
Every time I attempt this, it gets as far as checking through the various dependencies and this happens:
checking for python3... python3
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... checking for --without-gcc... no
checking for --with-icc... no
checking for gcc... clang
checking whether the C compiler works... no
Between the script deciding that gcc is clang, and that the C compiler's bad, a dialog comes up insisting I need to install the XCode command line tools. I've installed them multiple times, but it makes no difference. The script decides the C compiler is no good, and then dies like this:
configure: error: in `/var/folders/st/vtyr0y6902v7ylvbbq31vv9r0000gn/T/python-build.20210213162036.58382/Python-3.7.6':
configure: error: C compiler cannot create executables
See `config.log' for more details
Any ideas?
A version of the solution found in the comment at https://github.com/pyenv/pyenv/issues/1348#issuecomment-514057168 fixed this for me.
Don't let homebrew link binutils.
Hi Peter, I have enjoyed using pyenv for a long time, what I liked most for deployed python stuff is that since the app now has its own python, one does not risk issues when suddenly the OS package manager updates the OS python to a new version. The local version will always stay the same until you decide to run the app with a different version.
Over xmas I started to play with a full version handler, asdf. For python it does the same as pyenv, but it has the huge advantage of also handling a ton of other languages and tools in the same way through plugins, so I would recommend you to take a look and see if it could be something for you if you enjoy pyenv.
One thing that took me a while to figure out with asdf and python before I got to work as smooth as pyenv is that if you want to use venvs within python to keep your installed pips separate and such. After installing a python version, you also need to install your preferred python venv handler(-s) in every python version, I use virtualenvwrapper, but it's the same regardless of your venv handler of choice. Then it works smoothly! I have now migrated my env to use asdf instead of pyenv.
https://github.com/asdf-vm/asdf
Thanks!
The hardest part for me is to remember the various commands. I only very rarely create a new Python project and each time I have to reboot my brain/memory as to how it all works. Either it's a challenge of plainly remembering the commands, or it's a challenge of finding previous commands in the bash history, or it's a challenge of remembering where I wrote my notes on how to use it.
The tech isn't particularly important. It's more important about the cognitive challenge for things one doesn't do on a daily basis.
But it's good to know that there exists a project that can help with more versions and stuff. I've been lucky over the last couple of years on *not* having to worry about testing on multiple versions of Node. But once that changes again, I'll seriously consider asdf as a possible contender to other pure-Node based solutions.
Thank you so much. You saved 2 hours of my life