64bit, multi-arch python web stack (i.e. FAT apache, mod_wsgi, python and mysql) on OS X 10.5 Leopard

Trying to install mod_python or mod_wsgi on my Penryn Macbook with OS X 10.5 Leopard was not a “plug and play” experience.

The stock apache2 server included in OS X 10.5 comes as “FAT” binary, i.e. with both PowerPC and Intel versions for 32bit and 64bit:

$ file /usr/sbin/httpd 
/usr/sbin/httpd: Mach-O universal binary with 4 architectures
/usr/sbin/httpd (for architecture ppc7400):	Mach-O executable ppc
/usr/sbin/httpd (for architecture ppc64):	Mach-O 64-bit executable ppc64
/usr/sbin/httpd (for architecture i386):	Mach-O executable i386
/usr/sbin/httpd (for architecture x86_64):	Mach-O 64-bit executable x86_64

But the stock python version is only available as 32bit application (for both PowerPC and Intel):

$ file /usr/bin/python
/usr/bin/python: Mach-O universal binary with 2 architectures
/usr/bin/python (for architecture ppc7400):	Mach-O executable ppc
/usr/bin/python (for architecture i386):	Mach-O executable i386

Since Apple in its wisdom decided to configure Apache to run as 64bit Intel application (“x86_64″), building mod_python or mod_wsgi against this 64bit apache version fails – there is no “x86_64″ version of python to link to.

Hence, the easiest method to proceed is to strip apache and configure it such that it runs only as 32bit Intel application – a process I have not tried, but that is explained in a comment by Tom Gidden.

The other, more complicated and time consuming option is to leave Apache as it was configured by Apple (so that future updates don’t destroy the setup) and compile FAT binaries with both 32bit and 64bit support of python, mod_wsgi (or mod_python), mysql and mysql-python.

This is what I have done, and here is how:

GNU readline

We need a multi-architecture (“FAT”) version of readline, otherwise the FAT python interpreter we’re trying to compile will be usability challenged.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2.tar.gz
tar xfz readline-5.2.tar.gz 
cd readline-5.2
 
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-001
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-002
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-003
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-004
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-005
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-006
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-007
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-008
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-009
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-010
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-011
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-012
curl -O ftp://ftp.cwru.edu/pub/bash/readline-5.2-patches/readline52-013
 
patch -p0 < readline52-001
patch -p0 < readline52-002
patch -p0 < readline52-003
patch -p0 < readline52-004
patch -p0 < readline52-005
patch -p0 < readline52-006
patch -p0 < readline52-007
patch -p0 < readline52-008
patch -p0 < readline52-009
patch -p0 < readline52-010
patch -p0 < readline52-011
patch -p0 < readline52-012
patch -p0 < readline52-013
 
export MACOSX_DEPLOYMENT_TARGET=10.5
export CFLAGS="-arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -g -Os -pipe -no-cpp-precomp"
export CCFLAGS="-arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -g -Os -pipe"
export CXXFLAGS="-arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -g -Os -pipe"
export LDFLAGS="-arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -bind_at_load"
 
./configure
make && sudo make install

check that the result contains all 4 architectures

$ file /usr/local/lib/libreadline.5.2.dylib
/usr/local/lib/libreadline.5.2.dylib: Mach-O universal binary with 4 architectures
/usr/local/lib/libreadline.5.2.dylib (for architecture ppc7400): Mach-O dynamically linked shared library ppc
/usr/local/lib/libreadline.5.2.dylib (for architecture ppc64): Mach-O 64-bit dynamically linked shared library ppc64
/usr/local/lib/libreadline.5.2.dylib (for architecture i386): Mach-O dynamically linked shared library i386
/usr/local/lib/libreadline.5.2.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64

close and re-open Terminal, to get rid of CFLAGS etc. envvars.

Python 2.6

We’re compiling python 2.6 from source, since the different Mac OS X installers provided by python.org don’t support 64bit. We’re linking our python to the just compiled FAT version of the readline library.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
curl -O http://python.org/ftp/python/2.6.2/Python-2.6.2.tar.bz2
bzip2 -d Python-2.6.2.tar.bz2
tar xf Python-2.6.2.tar
cd Python-2.6.2
unset DYLD_LIBRARY_PATH
unset LD_LIBRARY_PATH
 
./configure --enable-framework=/Library/Frameworks \
--enable-universalsdk=/ \
MACOSX_DEPLOYMENT_TARGET=10.5 \
--with-universal-archs=all \
--with-readline-dir=/usr/local
 
make && make test
sudo make install

check that the result contains all 4 architectures

$ file /Library/Frameworks/Python.framework/Versions/2.6/Python 
/Library/Frameworks/Python.framework/Versions/2.6/Python: Mach-O universal binary with 4 architectures
/Library/Frameworks/Python.framework/Versions/2.6/Python (for architecture i386): Mach-O dynamically linked shared library i386
/Library/Frameworks/Python.framework/Versions/2.6/Python (for architecture ppc7400): Mach-O dynamically linked shared library ppc
/Library/Frameworks/Python.framework/Versions/2.6/Python (for architecture ppc64): Mach-O 64-bit dynamically linked shared library ppc64
/Library/Frameworks/Python.framework/Versions/2.6/Python (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64

Make sure that /usr/local/bin comes before /usr/bin in your shell PATH. Test the new version

$ /usr/local/bin/python
Python 2.6.2 (r262:71600, Apr 15 2009, 23:22:53) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

mod_wsgi

Now, we’re compiling a FAT version of mod_wsgi, using the just installed FAT python 2.6

1
2
3
4
5
6
7
8
9
10
curl -O http://modwsgi.googlecode.com/files/mod_wsgi-2.4.tar.gz
tar xvfz mod_wsgi-2.4.tar.gz
cd mod_wsgi-2.4
 
./configure \
--with-python=/usr/local/bin/python \
--with-apxs=/usr/sbin/apxs
 
make
sudo make install

Again, checking the result

$ file /usr/libexec/apache2/mod_wsgi.so 
/usr/libexec/apache2/mod_wsgi.so: Mach-O universal binary with 4 architectures
/usr/libexec/apache2/mod_wsgi.so (for architecture ppc7400): Mach-O bundle ppc
/usr/libexec/apache2/mod_wsgi.so (for architecture ppc64): Mach-O 64-bit bundle ppc64
/usr/libexec/apache2/mod_wsgi.so (for architecture i386): Mach-O bundle i386
/usr/libexec/apache2/mod_wsgi.so (for architecture x86_64): Mach-O 64-bit bundle x86_64

Change the apache configuration in /etc/apache2/httpd.conf and add the following line

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

starting and stopping apache

sudo /usr/sbin/apachectl stop
sudo /usr/sbin/apachectl start

should yield the following line in the error_log

$ tail -f /var/log/apache2/error_log
Apache/2.2.9 (Unix) mod_ssl/2.2.9 OpenSSL/0.9.7l DAV/2 mod_wsgi/2.4 Python/2.6.1 configured -- resuming normal operations

mod_wsgi is successfully loaded by the 64bit apache process.

The configuration and build of mod_python is left as an excercise for the inclined reader.

MySQL and mysql-python

in order to make mysql-python play nice with all this, we need to compile it against a FAT version of MySQL (mysql.com only provides 32bit-only or 64bit-only installers for Mac OS X. Sigh.).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
curl -O http://mysql.dataphone.se/Downloads/MySQL-5.1/mysql-5.1.33.zip
unzip mysql-5.1.33.zip
cd mysql-5.1.33
 
CC=gcc CXX=gcc MACOSX_DEPLOYMENT_TARGET=10.5 \
CFLAGS='-O3 -fno-common -fno-omit-frame-pointer -arch i386 -arch x86_64 -arch ppc7400 -arch ppc64' \
CXXFLAGS='-O3 -fno-omit-frame-pointer -felide-constructors -fno-exceptions -fno-rtti -arch i386 -arch x86_64 -arch ppc7400 -arch ppc64' \
LDFLAGS='-O3 -arch i386 -arch x86_64 -arch ppc7400 -arch ppc64' \
./configure \
'--prefix=/usr/local/mysql' \
'--disable-dependency-tracking' \
'--localstatedir=/usr/local/mysql/data' \
'--libexecdir=/usr/local/mysql/bin' \
'--enable-thread-safe-client' \
'--enable-local-infile' \
'--with-big-tables' \
'--with-extra-charsets=complex' \
'--enable-shared' \
'--with-plugins=innobase'
 
make
sudo make install

confirming one more time that all 4 architectures are present

$ file /usr/local/mysql/bin/mysql
/usr/local/mysql/bin/mysql: Mach-O universal binary with 4 architectures
/usr/local/mysql/bin/mysql (for architecture i386): Mach-O executable i386
/usr/local/mysql/bin/mysql (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/local/mysql/bin/mysql (for architecture ppc7400): Mach-O executable ppc
/usr/local/mysql/bin/mysql (for architecture ppc64): Mach-O 64-bit executable ppc64

we can now go over to compile mysql-python (just running easy_install mysql-python didn’t work for me somehow, and I lost patience there)

1
2
3
4
5
6
ARCHFLAGS='-arch ppc -arch ppc64 -arch i386 -arch x86_64'
export ARCHFLAGS
curl -O http://ovh.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.3c1.tar.gz
tar xfz MySQL-python-1.2.3c1.tar.gz
cd MySQL-python-1.2.3c1
sudo python setup.py install

Voilà!

Share and Enjoy:
  • Print
  • Digg
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • LinkedIn
  • Reddit
  • Slashdot
  • Suggest to Techmeme via Twitter
  • Technorati
  • Twitter

Tags: , , ,

  • @Graham Dumpleton

    I am setting up a brand new mac pro with 64 bit architecture, and seem to have only the two:

    file /System/Library/Frameworks/Python.framework/Versions/2.3/Python
    /System/Library/Frameworks/Python.framework/Versions/2.3/Python: Mach-O universal binary with 2 architectures
    /System/Library/Frameworks/Python.framework/Versions/2.3/Python (for architecture i386): Mach-O dynamically linked shared library i386
    /System/Library/Frameworks/Python.framework/Versions/2.3/Python (for architecture ppc7400): Mach-O dynamically linked shared library ppc
  • piergiorgioniero
    Everything worked fine for me.
    I installed python 2.6.2 from the source, as I tried to compile mysql-python I had this error:

    Traceback (most recent call last):
    File "setup.py", line 5, in <module>
    from setuptools import setup, Extension
    ImportError: No module named setuptools

    and just installed the module following these indications http://pypi.python.org/pypi/setuptools#cygwin-m...
  • repi
    Following the instructions successfully builds Python 2.6.2 and reports it includes the 64 bit executable. However, when starting python up from the command prompt, Activity Monitor reports the instance as Python-32 with kind "Intel".

    Interestingly, if I start python.exe from the build folder, it reports it as python.exe Intel (64 bit).

    Any ideas? I have followed the instructions closely and got the same result on two different computers (a Mac Book Pro running Mac OS X 10.5 and Mac Pro running Mac OS X 10.5 Server).
  • Sorry, you are wrong.

    Although /usr/bin/python on MacOS X 10.5 only provides 32 bit architectures, the Python framework (shared library) does include all architectures. Since both mod_python and mod_wsgi link only to the Python framework and do not use /usr/bin/python, they should work fine. Run:

    file /System/Library/Frameworks/Python.framework/Versions/2.3/Python

    and you should see:

    $ file /System/Library/Frameworks/Python.framework/Versions/Current/Python
    /System/Library/Frameworks/Python.framework/Versions/Current/Python:
    Mach-O universal binary with 4 architectures
    /System/Library/Frameworks/Python.framework/Versions/Current/Python
    (for architecture ppc7400): Mach-O dynamically linked shared library ppc
    /System/Library/Frameworks/Python.framework/Versions/Current/Python
    (for architecture ppc64): Mach-O 64-bit dynamically linked shared
    library ppc64
    /System/Library/Frameworks/Python.framework/Versions/Current/Python
    (for architecture i386): Mach-O dynamically linked shared library
    i386
    /System/Library/Frameworks/Python.framework/Versions/Current/Python
    (for architecture x86_64): Mach-O 64-bit dynamically linked shared
    library x86_64

    Only caveat to this is that you need to use mod_python source code from its subversion repository, as 3.3.1 tar ball has not got required changes to build properly.

    There shouldn't be any problem at all with mod_wsgi if you are using standard Apache and Python from Apple. The development of mod_wsgi is actually done on MacOS X 10.5, so know it works fine.

    Various details related to mod_wsgi and MacOS X 10.5 can be found at:

    http://code.google.com/p/modwsgi/wiki/Installat...

    So, the only thing you probably need to do anything special about is MySQL.
  • Adam Ginsburg
    I went through the python & readline bit of this install process and after some wrangling got it to work.... but apparently python 2.6.2 doesn't compile numpy correctly, which was my whole reason for installing 64 bit python. Is it specifically 2.6.2 that is 64 bit compileable on mac, or could 2.5 also do it?
  • captnswing
    I managed to compile python 2.5.4 as well using the instructions above!
blog comments powered by Disqus