Running PyRosetta on M1 Mac


IMPORTANT: Running x86 binaries on M1 Mac's is now deprecated, please instead use our native M1 builds available [here]

Running PyRosetta on M1 based Mac's using Apple emulation

by Taylor Jones

Introduction

PyRosetta is currently only available for x86 based macs. It can be run on M1 macs emulating x86 in Apple’s built in emulation tools. There is a large performance impact the first time you load the library in each working session but caching mechanisms make this a fairly workable solution for many workflows which rely upon software which is currently unavailable for the M1 architecture. If you use an M1 mac and depend upon scientific packages that have not been built for arm_64 yet running the x86 version of python and Conda may be the ideal solution for you until the teams that maintain your software get access to the necessary hardware to build packages for m1 macs.

IMPORTANT: IF YOU CURRENTLY HAVE THE M1 VERSION OF CONDA INSTALLED PROCEED AT YOUR OWN RISK. THINGS MAY GET COMPLICATED IF YOU HAVE ENVIRONMENTS OF MIXED ARCHITECTURES.

A couple of pieces of information

  1. Apple’s x86 emulator is also inconveniently named rosetta. You should be prompted to download it the first time you attempt to run an x86 program as listed on apples support page (https://support.apple.com/en-us/HT211861). You can also install it from terminal with “/usr/sbin/softwareupdate --install-rosetta --agree-to-license”

  2. Many of the programs on macs are distributed as universal binaries meaning that they include an x86 and arm_64 versions of the executable. More info on building universal binaries here: https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary

  3. If you would like to specifically run the x86_64 version of a binary you are able to specify it with the arch command “arch -x86_64 PATH_TO_EXECUTABLE” if you start a shell with the x86 architecture x86 versions of universal binaries will be run, If only an x86 executable is available at a given path then that will be run regardless of whether the architecture is specified.

  4. If you run the command “arch” with no arguments you will be able to tell what instruction set is being used by the current shell

Conda and PyRosetta install

  1. Download Anaconda’s installer script from their website I selected the mac installation shell script at the bottom of this page: https://www.anaconda.com/products/individual

  2. Installation of x86 python and conda
    arch -x86_64 /bin/bash
    bash ~/Downloads/Anaconda*.sh
    PROCEED WITH INSTALLATION AS NORMAL
    in the last step select yes to have it configure your .zshrc file in order to use conda by default

  3. Open a new terminal window and type “which python” to see that the path to python is not the system default path. In addition use the file command to see that the file is an x86 executable.
    (base) taylor2@Taylors-MacBook-Pro ~ % which python
    /Users/taylor2/anaconda3/bin/python
    (base) taylor2@Taylors-MacBook-Pro ~ % file $(which python)
    /Users/taylor2/anaconda3/bin/python: Mach-O 64-bit executable x86_64

  4. Now observe that when you deactivate your conda environment the python path defaults back toe the version of python which is bundled with the system. Note that this python is a universal binary containing both an arm and x86 versions of the executable.
    taylorjones@Taylors-MacBook-Pro ~ % which python
    /usr/bin/python
    taylorjones@Taylors-MacBook-Pro ~ % file $(which python)
    /usr/bin/python: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
    /usr/bin/python (for architecture x86_64): Mach-O 64-bit executable x86_64
    /usr/bin/python (for architecture arm64e): Mach-O 64-bit executable arm64e
    NOTE: Since your base conda environment only has an x86 version available at this path this will be run each time you call python from any shell with conda activated. In addition pip, pip3, and conda installs will install in the appropriate path and may be loaded as normal.

  5. With an anaconda environment loaded type “python --version” to get the version of wheel file that you need to download. Obtain a copy of the appropriate wheel file for your python version by following these instruction: https://www.rosettacommons.org/docs/latest/scripting_documentation/PyRosetta/PyRosetta Once you have downloaded the appropriate version of the wheel file install it with pip3 install PATH_TO_WHEEL_FILE The initial load of pyrosetta will take an astounding amount of time but there appears to be some JIT processing voodoo / caching mechanism that makes it so that the cost of emulation is largely borne largely upon the initial load of a library with manageable performance tradeoffs on subsequent loads of the libary (which take place in a reasonable amount of time following the load).
    (base) taylor2@Taylors-MacBook-Pro ~ % time python -c"import pyrosetta"
    python -c"import pyrosetta" 1.22s user 0.61s system 4% cpu 42.142 total
    (base) taylor2@Taylors-MacBook-Pro ~ % time python -c"import pyrosetta"
    python -c"import pyrosetta" 1.16s user 0.22s system 96% cpu 1.432 total
    (base) taylor2@Taylors-MacBook-Pro ~ % time python -c"import pyrosetta"
    python -c"import pyrosetta" 1.17s user 0.21s system 96% cpu 1.429 total