Install psycopg2 in Mac M1

install_psycopg2_mac_m1.md

Install psycopg2 in Mac M1

Error while installing through pip install psycopg2 looks like this:

Please add the directory containing pg_config to the PATH

or specify the full executable path with the option (...)

Reference to the solution here.

Steps followed in my local

Install PostgreSQL, start the deamon and install openssl:

brew install postgresql
pg_ctl -D /opt/homebrew/var/postgres start
brew install openssl

If you already have openssl installed, run:

brew reinstall openssl

Find the path of pg_config and openssl:

which pg_config
which openssl

In my case those were located in these folders:

/opt/homebrew/Cellar/postgresql/13.3/bin/pg_config
/opt/homebrew/Cellar/[email protected]/1.1.1k/bin/openssl

Add that to $PATH:

export PATH=$PATH:/opt/homebrew/Cellar/postgresql/13.3/bin/pg_config
export PATH=$PATH:/opt/homebrew/Cellar/[email protected]/1.1.1k/bin/openssl

Then run: pip install psycopg2.

Problems while executing django command

I encountered the following error while running a django command:

Traceback (most recent call last):
  File "/Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 25, in <module>
    import psycopg2 as Database
  File "/Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/psycopg2/__init__.py", line 51, in <module>
    from psycopg2._psycopg import (                     # noqa
ImportError: dlopen(/Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/psycopg2/_psycopg.cpython-37m-darwin.so, 2): Symbol not found: _PQbackendPID
  Referenced from: /Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/psycopg2/_psycopg.cpython-37m-darwin.so
  Expected in: flat namespace
 in /Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/psycopg2/_psycopg.cpython-37m-darwin.so

Following this thread, this finally worked:

brew install libpq --build-from-source
export LDFLAGS="-L/opt/homebrew/opt/libpq/lib"
pip install psycopg2-binary

Before using that, uninstall psycopg2 and psycopg2-binary:

pip uninstall psycopg2-binary
pip uninstall psycopg2

Thanks for this.

Just an update. I use which pg_config and which openssl - simpler.

Thanks for this.

Just an update. I use which pg_config and which openssl - simpler.

Way simpler @Goldziher! Thanks for that! I updated the gist with your recommendation.

This was wayyyyy simple compared to the others I've seen

This was wayyyyy simple compared to the others I've seen

I'm glad it helps @J-0RI 🙂

This is what I did to solve the problem:

1. Reinstall Openssl

% brew reinstall openssl                                                                          
==> Fetching openssl@3
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/manifests/3.1.1
Already downloaded: /Users/<username>/Library/Caches/Homebrew/downloads/a3831d258a8ccc63638c4ae11c47987af956f6db04046977e1583c1410df00a8--openssl@3-3.1.1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/blobs/sha256:ec0a9f2f744809e81aca2a8953479b490b2c5341238615540c15c74019187ac0
Already downloaded: /Users/<username>/Library/Caches/Homebrew/downloads/10e5960204046ee0ca02040ea350c3d7134158d7b491e148ae41c5ef58b3d475--openssl@3--3.1.1.arm64_ventura.bottle.tar.gz
==> Reinstalling openssl@3 
==> Pouring [email protected]_ventura.bottle.tar.gz
🍺  /opt/homebrew/Cellar/openssl@3/3.1.1: 6,495 files, 28.4MB
==> Running `brew cleanup openssl@3`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

2. Find out folder locations used to set LDFLAGS

After this, I checked where the bin and include folders are for openssl.

% ls /opt/homebrew/Cellar/openssl@3/3.1.1
AUTHORS.md              INSTALL_RECEIPT.json    NEWS.md                 bin                     lib
CHANGES.md              LICENSE.txt             README.md               include                 share

3. Install psycopg2 using LDFLAGS

Now that you know the folder locations, install psycopg2 using env vars:

% env LDFLAGS="-I/opt/homebrew/Cellar/openssl@3/3.1.1/include -L/opt/homebrew/Cellar/openssl@3/3.1.1/lib" pip install psycopg2
Collecting psycopg2
  Using cached psycopg2-2.9.6.tar.gz (383 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (setup.py) ... done
  Created wheel for psycopg2: filename=psycopg2-2.9.6-cp310-cp310-macosx_13_0_arm64.whl size=145782 sha256=701cefbc36bec6bf31751dc21888ee2d96072af99ade9c45a9c7c687d3a7461e
  Stored in directory: /Users/<user>/Library/Caches/pip/wheels/a2/65/83/78e6f42d3b8e22115e894576b71799d96ab5a790b8f7bcfa85
Successfully built psycopg2
Installing collected packages: psycopg2
Successfully installed psycopg2-2.9.6

Thanks @misterEggroll! So the only difference is that you installed psycopg2 with the LDFLAGS, right?

This is what worked for me in a M2 processor, because your instructions didn't work for me, since it was still complaining for not finding pg_config:

brew install postgresql@15

Note that you need to specify the version with @. Otherwise, it won't install.

After installing, you will see the message:

==> Caveats
This formula has created a default database cluster with:
  initdb --locale=C -E UTF-8 /opt/homebrew/var/postgresql@15
For more details, read:
  https://www.postgresql.org/docs/15/app-initdb.html

postgresql@15 is keg-only, which means it was not symlinked into /opt/homebrew,
because this is an alternate version of another formula.

If you need to have postgresql@15 first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"' >> ~/.zshrc

For compilers to find postgresql@15 you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/postgresql@15/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/postgresql@15/include"

To start postgresql@15 now and restart at login:
  brew services start postgresql@15
Or, if you don't want/need a background service you can just run:
  LC_ALL="C" /opt/homebrew/opt/postgresql@15/bin/postgres -D /opt/homebrew/var/postgresql@15
==> Summary
🍺  /opt/homebrew/Cellar/postgresql@15/15.4: 3,698 files, 61.5MB

Just copy the export commands and do one by one the export:

export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"

export LDFLAGS="-L/opt/homebrew/opt/postgresql@15/lib"

export CPPFLAGS="-I/opt/homebrew/opt/postgresql@15/include"

Now do the install: pip install psycopg2.

Thanks for all the details @rafa-munoz!

Just copy export commands and dont forget to run source ~/.zshrc for zsh shell .Use command accordingly for bash.
Now do the install: pip install psycopg2.

This is what worked for me in a M2 processor, because your instructions didn't work for me, since it was still complaining for not finding pg_config:

brew install postgresql@15

Note that you need to specify the version with @. Otherwise, it won't install.

After installing, you will see the message:

==> Caveats
This formula has created a default database cluster with:
  initdb --locale=C -E UTF-8 /opt/homebrew/var/postgresql@15
For more details, read:
  https://www.postgresql.org/docs/15/app-initdb.html

postgresql@15 is keg-only, which means it was not symlinked into /opt/homebrew,
because this is an alternate version of another formula.

If you need to have postgresql@15 first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"' >> ~/.zshrc

For compilers to find postgresql@15 you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/postgresql@15/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/postgresql@15/include"

To start postgresql@15 now and restart at login:
  brew services start postgresql@15
Or, if you don't want/need a background service you can just run:
  LC_ALL="C" /opt/homebrew/opt/postgresql@15/bin/postgres -D /opt/homebrew/var/postgresql@15
==> Summary
🍺  /opt/homebrew/Cellar/postgresql@15/15.4: 3,698 files, 61.5MB

Just copy the export commands and do one by one the export:

export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"

export LDFLAGS="-L/opt/homebrew/opt/postgresql@15/lib"

export CPPFLAGS="-I/opt/homebrew/opt/postgresql@15/include"

Now do the install: pip install psycopg2.

This worked for me on M2 processor. Thanks

Thank you!! I've tried so many different approaches until I've found this here.
The Apple silicon really is a pain in terms of software support...