Skip to content

Instantly share code, notes, and snippets.

@timlinux
Created September 3, 2019 10:52
Show Gist options
  • Save timlinux/df01023e6d5468fb47eb233fa56eff2c to your computer and use it in GitHub Desktop.
Save timlinux/df01023e6d5468fb47eb233fa56eff2c to your computer and use it in GitHub Desktop.
# (THIS DOCUMENT IS NOT UP TO DATE)
# Developing QGIS using Homebrew dependencies
In addition to using this tap to install [QGIS development
formulae](../Formula), you can also use it to fully set up a development
environment for an externally built QGIS from a clone of the current
[development (master) branch](https://github.com/qgis/QGIS) of the source code
tree.
> Note: This setup, though heavily tested, is currently _experimental_ and may change.
For the terminally lazy, who are already comfortable with Homebrew, [jump to sample Terminal session](#terminal).
## Development Tools
This tutorial is based upon the following software:
* [Qt Creator](http://qt-project.org/downloads) for CMake/C++ development of ([core source](https://github.com/qgis/QGIS/tree/master/src) and [plugins](https://github.com/qgis/QGIS/tree/master/src/plugins))
* [PyCharm Community Edition](http://www.jetbrains.com/pycharm/download/) for Python development of ([PyQGIS plugins/apps](http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/), [Python unit tests](https://github.com/qgis/QGIS/tree/master/tests/src/python), [reStructuredText for documentation](https://github.com/qgis/QGIS-Documentation)).
* macOS XCode (download via Mac App Store and _launch at least once_) and Xcode Command Line Tools (run `xcode-select --install` after installing Xcode), for Homebrew and building QGIS source. QGIS's [CMake](http://www.cmake.org) build process uses generated build files for building QGIS source directly with the `clang` compiler, _not via Xcode project files_.
## Homebrew
See http://brew.sh and [Homebrew docs](https://github.com/Homebrew/brew/tree/master/docs) for info on installing Homebrew.
After Homebrew is installed run the following and fix everything that it mentions, if you can:
```sh
brew doctor
```
### Homebrew configuration
Homebrew now defaults to _auto-updating_ itself (runs `brew update`) upon *every* `brew install` invocation. While this can be handy for keeping up with the latest changes, it can also quickly break an existing build's linked-to libraries. Consider setting the `HOMEBREW_NO_AUTO_UPDATE` environment variable to turn this off, thereby forcing manual running of `brew update`:
```sh
export HOMEBREW_NO_AUTO_UPDATE=1
```
See end of `man brew` for other environment variables.
### Homebrew prefix
While all of the formulae and scripts support building QGIS using Homebrew
installed to a non-standard prefix, e.g. `/opt/homebrew`, **do yourself a
favor** (especially if you are new to Homebrew) and [install in the default
directory of
`/usr/local`](https://github.com/Homebrew/brew/blob/master/docs/Installation.md).
QGIS has many dependencies which are available as
["bottles"](https://github.com/Homebrew/brew/blob/master/docs/Bottles.md)
(pre-built binary installs) from the Homebrew project. Installing Homebrew to a
non-standard prefix will force many of the bottled formulae to be built from
source, since many of the available bottles are built specific to `/usr/local`.
Such unnecessary building from source can comparatively take hours and hours
more, depending upon your available CPU cores.
If desired, this setup supports QGIS builds where the dependencies are in a
non-standard Homebrew location, e.g. `/opt/homebrew`, instead of `/usr/local`.
This allows for multiple build scenarios, though often requires a more
meticulous CMake configuration of QGIS.
You can programmatically find the prefix with `brew --prefix`. In formulae
code, it is denoted with the HOMEBREW_PREFIX environment variable.
### Homebrew formula install prefixes
By default, Homebrew installs to a versioned prefix in the 'Cellar', e.g.
`/usr/local/Cellar/gdal2/2.1.2/`. Using the versioned path for dependencies
_might_ lead to it being hardcoded into the linking phase of your builds. This
is an issue because the version can change often with formula changes that are
unrelated to the actual dependency's version, e.g. `2.1.2_1`. When defining
paths for CMake, always try the formula's 'opt prefix' first, e.g.
`/usr/local/opt/gdal2`, which is a symbolic link that _always_ points to the
latest versioned install prefix of the formula. This often avoids the issue,
though some CMake find modules _may_ resolve the symbolic link back to the
versioned install prefix.
### Install some basic formulae
Always read the 'Caveats' section output at the end of `brew info <formula>` or
`brew install <formula>`.
```sh
brew install bash-completion
brew install git
```
## Python Dependencies
### Select an interpreter
The first important decision to make is regarding whether to use Homebrew's or
another Python 3.x. Currently macOS does not ship with Python 3, and QGIS 3
should be built against Python 3.x.
> Note: The more Homebrew formulae you install from bottles, the higher
likelihood you will end up running into a formulae that requires installing
Homebrew's Python 3, since bottles are always built against that Python.
If using Homebrew Python 3.x, install with:
```sh
# review options
brew info python3
brew install python3 [--with-option ...]
```
Regardless of which Python interpreter you use, always ensure it is the first
found `python3` on PATH in your Terminal session where you run `brew` commands,
i.e. `which python3` points to the correct Python 3 you wish to use when
building formulae.
### Install required Python packages
Use [pip3](https://pypi.python.org/pypi/pip/) that was installed against the
same Python 3 you are using when installing formulae. You can also tap
[homebrew/python](https://github.com/Homebrew/homebrew-python) for some more
complex package installs.
Reference `pip3 --help` for info on usage (usually just `pip install <package>`).
* [future](https://pypi.python.org/pypi/future)
* [numpy](https://pypi.python.org/pypi/numpy)
* [psycopg2](https://pypi.python.org/pypi/psycopg2)
* [matplotlib](https://pypi.python.org/pypi/matplotlib)
* [pyparsing](https://pypi.python.org/pypi/pyparsing)
* [requests](https://pypi.python.org/pypi/requests)
* [mock](https://pypi.python.org/pypi/mock)
* [pyyaml](https://pypi.python.org/pypi/PyYAML)
* [nose2](https://pypi.python.org/pypi/nose2)
Other Python packages **automatically installed** by Homebrew from QGIS dependencies:
* [sip](https://github.com/Homebrew/homebrew-core/blob/master/Formula/sip.rb)
* [PyQt5](https://github.com/Homebrew/homebrew-core/blob/master/Formula/pyqt5.rb)
* [QScintilla2](https://github.com/Homebrew/homebrew-core/blob/master/Formula/qscintilla2.rb)
* [pyspatialite](https://github.com/OSGeo/homebrew-osgeo4mac/blob/master/Formula/pyspatialite.rb) (deprecated)
* [`osgeo.gdal` and `osgeo.ogr`, etc.](https://github.com/OSGeo/homebrew-osgeo4mac/blob/master/Formula/gdal2-python.rb)
## Install Build and Linked Library Dependencies
### Install dependencies from QGIS formulae
```sh
# fix for running out of file handles issue during maing install step
ulimit -n 1024
# add this tap
brew tap osgeo/osgeo4mac
# review options
brew info osgeo/osgeo4mac/osgeo-qgis
# see what dependencies will be included
brew deps --tree osgeo/osgeo4mac/osgeo-qgis [--with[out]-some-option ...]
# install dependencies, but not QGIS
brew install osgeo/osgeo4mac/osgeo-qgis --only-dependencies [--with[out]-some-option ...]
brew install osgeo/osgeo4mac/osgeo-pyqt-webkit
brew install osgeo/osgeo4mac/osgeo-qscintilla2
brew install exiv2
brew install osgeo/osgeo4mac/osgeo-qtkeychain
brew install qwtpolar
brew install qwt
brew install spatialindex
```
You do not have to actually do `brew install osgeo/osgeo4mac/osgeo-qgis` unless
you also want that version installed. If you do have QGIS 3 formulae installed,
and are planning on _installing_ your development build (not just running from
the build directory), you should unlink the formulae installs, e.g.:
```sh
brew unlink osgeo/osgeo4mac/osgeo-qgis
```
This will ensure the `qgis.core`, etc. Python modules of the formula(e)
installs are not overwritten by the development build upon `make install`. All
`qgis-xx` formulae QGIS applications will run just fine from their Cellar keg
install directory. _Careful_, though, as multiple QGIS installs will probably
all share the same application preference files; so, don't run them
concurrently.
### Optional External Dependencies
The [Processing
framework](http://docs.qgis.org/testing/en/docs/user_manual/processing/index.html)
of QGIS can leverage many external geospatial applications and utilities, which
_do not_ need to be built as dependencies prior to building QGIS:
* [`grass7`](https://github.com/OSGeo/homebrew-osgeo4mac/blob/master/Formula/grass7.rb) (`--with-grass` option) - [GRASS 7](http://grass.osgeo.org), which is also used by the GRASS core plugin in QGIS: ``brew install osgeo/osgeo4mac/grass7``
* [`orfeo5`](https://github.com/OSGeo/homebrew-osgeo4mac/blob/master/Formula/orfeo5.rb) (`--with-orfeo5` option) - [Orfeo Toolbox](http://orfeo-toolbox.org/otb/) <---- not working
* [`r`](http://www.r-project.org/) (`--with-r` option) - [R Project](http://www.r-project.org/): ``brew install r``
* [`saga-gis`](https://github.com/OSGeo/homebrew-osgeo4mac/blob/master/Formula/saga-gis.rb) (`--with-saga-gis` option) - [System for Automated Geoscientific Analyses](http://www.saga-gis.org): ``brew install saga-gis``
* [`taudem`](https://github.com/OSGeo/homebrew-osgeo4mac/blob/master/Formula/taudem.rb) - [Terrain Analysis Using Digital Elevation Models](http://hydrology.usu.edu/taudem/taudem5/index.html).: ``brew install taudem``
The `gpsbabel` formula can be installed as a dependency, though you may have to
define the path to its binary when using QGIS's [GPS
Tools](http://docs.qgis.org/testing/en/docs/user_manual/working_with_gps/plugins_gps.html).
> Note: if you install Processing external utilities _after_ installing a QGIS
> formula or building your own QGIS, you may need to configure the individual
> utility's paths in Processing's options dialog.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment