Traceback from a crash introduced by a recent Homebrew update when Python ctypes tries to load gdal

homebrew-gdal-ctypes.traceback

Traceback (most recent call last):
  File "/Users/cadams/.virtualenvs/wdl/bin/django-admin", line 11, in <module>
    sys.exit(execute_from_command_line())
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/core/management/__init__.py", line 328, in execute
    django.setup()
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/apps/config.py", line 86, in create
    module = import_module(entry)
  File "/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/reversion/__init__.py", line 11, in <module>
    from reversion.revisions import default_revision_manager, revision_context_manager, VersionAdapter
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/reversion/revisions.py", line 28, in <module>
    from reversion.models import Revision, Version, has_int_pk, pre_revision_commit, post_revision_commit
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/reversion/models.py", line 48, in <module>
    class Revision(models.Model):
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/models/base.py", line 139, in __new__
    new_class.add_to_class('_meta', Options(meta, **kwargs))
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/models/base.py", line 324, in add_to_class
    value.contribute_to_class(cls, name)
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/models/options.py", line 250, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/__init__.py", line 36, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/utils.py", line 241, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/utils.py", line 112, in load_backend
    return import_module('%s.base' % backend_name)
  File "/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/db/backends/mysql/base.py", line 5, in <module>
    from .features import DatabaseFeatures
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/db/backends/mysql/features.py", line 1, in <module>
    from django.contrib.gis.db.backends.base.features import BaseSpatialFeatures
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/db/backends/base/features.py", line 3, in <module>
    from django.contrib.gis.db.models import aggregates
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/db/models/__init__.py", line 7, in <module>
    from django.contrib.gis.geos import HAS_GEOS
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/geos/__init__.py", line 15, in <module>
    from .geometry import GEOSGeometry, wkt_regex, hex_regex
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/geos/geometry.py", line 10, in <module>
    from django.contrib.gis.gdal.error import SRSException
  File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/gdal/__init__.py", line 37, in <module>
    ctypes.CDLL('/usr/local/lib/libgdal.dylib')
  File "/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 362, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/lib/libgdal.dylib, 6): Symbol not found: _GEOSArea
  Referenced from: /usr/local/opt/libspatialite/lib/libspatialite.7.dylib
  Expected in: flat namespace
 in /usr/local/opt/libspatialite/lib/libspatialite.7.dylib

There's some sort of load-time dependency here as this can be “fixed” by putting something like this early in the load process – such as Django's settings module:

# MAGIC HACK: without this, a recent Homebrew update broke GDAL!
GDAL_LIBRARY_PATH = "/usr/local/lib/libgdal.dylib"
import ctypes
ctypes.CDLL(GDAL_LIBRARY_PATH)

Done later, such as by the call in the GeoDjango initialization, import ctypes; ctypes.CDLL('/usr/local/lib/libgdal.dylib') will fail with the same error as above.

@acdha Where should I put that code above?

@PaoloC68 You can add to the top of you settings.py

Is there a better solution than this? I'm not sure my production environment will have libgdal in that path...

See Homebrew/homebrew-core#5161 for a tracking ticket. It's not clear exactly where the problem is.

@acdha Fix worked .Thanks

Just an FYI I ran into this same issue and removing an old template_postgis database from an old dump/restore from older GeoDjangos apparently fixed the issue for me.

Ran into this same issue today and did some digging. For me, the issue was in an older Django app (1.8.12). It appears that the problem arises because of the way that HAS_GEOS is checked in that version of Django. There, in __init__.py in django.contrib.gis.models has the following code to check if GEOS is installed:

from django.contrib.gis.geos import HAS_GEOS

if not HAS_GEOS:
    raise ImproperlyConfigured(
        "GEOS is required and has not been detected. Are you sure it is installed? "
        "See also https://docs.djangoproject.com/en/%s/ref/contrib/gis/install/geolibs/" % get_docs_version())

This import causes ctypes to load the geos_c dynamic library. A short while later, django.contrib.gis.geos.geometry attempts to import something from django.contrib.gis.gdal. That causes ctypes to load the libgdal dynamic library. Unfortunately, it appears that if geos is loaded before gdal is loaded, then gdal fails to load with the given error.

Starting in In Django 1.9 this conditional check was moved and imports were reshuffled in such a way as to fix this problem. I have been unable to reproduce it in a version of Django after 1.9a1.

If you (like me) are forced to keep an older Django, then ensuring that gdal is loaded by ctypes at any point before geos is loaded is sufficient to avoid this error. Adding the block above in settings.py ensures that it is executed before any database setup occurs, which then ensures that gdal is loaded before geos and solves the problem.