Heroku CI compatible DiscoverRunner class for Django 1.10+ that does not require createdb or dropdb permissions.

heroku_test_runner.py

import os

from django.test.runner import DiscoverRunner

"""
WARNING:  WHEN USED INCORRECTLY THIS TEST RUNNER WILL DROP ALL TABLES IN YOUR PRODUCTION 
DATABASE!!!

Heroku does not give users createdb/dropdb permissions, therefore Heroku CI cannot run tests for django.
In order to fix this, use this test runner instead which attempts to minimally override the 
default test runner by a)  forcing keepdb=True to stop database create/drop, and b) by dropping all
tables after a test run and resetting the database to its initial blank state.

Usage:

1. In your django test settings file add the following two lines to ensure that the test 
database name is the same as the Heroku provided database name.

DATABASES['default'] = env.db('DATABASE_URL')  # or whatever you use to load the Heroku database settings
DATABASES['default']['TEST'] = {'NAME': DATABASES['default']['NAME']}

2. Set the testrunner to this file
TEST_RUNNER = 'your_modules.HerokuDiscoverRunner'

3. Set an environment variable on heroku CI of IS_HEROKU_TEST=1 to enable this runner, otherwise
the runner will exit as a safety measure.
"""

class HerokuDiscoverRunner(DiscoverRunner):
    def setup_databases(self, **kwargs):
        if not os.environ.get('IS_HEROKU_TEST'):
            raise ValueError(
                "The IS_HEROKU_TEST env variable must be set to enable this.  WARNING:  "
                "This test runner will wipe all tables in the database it targets!")
        self.keepdb = True
        return super(HerokuDiscoverRunner, self).setup_databases(**kwargs)

    def _wipe_tables(self, connection):
        with connection.cursor() as cursor:
            cursor.execute(
                """
                    DROP SCHEMA public CASCADE;
                    CREATE SCHEMA public;
                    GRANT ALL ON SCHEMA public TO postgres;
                    GRANT ALL ON SCHEMA public TO public;
                    COMMENT ON SCHEMA public IS 'standard public schema';
                """
            )

    def teardown_databases(self, old_config, **kwargs):
        self.keepdb = True
        for connection, old_name, destroy in old_config:
            if destroy:
                self._wipe_tables(connection)
        super(HerokuDiscoverRunner, self).teardown_databases(old_config, **kwargs)

Hey dude,
This was really helpful. Just FYI, I found another way to handle without a custom test runner in case adding outside code isn't an option

I added this to app.json. The main point is the "--keepdb" flag; it stops from creating the DB as long as database['default']['TEST'] has database information added.

  "environments":{
    "test":{
      "addons": [
          "heroku-postgresql:hobby-dev"
      ],
      "scripts":{
        "test-setup": "python app/manage.py collectstatic --noinput",
        "test": "python app/manage.py test --keepdb"
      }
    }
  },

I added a second config file that would be run in HerokuCI. (DATABASE_URL is a disposable DB so it is OK to use it for both)

from settings import *
db_from_env = dj_database_url.config(conn_max_age=500)
DATABASES['default'].update(db_from_env)
DATABASES['default']['TEST'].update(db_from_env)

Hope it helps out.

Heroku CI has an immutable environment variable CI which is true only on CI platforms.
I'm trying to determine from heroku whether this will be reliably true only for CI in the longer term (i.e. might be used for other purposes like review apps in future). If so, we could use it to either undertake a check, or as a more reliable alternative to IS_HEROKU_CI (avoiding the need for user modification). I'll get back to this thread if I hear back. cleaning this up now, it seems OK to use the CI variable.

I ended up merging both of the above.

For newbies like myself here's a detailed step-by-step:

Set up app.json

In app.json I've taken @jmahmood's attachment of the postgres addon, creating a new clean test database which is accessible in the DATABASE_URL environment variable (notice I didn't use --keepdb, as it's overridden anyhow by the test runner):

  "environments": {
    "test": {
      "scripts": {
        "test": "python manage.py test"
      },
      "addons":[
        "heroku-postgresql:hobby-dev",
      ]
    }
  }

Visit HerokuCI settings page

In the Heroku CI Settings page, I have:
DJANGO_SETTINGS_MODULE=config.settings.heroku_ci

Modify your app settings

In my config.settings.heroku_ci.py file, following @jmahmood with an extra few notes:

"""
Test settings for deployment on Heroku CI
"""
# Inherit from your regular test settings, here the settings files are arranged like those generated from the django-cookiecutter package
from .test import *

# Get the CI environment variable and die if it isn't set; we don't want to risk running this on a production machine as
# all tables will get dropped
CI = env.bool('CI', default=False) # note this uses django-environ
if not CI:
    raise ValueError('CI environment variable not set to true - avoiding start with this test configuration as a safety precaution')

# Register the heroku CI runner, which wipes tables instead of dropping the whole database
# (Heroku users don't have database drop permission)
TEST_RUNNER='myapp.heroku_test_runner.HerokuDiscoverRunner'

# Register the test database as being the same as the default (already set in the settings we inherited from). We've used use a heroku addon to provision a new, clean, database into the CI environment, so this should be OK here but DEFINITELY SHOULD NOT BE USED IN PRODUCTION
DATABASES['default']['TEST'] = env.db('DATABASE_URL')  # note this uses django-environ

Create the test runner file in your_app/heroku_test_runner.py

And finally in my app/heroku_test_runner.py I use the original gist from @cordery, modified slightly to use Heroku's CI variable:

import os

from django.test.runner import DiscoverRunner


class HerokuDiscoverRunner(DiscoverRunner):
    """

    WARNING:  WHEN USED INCORRECTLY THIS TEST RUNNER WILL DROP ALL TABLES IN YOUR PRODUCTION DATABASE

    Heroku does not give users createdb/dropdb permissions, therefore Heroku CI cannot run tests for django.
    In order to fix this, use this test runner instead, which attempts to minimally override the
    default test runner by a) forcing keepdb=True to stop database create/drop, and b) by dropping all
    tables after a test run and resetting the database to its initial blank state.

    """
    def setup_databases(self, **kwargs):
        if not os.environ.get('CI'):
            raise ValueError(
                "The CI env variable must be set to enable this.  WARNING:  "
                "This test runner will wipe all tables in the database it targets!")
        self.keepdb = True
        return super(HerokuDiscoverRunner, self).setup_databases(**kwargs)

    def _wipe_tables(self, connection):
        with connection.cursor() as cursor:
            cursor.execute(
                """
                    DROP SCHEMA public CASCADE;
                    CREATE SCHEMA public;
                    GRANT ALL ON SCHEMA public TO postgres;
                    GRANT ALL ON SCHEMA public TO public;
                    COMMENT ON SCHEMA public IS 'standard public schema';
                """
            )

    def teardown_databases(self, old_config, **kwargs):
        self.keepdb = True
        for connection, old_name, destroy in old_config:
            if destroy:
                self._wipe_tables(connection)
        super(HerokuDiscoverRunner, self).teardown_databases(old_config, **kwargs)

For anyone who finds this, Heroku now allows in-dyno databases that allow creation of databases.

I'm on Django 2.0, Python 3.6, trying to get my tests to pass in Heroku CI. I tried in-dyno databases, I tried the HerokuTestRunner, I tried --keepdb. For the life of me, I just could not get around

Got an error creating the test database: permission denied to create database

Here's my hack. First, heroku_ci_test_settings.py

# DANGER DANGER DANGER: If you use these settings and run tests, they will
# install vacuous testing data on your database.

# Note that the default ./manage.py test behavior creates a fresh database
# to test on. But the whole point of Heroku is to magically create all the
# stuff you need. It's totally appropriate that THEY make the testing database
# and we just use it. Simple. Easy. But the assumption that your web framework
# will administer your testing databases is pretty well baked into Django,
# hence this kind of hack.

# See https://gist.github.com/cordery/d52d9ba44541fabaf4b012f4e62d675b
# for more discussion.

# The environments.test section in app.json is really important here.

from django.test.runner import DiscoverRunner
from <my app>.settings import *

TEST_RUNNER = '<my app>.heroku_ci_test_settings.ExistingDBDiscoverRunner'

class ExistingDBDiscoverRunner(DiscoverRunner):
  def setup_databases(self, **kwargs):
    pass

  def teardown_databases(self, old_config, **kwargs):
    pass

Then this is in app.json

"environments": {
  "test": {
    "scripts": {
      "test-setup": "python manage.py migrate",
      "test": "python manage.py test --settings=<my app>.heroku_ci_test_settings"
    }
  }
}

For anyone who finds this, Heroku now allows in-dyno databases that allow creation of databases.

Yes! This fixed it for me. I didn't need to change anything other than have this in my app.json to use the in-dyno db:

"environments": {
    "test": {
      "addons": ["heroku-postgresql:in-dyno"],
      "scripts": {
        "test": "python manage.py test"
      }
    }
  }