First: Remove references in the code
remove app_to_remove from settings.INSTALLED_APPS
remove other references in urls.py or other places
Second: Clean the database
Create an empty migration for your django-project:
manage.py makemigrations your_django_project --empty