matthewhegarty

Django Import Export

Bulk import testing

This scripts outlines the steps used to profile bulk loading. The bulk_import.py script is used to profile run duration and memory during bulk load testing.

Pre-requisites

• Docker

matthewhegarty

Performance improvements for bulk import

I implemented for my own use case - not fully tested - use at your own risk

This is how I improved performance of django-import-export when importing a large set of new rows.

• Thinkpad T470 i5 processor (Ubuntu 18.04)
• 20,000 new rows to be inserted
• total import duration 5.4 seconds

matthewhegarty

" Vim color file
" Maintainer:  Matthew Hegarty  <[email protected]>
" Last Change: 03/10/2002

" Dark color scheme

set background=dark
hi clear
if exists("syntax_on")
    syntax reset

matthewhegarty

" Vim color file
" A dark colour scheme
"
" Maintainer: Matthew Hegarty <[email protected]>
" Last Change:  02/06/2011 

" Dark color scheme

set background=dark
hi clear

matthewhegarty

openapi: 3.0.1
info:
  title: Django Rest Framework Tutorial
  description: |
    This is the OpenAPI definition for the Django Rest Framework tutorial.
  version: 1.0.0
externalDocs:
  description: Find out more about DRF Tutorial.
  url: https://www.django-rest-framework.org/tutorial/1-serialization/
paths:

matthewhegarty

 INFO global: Vagrant version: 2.1.2
 INFO global: Ruby version: 2.4.4
 INFO global: RubyGems version: 2.6.14.1
 INFO global: VAGRANT_INSTALLER_VERSION="2"
 INFO global: VAGRANT_INSTALLER_EMBEDDED_DIR="/opt/vagrant/embedded"
 INFO global: VAGRANT_EXECUTABLE="/opt/vagrant/embedded/gems/2.1.2/gems/vagrant-2.1.2/bin/vagrant"
 INFO global: VAGRANT_INSTALLER_ENV="1"
 INFO global: VAGRANT_LOG="debug"
 WARN global: resolv replacement has not been enabled!
 INFO global: Plugins: