Skip to content

Instantly share code, notes, and snippets.

@kinow

kinow/gist:9e3760ddfa036b82df0f587323e17ee6

Created February 4, 2019 23:55
Show Gist options
  • Save kinow/9e3760ddfa036b82df0f587323e17ee6 to your computer and use it in GitHub Desktop.
Save kinow/9e3760ddfa036b82df0f587323e17ee6 to your computer and use it in GitHub Desktop.
Download ZIP
csv benchmark 1
Raw
gistfile1.txt
asv continuous -f 1.1 upstream/master HEAD -b ^io.csv
· Creating environments
· Discovering benchmarks
· Running 32 total benchmarks (2 commits * 1 environments * 16 benchmarks)
[ 0.00%] · For pandas commit 2e38d555 (round 1/2):
[ 0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt......................................................................
[ 0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 1.56%] ··· Running (io.csv.ReadCSVCategorical.time_convert_direct--)..............
[ 25.00%] ··· Running (io.csv.ToCSVDatetime.time_frame_date_formatting--).
[ 25.00%] · For pandas commit fe6fa7b7 <fix-issue-gh25099-csv-narep> (round 1/2):
[ 25.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
[ 25.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 26.56%] ··· Running (io.csv.ReadCSVCategorical.time_convert_direct--)............
[ 46.88%] ··· Running (io.csv.ReadUint64Integers.time_read_uint64_neg_values--)...
[ 50.00%] · For pandas commit fe6fa7b7 <fix-issue-gh25099-csv-narep> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 51.56%] ··· io.csv.ReadCSVCategorical.time_convert_direct 135±4ms
[ 53.12%] ··· io.csv.ReadCSVCategorical.time_convert_post 190±4ms
[ 54.69%] ··· io.csv.ReadCSVComment.time_comment 80.1±4ms
[ 56.25%] ··· io.csv.ReadCSVDInferDatetimeFormat.time_read_csv ok
[ 56.25%] ··· ======================= ============ ============ ============
-- format
----------------------- --------------------------------------
infer_datetime_format custom iso8601 ymd
======================= ============ ============ ============
True 16.3±0.4ms 6.07±0.3ms 6.54±0.2ms
False 243±6ms 5.03±0.2ms 4.80±0.2ms
======================= ============ ============ ============
[ 57.81%] ··· io.csv.ReadCSVFloatPrecision.time_read_csv ok
[ 57.81%] ··· ===== ============ ============ ================ ============ ============ ================
-- decimal / float_precision
----- -------------------------------------------------------------------------------------
sep . / None . / high . / round_trip _ / None _ / high _ / round_trip
===== ============ ============ ================ ============ ============ ================
, 4.64±0.1ms 4.56±0.2ms 5.67±0.1ms 5.25±0.2ms 4.95±0.1ms 5.04±0.2ms
; 5.13±0.4ms 4.60±0.3ms 5.97±0.5ms 5.03±0.3ms 5.32±0.4ms 4.80±0.1ms
===== ============ ============ ================ ============ ============ ================
[ 59.38%] ··· io.csv.ReadCSVFloatPrecision.time_read_csv_python_engine ok
[ 59.38%] ··· ===== ============ ============ ================ ============ ============ ================
-- decimal / float_precision
----- -------------------------------------------------------------------------------------
sep . / None . / high . / round_trip _ / None _ / high _ / round_trip
===== ============ ============ ================ ============ ============ ================
, 12.3±0.4ms 12.6±0.7ms 12.6±0.6ms 11.7±0.6ms 10.9±0.4ms 10.3±0.3ms
; 12.9±0.8ms 12.7±1ms 12.9±2ms 11.2±0.2ms 10.9±0.4ms 10.6±1ms
===== ============ ============ ================ ============ ============ ================
[ 60.94%] ··· io.csv.ReadCSVMemoryGrowth.mem_parser_chunks 0
[ 62.50%] ··· io.csv.ReadCSVParseDates.time_baseline 4.37±0.07ms
[ 64.06%] ··· io.csv.ReadCSVParseDates.time_multiple_date 5.28±0.4ms
[ 65.62%] ··· io.csv.ReadCSVSkipRows.time_skipprows ok
[ 65.62%] ··· ========== ============
skiprows
---------- ------------
None 47.5±2ms
10000 33.5±0.9ms
========== ============
[ 67.19%] ··· io.csv.ReadCSVThousands.time_thousands ok
[ 67.19%] ··· ===== ========== ==========
-- thousands
----- ---------------------
sep None ,
===== ========== ==========
, 41.8±1ms 46.7±2ms
| 41.8±2ms 45.4±2ms
===== ========== ==========
[ 68.75%] ··· io.csv.ReadUint64Integers.time_read_uint64 9.30±0.4ms
[ 70.31%] ··· io.csv.ReadUint64Integers.time_read_uint64_na_values 13.6±0.6ms
[ 71.88%] ··· io.csv.ReadUint64Integers.time_read_uint64_neg_values 13.2±0.6ms
[ 73.44%] ··· io.csv.ToCSV.time_frame ok
[ 73.44%] ··· ======= ==========
kind
------- ----------
wide 257±8ms
long 551±20ms
mixed 65.5±2ms
======= ==========
[ 75.00%] ··· io.csv.ToCSVDatetime.time_frame_date_formatting 37.6±0.7ms
[ 75.00%] · For pandas commit 2e38d555 (round 2/2):
[ 75.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
[ 75.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 76.56%] ··· io.csv.ReadCSVCategorical.time_convert_direct 119±1ms
[ 78.12%] ··· io.csv.ReadCSVCategorical.time_convert_post 171±5ms
[ 79.69%] ··· io.csv.ReadCSVComment.time_comment 75.0±2ms
[ 81.25%] ··· io.csv.ReadCSVDInferDatetimeFormat.time_read_csv ok
[ 81.25%] ··· ======================= ============ ============ ============
-- format
----------------------- --------------------------------------
infer_datetime_format custom iso8601 ymd
======================= ============ ============ ============
True 15.4±0.3ms 5.57±0.2ms 5.59±0.2ms
False 231±3ms 4.79±0.2ms 4.58±0.1ms
======================= ============ ============ ============
[ 82.81%] ··· io.csv.ReadCSVFloatPrecision.time_read_csv ok
[ 82.81%] ··· ===== ============= ============ ================ ============= ============ ================
-- decimal / float_precision
----- ---------------------------------------------------------------------------------------
sep . / None . / high . / round_trip _ / None _ / high _ / round_trip
===== ============= ============ ================ ============= ============ ================
, 4.40±0.07ms 4.71±0.2ms 5.51±0.1ms 4.84±0.09ms 5.18±0.3ms 5.13±0.2ms
; 4.39±0.1ms 4.48±0.2ms 5.80±0.3ms 4.90±0.04ms 5.02±0.2ms 5.17±0.5ms
===== ============= ============ ================ ============= ============ ================
[ 84.38%] ··· io.csv.ReadCSVFloatPrecision.time_read_csv_python_engine ok
[ 84.38%] ··· ===== ============ ============ ================ ============ ============ ================
-- decimal / float_precision
----- -------------------------------------------------------------------------------------
sep . / None . / high . / round_trip _ / None _ / high _ / round_trip
===== ============ ============ ================ ============ ============ ================
, 11.5±0.2ms 11.9±0.3ms 11.3±0.2ms 9.92±0.1ms 10.6±0.1ms 10.8±0.2ms
; 12.7±0.4ms 12.4±0.4ms 12.0±0.9ms 10.3±0.4ms 11.1±1ms 10.6±0.2ms
===== ============ ============ ================ ============ ============ ================
[ 85.94%] ··· io.csv.ReadCSVMemoryGrowth.mem_parser_chunks 0
[ 87.50%] ··· io.csv.ReadCSVParseDates.time_baseline 4.21±0.06ms
[ 89.06%] ··· io.csv.ReadCSVParseDates.time_multiple_date 4.97±0.2ms
[ 90.62%] ··· io.csv.ReadCSVSkipRows.time_skipprows ok
[ 90.62%] ··· ========== ==========
skiprows
---------- ----------
None 44.1±1ms
10000 33.9±2ms
========== ==========
[ 92.19%] ··· io.csv.ReadCSVThousands.time_thousands ok
[ 92.19%] ··· ===== ========== ==========
-- thousands
----- ---------------------
sep None ,
===== ========== ==========
, 42.6±1ms 42.9±2ms
| 41.4±1ms 44.6±2ms
===== ========== ==========
[ 93.75%] ··· io.csv.ReadUint64Integers.time_read_uint64 8.82±0.2ms
[ 95.31%] ··· io.csv.ReadUint64Integers.time_read_uint64_na_values 12.8±0.2ms
[ 96.88%] ··· io.csv.ReadUint64Integers.time_read_uint64_neg_values 12.9±0.5ms
[ 98.44%] ··· io.csv.ToCSV.time_frame ok
[ 98.44%] ··· ======= ============
kind
------- ------------
wide 251±4ms
long 524±6ms
mixed 63.8±0.7ms
======= ============
[100.00%] ··· io.csv.ToCSVDatetime.time_frame_date_formatting 37.9±0.7ms
before after ratio
[2e38d555] [fe6fa7b7]
<fix-issue-gh25099-csv-narep>
+ 9.92±0.1ms 11.7±0.6ms 1.18 io.csv.ReadCSVFloatPrecision.time_read_csv_python_engine(',', '_', None)
+ 119±1ms 135±4ms 1.14 io.csv.ReadCSVCategorical.time_convert_direct
+ 171±5ms 190±4ms 1.11 io.csv.ReadCSVCategorical.time_convert_post
SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment