Last active
December 11, 2016 13:00
-
-
Save paneq/a643b9a3cc694ba3eb6e to your computer and use it in GitHub Desktop.
cost of using exceptions for control flow compared to one SQL statement (ruby 2.1.4, rails 4.1.7, sqlite) for rails-refactoring.com . Development mode executed in rails console.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require 'benchmark' | |
ActiveRecord::Base.logger = nil | |
Benchmark.bmbm do |bench| | |
bench.report("SQL query") do | |
1000.times { Whatever.count } | |
end | |
bench.report("exception hit") do | |
1000.times do | |
begin | |
raise StandardError.new | |
rescue | |
end | |
end | |
end | |
bench.report("exception miss") do | |
1000.times do | |
begin | |
raise StandardError.new if false | |
rescue | |
end | |
end | |
end | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Rehearsal -------------------------------------------------- | |
SQL query 0.180000 0.010000 0.190000 ( 0.199253) | |
exception hit 0.050000 0.000000 0.050000 ( 0.050654) | |
exception miss 0.000000 0.000000 0.000000 ( 0.000050) | |
----------------------------------------- total: 0.240000sec | |
user system total real | |
SQL query 0.180000 0.010000 0.190000 ( 0.179669) | |
exception hit 0.050000 0.000000 0.050000 ( 0.048447) | |
exception miss 0.000000 0.000000 0.000000 ( 0.000054) |
Just for kicks ;) I redid the above: ~two years later, on the same machine, with activerecord 5.0.0.1 and pg 0.19.0:
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
Warming up --------------------------------------
SQL query 353.000 i/100ms
exception hit 62.811k i/100ms
exception miss 297.570k i/100ms
Calculating -------------------------------------
SQL query 2.921k (±32.9%) i/s - 13.061k in 5.052108s
exception hit 831.142k (± 3.2%) i/s - 4.208M in 5.068312s
exception miss 10.467M (± 3.2%) i/s - 52.372M in 5.008549s
Comparison:
exception miss: 10466869.0 i/s
exception hit: 831142.0 i/s - 12.59x slower
SQL query: 2921.4 i/s - 3582.81x slower
ruby 2.4.0preview2 (2016-09-09 trunk 56129) [x86_64-linux]
Warming up --------------------------------------
SQL query 262.000 i/100ms
exception hit 62.278k i/100ms
exception miss 344.852k i/100ms
Calculating -------------------------------------
SQL query 3.306k (±24.1%) i/s - 15.458k in 5.080964s
exception hit 871.144k (± 0.9%) i/s - 4.359M in 5.004697s
exception miss 11.577M (± 1.1%) i/s - 57.935M in 5.005123s
Comparison:
exception miss: 11576670.6 i/s
exception hit: 871144.2 i/s - 13.29x slower
SQL query: 3306.0 i/s - 3501.78x slower
Meanwhile @andrzejkrzywda duly pointed out that comparing exception handling time with writes (rather than – especially easily cacheable – reads) would be much more telling, so here’s the comparison with Whatever.create(text: 'meh')
instead of Whatever.count
:
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
Warming up --------------------------------------
SQL query 18.000 i/100ms
exception hit 57.640k i/100ms
exception miss 269.546k i/100ms
Calculating -------------------------------------
SQL query 182.124 (±15.9%) i/s - 882.000 in 5.003193s
exception hit 808.613k (± 4.8%) i/s - 4.035M in 5.004163s
exception miss 10.129M (± 3.2%) i/s - 50.675M in 5.007747s
Comparison:
exception miss: 10129279.5 i/s
exception hit: 808613.1 i/s - 12.53x slower
SQL query: 182.1 i/s - 55617.43x slower
ruby 2.4.0preview2 (2016-09-09 trunk 56129) [x86_64-linux]
Warming up --------------------------------------
SQL query 20.000 i/100ms
exception hit 58.768k i/100ms
exception miss 344.645k i/100ms
Calculating -------------------------------------
SQL query 186.218 (±20.4%) i/s - 880.000 in 5.005586s
exception hit 818.088k (± 6.9%) i/s - 4.114M in 5.069060s
exception miss 11.347M (± 0.6%) i/s - 56.866M in 5.011838s
Comparison:
exception miss: 11346771.3 i/s
exception hit: 818087.5 i/s - 13.87x slower
SQL query: 186.2 i/s - 60932.79x slower
For me the take home is that when a (SQL) database is involved efforts to optimize the queries has a greater chance of improving app performance than futzing with flow control techniques. Not a surprise but it's nice to have some real numbers to point to. Thanks.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I’m not sure how much (a) the used database engine, (b) the type of database, (c) the size of the table, (d) disk caching, (e) database’s caching and (f) Active Record’s caching matters, but FWIW here is my script that runs against an on-disk PostgreSQL empty table with a single text column:
ruby -v bench.rb
runs: