Last active
November 18, 2024 23:29
-
-
Save mperham/42307b8b135cd546ed68550e9af8a631 to your computer and use it in GitHub Desktop.
Benchmarking sidekiq and solid_queue
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## Setup | |
| MacBookPro, M1 Pro w/ 32GB RAM, Sequoia 15.0.1 | |
| Ruby 3.3.5 | |
| Sidekiq 7.3.5 | |
| Solid Queue 1.0.2 | |
| Rails 8.0.0 | |
| ## Results | |
| To process 500,000 no-op jobs: | |
| | Job setup | Time | | |
| |----------------|------| | |
| | Sidekiq/native | 19.0 | | |
| | Sidekiq/AJ | 25.6 | | |
| | Solid Queue/AJ | 293 | | |
| Solid Queue is much slower due to database overhead -- Redis queues are O(1) and ACID semantics aren't free! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| % AJ=1 LATENCY=0 THREADS=5 bin/sidekiqload | |
| ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [arm64-darwin23] | |
| THREADS: 5, LATENCY: 0, AJ: 1, PROFILE: nil | |
| Using ActiveJob 7.2.1 | |
| 2024-11-18T22:41:09.160Z pid=36225 tid=pm9 ERROR: Setup RSS: 47824 | |
| 2024-11-18T22:41:17.455Z pid=36225 tid=pm9 WARN: Created 500000 jobs in 8.29326 sec | |
| 2024-11-18T22:41:17.455Z pid=36225 tid=pm9 WARN: Starting load | |
| 2024-11-18T22:41:17.455Z pid=36225 tid=pm9 ERROR: Simulating 0ms of latency between Sidekiq and redis | |
| 2024-11-18T22:41:53.946Z pid=36225 tid=pax ERROR: Done, 500000 jobs in 36.490661 sec, 13702 jobs/sec | |
| 2024-11-18T22:41:53.951Z pid=36225 tid=pax ERROR: Ending RSS: 62208 | |
| 2024-11-18T22:41:53.951Z pid=36225 tid=pax ERROR: Now here's the latency for three jobs | |
| 0.0005128383636474609 | |
| 0.00025200843811035156 | |
| 0.0002219676971435547 | |
| % AJ=1 LATENCY=0 THREADS=5 bin/sidekiqload | |
| ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [arm64-darwin23] | |
| THREADS: 5, LATENCY: 0, AJ: 1, PROFILE: nil | |
| Using ActiveJob 8.0.0 | |
| 2024-11-18T22:42:40.488Z pid=36302 tid=pli ERROR: Setup RSS: 49696 | |
| 2024-11-18T22:42:48.843Z pid=36302 tid=pli WARN: Created 500000 jobs in 8.353052 sec | |
| 2024-11-18T22:42:48.843Z pid=36302 tid=pli WARN: Starting load | |
| 2024-11-18T22:42:48.843Z pid=36302 tid=pli ERROR: Simulating 0ms of latency between Sidekiq and redis | |
| 2024-11-18T22:43:25.738Z pid=36302 tid=pam ERROR: Done, 500000 jobs in 36.894953 sec, 13551 jobs/sec | |
| 2024-11-18T22:43:25.748Z pid=36302 tid=pam ERROR: Ending RSS: 65296 | |
| 2024-11-18T22:43:25.748Z pid=36302 tid=pam ERROR: Now here's the latency for three jobs | |
| 0.0009357929229736328 | |
| 0.0004470348358154297 | |
| 0.00043392181396484375 | |
| % LATENCY=0 THREADS=5 bin/sidekiqload | |
| ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [arm64-darwin23] | |
| THREADS: 5, LATENCY: 0, AJ: nil, PROFILE: nil | |
| 2024-11-18T22:44:13.357Z pid=36355 tid=ppb ERROR: Setup RSS: 40464 | |
| 2024-11-18T22:44:16.463Z pid=36355 tid=ppb WARN: Created 500000 jobs in 3.042817 sec | |
| 2024-11-18T22:44:16.463Z pid=36355 tid=ppb WARN: Starting load | |
| 2024-11-18T22:44:16.463Z pid=36355 tid=ppb ERROR: Simulating 0ms of latency between Sidekiq and redis | |
| 2024-11-18T22:44:20.982Z pid=36355 tid=pub class=LoadWorker jid=ff06d57f1b8cdbb8ddf7967f WARN: 2024-11-18 14:44:20 -0800 Done 100000 | |
| 2024-11-18T22:44:25.460Z pid=36355 tid=pwj class=LoadWorker jid=5add6479e82a02f002490ca7 WARN: 2024-11-18 14:44:25 -0800 Done 200000 | |
| 2024-11-18T22:44:29.918Z pid=36355 tid=pub class=LoadWorker jid=80737f88b693967bfd4273fc WARN: 2024-11-18 14:44:29 -0800 Done 300000 | |
| 2024-11-18T22:44:34.364Z pid=36355 tid=pub class=LoadWorker jid=7a49402f6b241ff7b66d3045 WARN: 2024-11-18 14:44:34 -0800 Done 400000 | |
| 2024-11-18T22:44:38.804Z pid=36355 tid=pub class=LoadWorker jid=b895226b0c41f15d3a18683b WARN: 2024-11-18 14:44:38 -0800 Done 500000 | |
| 2024-11-18T22:44:38.805Z pid=36355 tid=px3 ERROR: Done, 500000 jobs in 22.342346 sec, 22379 jobs/sec | |
| 2024-11-18T22:44:38.809Z pid=36355 tid=px3 ERROR: Ending RSS: 46176 | |
| 2024-11-18T22:44:38.809Z pid=36355 tid=px3 ERROR: Now here's the latency for three jobs | |
| 0.0003459453582763672 | |
| 0.0001957416534423828 | |
| 0.00016999244689941406 | |
| With YJIT activated, Sidekiq 7.3 vs Active Job 8.0 using Redis | |
| ############################################### | |
| % RUBYOPT=--yjit LATENCY=0 THREADS=5 bin/sidekiqload | |
| ruby 3.3.5 (2024-09-03 revision ef084cc8f4) +YJIT [arm64-darwin23] | |
| THREADS: 5, LATENCY: 0, AJ: nil, PROFILE: nil | |
| 2024-11-18T23:11:19.510Z pid=36863 tid=q1v ERROR: Setup RSS: 42064 | |
| 2024-11-18T23:11:22.305Z pid=36863 tid=q1v WARN: Created 500000 jobs in 2.79337 sec | |
| 2024-11-18T23:11:22.305Z pid=36863 tid=q1v WARN: Starting load | |
| 2024-11-18T23:11:22.305Z pid=36863 tid=q1v ERROR: Simulating 0ms of latency between Sidekiq and redis | |
| 2024-11-18T23:11:26.118Z pid=36863 tid=pwb class=LoadWorker jid=13901a319c429a63461bd0d1 WARN: 2024-11-18 15:11:26 -0800 Done 100000 | |
| 2024-11-18T23:11:29.905Z pid=36863 tid=pv7 class=LoadWorker jid=423eb22c392bc12307cf545b WARN: 2024-11-18 15:11:29 -0800 Done 200000 | |
| 2024-11-18T23:11:33.698Z pid=36863 tid=pun class=LoadWorker jid=f1157c75681663a903865739 WARN: 2024-11-18 15:11:33 -0800 Done 300000 | |
| 2024-11-18T23:11:37.473Z pid=36863 tid=pv7 class=LoadWorker jid=28bcd9d395535f3b931c1e1b WARN: 2024-11-18 15:11:37 -0800 Done 400000 | |
| 2024-11-18T23:11:41.268Z pid=36863 tid=pwb class=LoadWorker jid=92a8aa4f74e7b0f604147810 WARN: 2024-11-18 15:11:41 -0800 Done 500000 | |
| 2024-11-18T23:11:41.374Z pid=36863 tid=pu3 ERROR: Done, 500000 jobs in 19.068405 sec, 26221 jobs/sec | |
| 2024-11-18T23:11:41.380Z pid=36863 tid=pu3 ERROR: Ending RSS: 51008 | |
| 2024-11-18T23:11:41.380Z pid=36863 tid=pu3 ERROR: Now here's the latency for three jobs | |
| 0.000797271728515625 | |
| 0.00020813941955566406 | |
| 0.0001399517059326172 | |
| % RUBYOPT=--yjit LATENCY=0 THREADS=5 AJ=1 bin/sidekiqload | |
| ruby 3.3.5 (2024-09-03 revision ef084cc8f4) +YJIT [arm64-darwin23] | |
| THREADS: 5, LATENCY: 0, AJ: 1, PROFILE: nil | |
| Using ActiveJob 8.0.0 | |
| 2024-11-18T23:11:52.768Z pid=36891 tid=uv7 ERROR: Setup RSS: 51888 | |
| 2024-11-18T23:11:59.704Z pid=36891 tid=uv7 WARN: Created 500000 jobs in 6.933166 sec | |
| 2024-11-18T23:11:59.704Z pid=36891 tid=uv7 WARN: Starting load | |
| 2024-11-18T23:11:59.704Z pid=36891 tid=uv7 ERROR: Simulating 0ms of latency between Sidekiq and redis | |
| 2024-11-18T23:12:25.323Z pid=36891 tid=v6j ERROR: Done, 500000 jobs in 25.618275 sec, 19517 jobs/sec | |
| 2024-11-18T23:12:25.329Z pid=36891 tid=v6j ERROR: Ending RSS: 69872 | |
| 2024-11-18T23:12:25.329Z pid=36891 tid=v6j ERROR: Now here's the latency for three jobs | |
| 0.0020117759704589844 | |
| 0.0008718967437744141 | |
| 0.00018095970153808594 | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class BenchJob < ApplicationJob | |
| queue_as :default | |
| $count = 0 | |
| $lock = Mutex.new | |
| def perform(string, number, options) | |
| $lock.synchronize do | |
| $count += 1 | |
| if $count % 100_000 == 0 | |
| logger.warn("#{Time.now} Done #{$count}") | |
| end | |
| end | |
| end | |
| end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SolidQueue.on_start do | |
| puts "#{Time.now} Pushing 500,000 jobs" | |
| ActiveJob.perform_all_later(100_000.times.map { BenchJob.new("string", 123, mike: "bob") }) | |
| ActiveJob.perform_all_later(100_000.times.map { BenchJob.new("string", 123, mike: "bob") }) | |
| ActiveJob.perform_all_later(100_000.times.map { BenchJob.new("string", 123, mike: "bob") }) | |
| ActiveJob.perform_all_later(100_000.times.map { BenchJob.new("string", 123, mike: "bob") }) | |
| ActiveJob.perform_all_later(100_000.times.map { BenchJob.new("string", 123, mike: "bob") }) | |
| puts "#{Time.now} Done pushing" | |
| end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| % LATENCY=0 THREADS=5 bin/sidekiqload | |
| ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [arm64-darwin23] | |
| THREADS: 5, LATENCY: 0, AJ: nil, PROFILE: nil | |
| 2024-11-18T19:51:34.807Z pid=34506 tid=rwe ERROR: Setup RSS: 43056 | |
| 2024-11-18T19:51:37.943Z pid=34506 tid=rwe WARN: Created 500000 jobs in 3.133927 sec | |
| 2024-11-18T19:51:37.943Z pid=34506 tid=rwe WARN: Starting load | |
| 2024-11-18T19:51:37.943Z pid=34506 tid=rwe ERROR: Simulating 0ms of latency between Sidekiq and redis | |
| 2024-11-18T19:51:42.559Z pid=34506 tid=ram class=LoadWorker jid=744b9d7fb1d02ac0d7ec671c WARN: 2024-11-18 11:51:42 -0800 Done 100000 | |
| 2024-11-18T19:51:47.148Z pid=34506 tid=rcm class=LoadWorker jid=c2e53fc7bbf0036349909d10 WARN: 2024-11-18 11:51:47 -0800 Done 200000 | |
| 2024-11-18T19:51:51.750Z pid=34506 tid=rc2 class=LoadWorker jid=2d19bdd5421ec2541c53489c WARN: 2024-11-18 11:51:51 -0800 Done 300000 | |
| 2024-11-18T19:51:56.338Z pid=34506 tid=rc2 class=LoadWorker jid=d0d3285aca286c0f05f44f4f WARN: 2024-11-18 11:51:56 -0800 Done 400000 | |
| 2024-11-18T19:52:00.922Z pid=34506 tid=ram class=LoadWorker jid=26259e463212bf76f057e7ce WARN: 2024-11-18 11:52:00 -0800 Done 500000 | |
| 2024-11-18T19:52:01.058Z pid=34506 tid=rly ERROR: Done, 500000 jobs in 23.114533 sec, 21631 jobs/sec |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| % RAILS_LOG_LEVEL=warn RAILS_ENV=production bin/jobs | |
| SolidQueue-1.0.2 Fail claimed jobs (5.5ms) job_ids: [], process_ids: [] | |
| 2024-11-18 11:40:29 -0800 Pushing 500,000 jobs | |
| 2024-11-18 11:41:15 -0800 Done pushing | |
| [ActiveJob] [BenchJob] [2630c000-55c2-445c-8534-63d02249a284] 2024-11-18 11:42:15 -0800 Done 100000 | |
| [ActiveJob] [BenchJob] [425ff998-32af-4dc8-87b7-44cd2e135c42] 2024-11-18 11:43:14 -0800 Done 200000 | |
| [ActiveJob] [BenchJob] [f4b84b6e-bf09-4054-8e2b-5031a18b4a05] 2024-11-18 11:44:12 -0800 Done 300000 | |
| [ActiveJob] [BenchJob] [3d091c52-6ac2-47a5-b2a5-1ca2ecb41dc1] 2024-11-18 11:45:15 -0800 Done 400000 | |
| [ActiveJob] [BenchJob] [50da92fd-2341-41e1-ab20-b596003859e8] 2024-11-18 11:46:16 -0800 Done 500000 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Both solid_queue and sidekiq are using 5 threads. Sidekiq is using the native Sidekiq::Job API, not Active Job.