Last active
September 19, 2016 14:04
-
-
Save ikouchiha47/ba89b2a06c35069b9003df0d94e06725 to your computer and use it in GitHub Desktop.
ruby csv parser memory test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class Config | |
class << self | |
def get | |
env = ENV["RUBY_ENV"] || "developement" | |
env = env.downcase | |
require_relative "./environments/" + downcase | |
end | |
end | |
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require_relative './helpers' | |
require 'csv' | |
print_memory_usage do | |
print_time_spent do | |
csv = CSV.open('./data/test_data.csv') | |
csv.drop(13).each do |row| | |
p row | |
end | |
CSV.foreach('./data/test_data.csv') do |row| | |
p row | |
end | |
end | |
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require 'benchmark' | |
def print_memory_usage | |
memory_before = `ps -o rss= -p #{Process.pid}`.to_i | |
yield | |
memory_after = `ps -o rss= -p #{Process.pid}`.to_i | |
puts "Memory: #{((memory_after - memory_before) / 1024.0).round(2)} MB" | |
end | |
def print_time_spent | |
time = Benchmark.realtime do | |
yield | |
end | |
puts "Time: #{time.round(2)}" | |
end |
We can make this file beautiful and searchable if this error is corrected: It looks like row 13 should actually have 2 columns, instead of 12 in line 12.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Report,Reviews | |
App ID,com.gojek.app | |
App,GO-JEK | |
Version,All Versions | |
Market,Google Play | |
Language,All Languages | |
Total Reviews,6603 | |
Average Rating,3.5 | |
Search Term, | |
Start Date,08/13/2016 | |
End Date,09/12/2016 | |
Date Generated,09/14/2016 00:50:51 PDT | |
Platform,Language,Date,App ID,App Name,Publisher ID,Publisher Name,User,Version,Rating,Title,Review | |
Google Play,English,09/12/2016,com.gojek.app,GO-JEK,20200000891258,PT GO-JEK INDONESIA,,,1,," Saldo saya hilang, padahal saya sebelumnya top up 2x dan belum terpakai " | |
Google Play,English,09/12/2016,com.gojek.app,GO-JEK,20200000891258,PT GO-JEK INDONESIA,,,4,Rew, Dwwww | |
Google Play,English,09/12/2016,com.gojek.app,GO-JEK,20200000891258,PT GO-JEK INDONESIA,,,2,A little bit disappointed," Cari driver lsg ""booking in progress"", tp ditunggu 15menit gk dtg. Ditelp gk diangkat. Cancel booking trus find lg, ditlp gk diangkat jg pdhl beda driver. Cukup. " | |
Google Play,English,09/12/2016,com.gojek.app,GO-JEK,20200000891258,PT GO-JEK INDONESIA,,,5,Smart, Very Easy take something |
matthewd
commented
Sep 19, 2016
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment