Bill Dueber billdueber

I upgraded my ruby 1.8 to the latest patchlevel and all of a sudden ruby-marc was super-slow. I found the same thing on 1.9 and in JRuby, so I investigated.

There's a marc.bytes.to_a call inside the loop in Reader#decode. All the fix does is move it outside the loop so it only happens once.

You can see the patch in the "slowreadfix" branch at https://github.com/ruby-marc/ruby-marc/commit/beba83745ebe0848218496e967edd65d632fb01e

As you can see, the speedup is about a factor of five.

Test case is reading in a Marc21 file with about 18K records in it.

	require 'rubygems'
	require 'threach'
	require 'jruby_streaming_update_solr_server'

	solrURL = 'your solr url'
	sussQueueSize = 128 # number of docs to queue up
	sussThreads = 1 # number of threads to use to send stuff to solr

	threads = 3 # number of threads to use to process the data

	1006 DLI
	1040 GyWOH
	1059 SciDir
	1062 NYU
	1077 ItFiC
	1150 CSt
	1175 FrPJT
	1176 DGPO
	1182 NjP
	1196 BLKDR

	# Then just create a simple XML file pointing to the 50k line files
	# Don't forget to gzip the files first

	#!/usr/local/bin/perl

	my $numfiles = ARGV[0]; # number of files generated before
	my $urlToSitemapDir = 'http://www.my.machine.edu/dir/for/sitemaps';

	print q{<?xml version="1.0" encoding="UTF-8"?>
	<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">


	{
	"responseHeader":{
	"status":0,
	"QTime":0,
	"params":{
	"explainOther":"",
	"fl":"*,score",
	"indent":"on",
	"start":"0",

	require 'rubygems'
	require 'marc'
	require 'open-uri'

	r = MARC::Reader.new(open('http://mirlyn.lib.umich.edu/Record/000039829.marc')).first
	puts r

	require 'java'


	def showBreakProblem &blk
	threads = 2
	consumers = []
	threads.times do \|i\|
	consumers << Thread.new(i) do \|i\|
	Thread.current[:num] = i
	begin

	1. Go to the app directory

	cd /Applications/Google\ Chrome.app/Contents/MacOS/

	2. Rename the app to app.orig

	mv Google\ Chrome Google\ Chrome.orig

	3. Create a shell script with the original name that uses the args you want

	require 'parslet'
	require 'pp'

	class AdvParser < Parslet::Parser
	rule(:space) { match['\\s\\t'].repeat(1) } # at least one space/tab
	rule(:space?) { space.maybe } # zero or 1 things that match the 'space' rule

	rule(:startexpr) { str('(') >> space? } # '(' followed by optional space
	rule(:endexpr) { space? >> str(')') }

	module A
	def a
	end
	end

	class C
	end

	n = 100_000