Marcql preview

Marcql example

#|
Here's a sample of the Marcql DSL
This is a the direct input and output of the query copied from the REPL
|#

(marcql "~/Dropbox/code/marc/will-books-5.out"
	  select
	     "008" => count-me
	     "245" => print-first-30
	  where
	     "245" => contains-dog)

^_aJust a dog :^_bunderstanding ... 
^-aThe orchid shroud :^_ba novel... 
^_aBest in show :^_bthe dog in a... 
^_aHow to identify and control ... 
NIL
? *counter*
4

#|
Now you may be asking "What is going on"
The synatx should be pretty straight forward, 
Items below 'select' are the fields we want to select
and the 'where' statement are the conditions

the '=>' indicates an 'action' to be performed on the field
here are the missing functions for those actions:

(defun count-me (x)
  (incf *counter*))

(defun print-first-30 (x)
  (let ((end (if (< (length x) 30)
		(length x)
	      30)))
  (format T "~a... ~%" (subseq x 0 end))))

(defun contains-dog (x)
  (search "dog" x))

'actions' allow you the flexibility to do a lot with the records
while at the same time having optimal speed.

Here's the same output for this query on a 100k records (the above is only 10k)
with the time.
tl;dr it only take 6.3 seconds.
|#
(time
(marcql "~/Dropbox/code/marc/will-books-5.out"
	  select
	     "008" => count-me
	     "245" => print-first-30
	  where
	     "245" => contains-dog))
aJust a dog :bunderstanding ... 
aThe orchid shroud :ba novel... 
aBest in show :bthe dog in a... 
aHow to identify and control ... 
aLassie :bthe extraordinary ... 
aEffect of glyceryl trinitrat... 
aSister Bernadette's barking ... 
aWho's watching the watchdog?... 
aHot dog skiing /cby Bob "Bo... 
aThe hair of the dog :bthe b... 
aRoad dogs and loners :bfami... 
aBetween dog & wolf :bessays... 
aPedogenic isotopic indicator... 
aHotel for dogs /cLois Dunca... 
aPopulation ageing and econom... 
aClinical veterinary advisor ... 
aThe dialogue of the dogs.  ... 
aA history of travel in Ameri... 
aThe physiological clock :be... 
aThe way of a dog:bbeing the... 
aIt's okay to miss the bed on... 
aCruel and degrading :bthe u... 
aThe underdogs :bpictures an... 
aYear of the dog :ba novel /... 
aThe year of the dog :ba nov... 
aTheir dogs came with them :... 
aIn dogged loyalty :bthe rel... 
aThe complete dog book.  ... 
aMoods, songs, and doggerels ... 
aExercise for dogs /cJanice ... 
aThe dog :bJanuary 1985 - Ja... 
aHousing, husbandry, and welf... 
aSniffing out terrorism :bth... 
aA study of effects of hypert... 
aShort dog :bcab driver stor... 
aEstimation of the demand for... 
aLeashing the dogs of war :b... 
aThe Defense Travel System :... 
aThe curious incident of the ... 
aTolerance endpoint for evalu... 
aControl of dogfennel (Eupato... 
aTollerance [sic] by unacclim... 
aA temperature/humidity toler... 
aMerle's door :blessons from... 
aReframing rhetoric :ba libe... 
aThe politics of public money... 
aTransient thermal stresses i... 
aA cylindrical shell with a s... 
aA pressurized cylindrical sh... 
a"A dog's tale" ...cBy Mark ... 
aMy lead dog was a lesbian :... 
aThe dog fiend;bor, Snarleyy... 
aSurface cracks in a plate of... 
aStress intensity factors in ... 
aThe crack-inclusion interact... 
aExplosives, narcotics, and p... 
aWatchdogs, blogs and wild ho... 
aEstimating linear regression... 
aRescuing Sprite :ba dog lov... 
aGrowing and maintaining heal... 
aGrowing and maintaining heal... 
aColorful Creede, Colorado, 1... 
aEarthquakes and acoustic emi... 
aFuture bioethics :bovercomi... 
aLivestock guarding dogsh[el... 
aEffects of removing Good Hop... 
aMore than 200 years of Coast... 
aCoastieh[electronic resourc... 
aDepartment of Defense energy... 
aUnderdog advertisingh[elect... 
aNebraska and South Dakota bl... 
(MARCQL "~/Dropbox/code/marc/will-books-5.out" SELECT "008" => COUNT-ME "245" => PRINT-FIRST-30 WHERE "245" => CONTAINS-DOG) took 7,382,400 microseconds (7.382400 seconds) to run 
                    with 2 available CPU cores.
During that period, 6,322,842 microseconds (6.322842 seconds) were spent in user mode
                    804,275 microseconds (0.804275 seconds) were spent in system mode
573,030 microseconds (0.573030 seconds) was spent in GC.
 611,876,352 bytes of memory allocated.
 129 minor page faults, 0 major page faults, 0 swaps.
NIL