Skip to content

Instantly share code, notes, and snippets.

View sstults's full-sized avatar

Scott Stults sstults

22 followers · 0 following
View GitHub Profile
@sstults
sstults / gist:5153709
Created March 13, 2013 16:16
This is the magic incantation I used to install gevent and zeromq in the "lucid32" vagrant box.
sudo apt-get update
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:chris-lea/zeromq
sudo add-apt-repository ppa:chris-lea/libpgm
sudo apt-get update
sudo apt-get install -y python-pyzmq python-setuptools python-dev python-pip libevent-dev
sudo pip install gevent
@sstults
sstults / update_solr.sh
Created March 14, 2013 19:00
Reads three words on a line from STDIN, then uses the first and last to post an update to Solr
while read line
do
declare -a arr=( $line )
curl --url "http://localhost:8983/solr/us_patent_grant/update/?commit=false" \
-H "Content-Type: text/xml" -d "<add><doc><field name='id'>0${arr[0]}</field><field name='page_count_int' update='set'>${arr[2]//[[:space:]]/}</field></doc></add>"
done
@sstults
sstults / 2009_cumulative.csv
Created March 14, 2013 19:03
Sample of the cumulative page counts of multi-page patent grant TIFFs
We can make this file beautiful and searchable if this error is corrected: No commas found in this CSV file in line 0.
0014317 USP2009w20 2
0015257 USP2009w20 1
0028699 USP2009w43 3
0032450 USP2009w20 2
0032451 USP2009w20 2
0056575 USP2009w12 3
0066524 USP2009w12 3
0072873 USP2009w19 4
0087053 USP2009w19 4
0087923 USP2009w19 4
@sstults
sstults / loggit.sh
Created March 20, 2013 03:13
Count down (percentage done) of page_count_int updates
maxdocs=743364
while (( numFound < maxdocs ))
do
numFound=`curl -s "http://solr.demo.dir.o19s.com:8983/solr/us_patent_grant/select?q=+invention_title_html:*%20AND%20+page_count_int:%5B1%20TO%20*%5D&qt=lucene&fl=id,invention_title_html,page_count_int&rows=0&indent=true&omitHeader=true" | grep numFound | awk -F '"' '{print $4}'`
echo $numFound $((100*${numFound}/${maxdocs}))"%"
sleep 60
done
@sstults
sstults / Vagrantfile
Created January 6, 2014 16:05
Stands up an Ubuntu VM and installs NodeJS, git, Yeoman, and Compass
# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "precise64"
config.vm.box_url = "http://files.vagrantup.com/precise64_vmware.box"
@sstults
sstults / site.pp
Last active January 2, 2016 09:39
The git, yeoman, and compass portion of the install
node default {
class { 'nodejs':
version => 'stable',
}
package { 'git':
ensure => 'installed',
}
@sstults
sstults / yo-angular.sh
Last active January 2, 2016 10:09
Starting an Angular project and viewing the default page
vagrant up
vagrant ssh
sudo npm install -g yo
sudo chown -R vagrant:vagrant .npm
cd /vagrant
mkdir myproj
cd myproj
npm install generator-angular
yo angular
sed -i 's/localhost/0.0.0.0/' Gruntfile.js
@sstults
sstults / wordnet-indexer.py
Created November 11, 2015 02:31
#!/usr/bin/env python -x
import pysolr
import sys
from nltk.corpus import wordnet as wn
class Indexer:
"""
@sstults
sstults / clear_fields.py
Created March 6, 2018 22:29
Quick and sloppy script to clear out unused fields from Solr's schema API
import json
from urllib.parse import urlencode
from urllib.request import Request, urlopen
solr_url = 'http://localhost:8983/solr/gettingstarted/schema'
luke_file = open('luke-json-output.json', 'r')
luke_json = json.load(luke_file)
luke_file.close()
@sstults
sstults / security-testing.sh
Created March 26, 2018 19:14
Test Solr Auth/z
solr-6.6.2
solr start -c -e cloud
curl "http://localhost:8983/solr/gettingstarted/select?q=*:*"
curl "http://localhost:8983/solr/gettingstarted/update?commit=true" -H "Content-Type: text/xml" --data-binary '<delete><query>*:*</query></delete>'
cat > security.json <<EOM
{