Colin grosscol
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require 'watir' | |
| require 'fileutils' | |
| require 'pry' | |
| # Given url and destination directory | |
| def scrape_tables(browser, url, dest_dir) | |
| # create subdirectory | |
| FileUtils.mkdir_p dest_dir | |
| browser.goto url |
grosscol
/ ancestor_list_child_tree.rb
Last active
December 6, 2016 22:52
List of nodes with ancestors and corresponding tree structure with children.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| nodes = [ | |
| {id: '1', ancestor_ids: []}, | |
| {id: '12', ancestor_ids: ['1']}, | |
| {id: '13', ancestor_ids: ['1']}, | |
| {id: '124', ancestor_ids: ['1','12']}, | |
| {id: '125', ancestor_ids: ['1','12']}, | |
| {id: '136', ancestor_ids: ['1','13']}, | |
| {id: '1367', ancestor_ids: ['1','13','136']}, | |
| {id: '1368', ancestor_ids: ['1','13','136']}, | |
| {id: '19', ancestor_ids: ['1']} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require('dplyr') | |
| require('ggplot2') | |
| # Simulate two types of queries: fast and slow. More fast queries. | |
| num_samples = 10000 | |
| days <- sample(seq(1,30), num_samples, replace=TRUE) | |
| qtimes <- rpois(num_samples, c(25,35,100)) | |
| qlog <- data.frame(day=days, qtime=qtimes) | |
| # take a quick look. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # Performance testing normalization for disk related operations | |
| # Regression modeling for fcrepo performance (P) modeled by device write (W) and java startup time (J) | |
| # P ~ W + J | |
| ## CSV format for time output. Will be munged by subsequent analysis | |
| FORMAT_HEADER="wall_time,user_time,system_time" | |
| FORMAT_STRING="%e,%U,%S" |
grosscol
/ FeatureError Stacktrace
Created
October 10, 2016 20:39
assign_admin_set Stacktrace from Sufia 7.2 upgrade
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Started GET "/" for 127.0.0.1 at 2016-10-10 16:37:58 -0400 | |
| ActiveRecord::SchemaMigration Load (0.4ms) SELECT "schema_migrations".* FROM "schema_migrations" | |
| ActiveFedora: loading fedora config from /home/grosscol/workspace/vanilla-sufia/config/fedora.yml | |
| ActiveFedora: loading solr config from /home/grosscol/workspace/vanilla-sufia/config/solr.yml | |
| Processing by Sufia::HomepageController#index as HTML | |
| Usergroups are ["public"] | |
| ContentBlock Load (0.2ms) SELECT "content_blocks".* FROM "content_blocks" WHERE "content_blocks"."name" = ? ORDER BY created_at DESC LIMIT 1 [["name", "featured_researcher"]] | |
| ContentBlock Load (0.2ms) SELECT "content_blocks".* FROM "content_blocks" WHERE "content_blocks"."name" = ? LIMIT 1 [["name", "marketing_text"]] | |
| ContentBlock Load (0.1ms) SELECT "content_blocks".* FROM "content_blocks" WHERE "content_blocks"."name" = ? LIMIT 1 [["name", "announcement_text"]] | |
| Completed 500 Internal Server Error in 129ms (ActiveRecord: 1.1ms) |
grosscol
/ irb_and_stack_trace
Created
September 15, 2016 19:07
Hydra PCDM::Object (FileSet) and PCDM::File (original_file) results in Fedora LDP 500 response.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| irb(main):004:0> file_set = FileSet.find("0c483j36g") | |
| => #<FileSet id: "0c483j36g", label: "Evans_Old_Field_Plant_Database.zip", relative_path: nil, import_url: nil, part_of: [], resource_type: [], creator: [], contributor: [], description: [], keyword: [], rights: [], publisher: [], date_created: [], subject: [], language: [], identifier: [], based_near: [], related_url: [], bibliographic_citation: [], source: [], head: [], tail: [], depositor: "[email protected]", title: ["Evans_Old_Field_Plant_Database.zip"], date_uploaded: "2016-03-04", date_modified: nil, access_control_id: "5b97fa6e-ed85-4293-a188-294331ac8904", embargo_id: nil, lease_id: nil> | |
| irb(main):005:0> file_set.original_file | |
| => #<Hydra::PCDM::File uri="http://localhost:8301/fedora/rest/umrdr/0c/48/3j/36/0c483j36g/files/d39bd72b-ada0-4c69-9b3d-40ec151d5525" > | |
| irb(main):006:0> file_set.original_file.metadata | |
| => #<ActiveFedora::WithMetadata::MetadataNode:0x3f8da1e337d0(#<ActiveFedora::WithMetadata::MetadataNode:0x007f1b43c66fa0>)> | |
| irb(main):007:0> f |
grosscol
/ Migration Ldp 500
Created
September 14, 2016 04:16
Sufia 7.pre -> Sufia 7.1 Migration Error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| irb(main):002:0> fs = FileSet.find('gb19f580q') | |
| => #<FileSet id: "gb19f580q", label: "it150815_074300_000.idl", relative_path: "Data/Simulation Output/Ionosphere Output/it150815_...", import_url: nil, part_of: [], resource_type: [], creator: [], contributor: [], description: [], keyword: [], rights: [], publisher: [], date_created: [], subject: [], language: [], identifier: [], based_near: [], related_url: [], bibliographic_citation: [], source: [], head: [], tail: [], depositor: "[email protected]", title: ["it150815_074300_000.idl"], date_uploaded: "2016-04-21 18:49:04", date_modified: "2016-04-21 18:49:04", access_control_id: "10d89226-68ee-4661-afff-9e3056f45e1a", embargo_id: nil, lease_id: nil> | |
| irb(main):003:0> fs.original_file | |
| => #<Hydra::PCDM::File uri="http://localhost:8301/fedora/rest/umrdr/gb/19/f5/80/gb19f580q/files/90e50337-8c1a-4482-9842-3eaa9641fc58" > | |
| irb(main):004:0> fs.characterization_proxy | |
| => #<Hydra::PCDM::File uri="http://localhost:8301/fedora/rest/umrdr/gb/19/f5/80/gb19f580q/files/90e50 |
grosscol
/ print_af_base_properties.rb
Created
August 7, 2016 16:03
Print properties and predicates of an ActiveFedora::Base subclass
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Use given single class name arg or default to GenericWork | |
| desc "Print properties of a sub-class of ActiveFedora::Base" | |
| task :properties_to_csv, [:class_name] => :environment do |t, args| | |
| Rails.application.eager_load! | |
| class_name = args.one? ? args[:class_name] : "GenericWork" | |
| print_properties_of class_name.constantize | |
| end | |
| # Given a class, print some properties information as csv | |
| def print_properties_of(klass) |
grosscol
/ print_af_base_properties.rb
Created
August 7, 2016 16:03
Print properties and predicates of an ActiveFedora::Base subclass
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Use given single class name arg or default to GenericWork | |
| desc "Print properties of a sub-class of ActiveFedora::Base" | |
| task :properties_to_csv, [:class_name] => :environment do |t, args| | |
| Rails.application.eager_load! | |
| class_name = args.one? ? args[:class_name] : "GenericWork" | |
| print_properties_of class_name.constantize | |
| end | |
| # Given a class, print some properties information as csv | |
| def print_properties_of(klass) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"failed_at"=>"2016/07/22 17:22:34 EDT", | |
| "payload"=>{"class"=>"ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper", | |
| "args"=>[{"job_class"=>"AttachFilesToWorkJob", | |
| "job_id"=>"743a3137-e4bd-4761-8031-b14f7f2e8c24", | |
| "queue_name"=>"ingest", | |
| "arguments"=>[{"_aj_globalid"=>"gid://vs/Work/jq085j963"}, [{"_aj_globalid"=>"gid://vs/Sufia::UploadedFile/14"}, {"_aj_globalid"=>"gid://vs/Sufia::UploadedFile/15"}, {"_aj_globalid"=>"gid://vs/Sufia::UploadedFile/16"}, {"_aj_globalid"=>"gid://vs/Sufia::UploadedFile/17"}]], "locale"=>"en"}]}, | |
| "exception"=>"ActiveJob::DeserializationError", | |
| "error"=>"Error while trying to deserialize arguments: uninitialized constant Work", | |
| "backtrace"=>["/hydra-dev/umrdr-staging/shared/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.6/lib/active_support/inflector/methods.rb:261:in `const_get'", "/hydra-dev/umrdr-staging/shared/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.6/lib/active_support/inflector/methods.rb:261:in `block in constantize'", "/hydra-dev/umrdr-staging/shared/vendor |