Skip to content

Instantly share code, notes, and snippets.

View lbjay's full-sized avatar

Jay Luker lbjay

View GitHub Profile
// put this in yr custom json
"elk": {
"kibana_auth": {
"user": "foo",
"pass": "bar"
},
"harvester_release": "v0.1.0"
}
Starting storage, including a deploy
Starting db, including a deploy
Waiting on "bash /usr/bin/vagrant up storage" to finish . . .
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.
Command: ["setextradata", "07be0611-9249-468c-b3ee-96b6b6894db8", "VBoxInternal2/SharedFoldersEnableSymlinksCreate/tmp_vagrant-opsworks", "1"]
Stderr: VBoxManage: error: The machine 'mh-opsworks_storage_1459356738934_15160' already has a lock request pending
VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component MachineWrap, interface IMachine, callee nsISupports
@lbjay
lbjay / bench.py
Last active April 5, 2016 18:18
mh_user_action inserter
import sys
import time
import boto3
import logging
import argparse
from faker import Factory
from random import choice, shuffle
from multiprocessing import Process, JoinableQueue
from mysql.connector import connect
Hi all,
At Harvard DCE we've been developing a few strategies for automated horizontal scaling of workers. Some involve analysis of AWS CloudWatch metrics and others look at data from Matterhorn itself, e.g. queued jobs. All of them, to a greater or lesser degree, are hindered by the behavior of the job dispatching mechanisms. I'll give a quick example of a typical scenario and why it causes problems for both types of strategies.
Say we have a cluster that has 10 workers at it's disposal but only 1 currently online. A burst of producer activity creates a bunch of new workflows and quickly there are a dozen media inspection and compose jobs generated. What we would expect is that the `max_jobs: n` setting on the worker host would prevent more than n of those jobs from being assigned to the single worker, the rest being put into some sort of queued state. What actually happens is that the 1 worker happily accepts all of them because the dispatching logic in `matterhorn-serviceregistry/.../ServiceRegistryJpaIm

Keybase proof

I hereby claim:

  • I am lbjay on github.
  • I am lbjay (https://keybase.io/lbjay) on keybase.
  • I have a public key ASAthX1IDfS1Dlsz8-JBZaXF6ScVlVXY7OqGpmBOTO1I9wo

To claim this, I am signing this object:

@lbjay
lbjay / gitwishes.json
Last active June 9, 2022 13:27
gitwishes exclude repos & patterns
{
"exclude_repos": ["oh_my_fortune","gotgit","GithubBotRepo","dlpkuhole2"],
"exclude_terms": ["Garbage Man", "roomservice", "fluent in silence"],
"queries": [
"i wish",
"i just wish",
"i really wish",
"i seriously wish",
"i seriously hope",
"i really hope"
Add a `.env` variable, `NOTIFICATION_EMAIL`
Then update the `run_stack_command` fabric task to pass that to cloudformattion as `NotificationEmail`.
# then in template.yml
Parameters:
...
NotificationEmail:
Type: String
@lbjay
lbjay / DCE_LHT_developer.md
Last active December 1, 2021 21:18
Part-time Software Developer position at DCE

Part-time Software Developer

Harvard Division of Continuing Education (DCE)

Join us as we design and develop the tools and systems that power DCE's teaching and learning environments!

DCE is interested in hiring Less Than Half-Time (LHT) software developers. The DCE Software & Systems Development Group is responsible for creating and maintaining the services used by faculty, producers, instructional design staff, and thousands of DCE students around the world. Projects are varied and range from front-end, JavaScript-powered UI elements, to Canvas LTI integrations, Slack bots, Python and R analytics work, DevOps, AWS infrastructure (including "serverless" apps), and a back-end, distributed, Java application.

Almost all of our development relies on Open Source libraries, and much of our work is released into, or contributed back to, Open Source projects, such as the Opencast Lecture Capture System (https://opencast.org/) and the Paella video player (https://paellaplayer.upv.es/). Feel free to peru

@lbjay
lbjay / zadara_graphs.html
Created December 14, 2018 13:47
zadara atime graphs
<!DOCTYPE html>
<html>
<head><meta charset="utf-8" />
<title>zadara_graphs</title><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>
<style type="text/css">
/*!
*
* Twitter Bootstrap