Skip to content

Instantly share code, notes, and snippets.

View Wildcarde's full-sized avatar

Garrett Mc. Wildcarde

View GitHub Profile
@Wildcarde
Wildcarde / docker-compose.yaml
Last active August 21, 2024 18:09
A quick docker compose for setting up openldap and ldap-ui behind traefik using tls termination for LDAPS
version: '2'
services:
ldap:
image: bitnami/openldap:2.6
env_file: .env
networks:
- default
- proxy
ports:
- 1389:1389
@Wildcarde
Wildcarde / 00_Notes.md
Last active September 30, 2021 16:04
Quick demos for submitting jobs to slurm under various circumstances

Slurm Getting Started / How-Tos

This document is meant to be provide some quick reference materials for gettings started with the basics of working with scripts against a slurm based computational cluster.

The 5 scripts below should all be created or downloaded into the same folder and submit commands run from that base directory.

diff --git a/pam.d/fingerprint-auth-ac b/pam.d/fingerprint-auth-ac
index 162f0bb..e09996c 100644
--- a/pam.d/fingerprint-auth-ac
+++ b/pam.d/fingerprint-auth-ac
@@ -8,6 +8,7 @@ auth required pam_deny.so
account required pam_unix.so
account sufficient pam_localuser.so
account sufficient pam_succeed_if.so uid < 1000 quiet
+account [default=bad success=ok user_unknown=ignore] pam_sss.so
account required pam_permit.so
@Wildcarde
Wildcarde / turkprime-custom.py
Created March 15, 2019 17:03
Glue code to make a 'turkprime' friendly URL in psiturk - untested beyond the test URL at this time
@custom_code.route('/turkprime')
def turkprime():
newrstring=request.query_string+'&mode=sdbx'
url=request.url_root+'/ad?'+newrstring
print("url: ",url)
return redirect(url)
## test url to use with this: http://localhost:22362/turkprime?workerId=ABBD1435&hitId=35hgfsbdu&assignmentId=vbwjebwufg
@Wildcarde
Wildcarde / idrac_addchassis.py
Created July 6, 2017 21:06
Small script to add entire idrac to cobbler in one step
#!/usr/bin/env python
"""
Functionally a rewrite of the CSES tool 'add_chassis_nodes(.sh)' however the approach
is slightly different. This script will add nodes to cobbler using the absolute minimum
information available. It automates the cobbler command line instead of using the API as
the developers have deemed the api to be unsafe in it's present state. It could be adapted to
do XMLRPC as well but is currently assumed to be run on the cobbler server itself so there is
no need. Some frameworking is done to make it possible to use this file as a component in a bigger
library, but as of now this approach to using the code is untested.
@Wildcarde
Wildcarde / 01_notes.md
Last active May 4, 2021 05:13
Check NFS script. This is designed to be run as a cron job on a system. It will retrieve a list of all NFS mounts from the mount command and verify that they are still mounted and not stale. If a mount is hung it will issue a lazy unmount and then attempt a remount. Activity will be logged to syslog and is grepable via the term `CHECKNFS`.

Notes for use of parallel pcap and auto termination of job on isilon

On Isilon nodes:

Run capstart.sh command to begin rotating parallel tcpdumps on all nodes with external interfaces.

On computation node:

  • the scripts check-nfs.sh and check-nfs-datacapture.sh must be loaded onto the node and stored in /root to be used via cron.
  • configure an ssh key on the node that can reach a specific node on the storage cluster as root (required to terminate the server tcpdump process by calling capstop.sh)
  • Make sure the file /tmp/nfsissuecaptured does not exist
  • run echo 0 &gt; /tmp/nfsmonitor.semaphore to reset the event notification from previous captures
@Wildcarde
Wildcarde / 01_Readme.md
Last active February 7, 2023 21:15
Quick demo embedding a Bokeh chart inside a Flask application

This works in a python 3.6 environment with bokeh and flask installed. To use this you need the following directory structure:

app/
 - templates/
   - hello.html
 - bokeh-slider.py
 - hello.py

After that you need to open two terminals at app/, in the first one you need to run the bokeh server with the command:

@Wildcarde
Wildcarde / unravel.py
Created May 10, 2016 21:09
Quick hack to reverse a list of IPs to their hostnames from firewall logs, this assumes python 2.7/3 and pandas are available
#!/bin/env python
import pandas as pd
import socket
df = pd.read_csv('<sourcecsv>.csv')
ips=df["Source address"]
for ip in ips:
print (socket.gethostbyaddr(str(ip)))
@Wildcarde
Wildcarde / Intro.md
Last active October 26, 2020 20:33
Spock Basics Talk

Spock Crash Course

Basic Architecture Info:

  • ~60 nodes
  • mixture of: ** 24 cores X 250GB memory / node ** 32 cores x 252GB memory / node
  • No Swap
  • Dedicated 10Gbps connection to Bucket / node
  • IB Backend for MPI - not available on 'redshirt' machines
@Wildcarde
Wildcarde / matlabparscript.m
Last active October 11, 2020 02:01
Parallel Matlab Spock Demo
function [] = parbench()
%main function for building pool and running benchmark
% this is adapted from the mathworks parallel benchmark demo
% to occupy only one file instead of several
% original article here: https://www.mathworks.com/help/distcomp/examples/simple-benchmarking-of-parfor-using-blackjack.html
%% Setup
%If you want more than 12 core you MUST create the pool by hand
% Below that it will detect the number of cores requested by and
% create a pool of the appropriate size. If this pool size is too