On the head node:
# yum update -y
# yum install -y docker
# systemctl enable docker && systemctl start docker
# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
| #!/usr/bin/env python | |
| BLOCKSIZE=32*1024*1024 | |
| import sys | |
| from zlib import adler32 | |
| for f in sys.argv[1:]: | |
| val = 1 | |
| if f=='-': | |
| fp=sys.stdin | |
| else: |
| #!/usr/bin/env python | |
| # | |
| # bestman callout adler32 | |
| # | |
| # This reads or sets the extended attribute user.adler32 for files in Lustre | |
| # | |
| # use: | |
| # | |
| # adler32.py | |
| # |
| #!/usr/bin/env python | |
| # | |
| # bestman callout adler32 | |
| # | |
| # This reads or sets the extended attribute user.adler32 for files in Lustre | |
| # | |
| # use: | |
| # | |
| # adler32.py | |
| # |
| #!/usr/bin/env python | |
| # | |
| # bestman callout adler32 | |
| # | |
| # This reads or sets the extended attribute user.adler32 for files in Lustre | |
| # | |
| # use: | |
| # | |
| # adler32.py | |
| # |
| #!/bin/bash | |
| ################################################################################ | |
| # blahp-over-ssh shim script | |
| # | |
| # command [options] remote_hostname [remote options/arguments] | |
| # | |
| # Options: | |
| # see usage string | |
| # |
The "Campus Factory" script basically monitors schedds for idle jobs and submits as appropriate to the remote cluster. A typical submission looks like this:
condor_submit /home/ruc.clemson/bosco/libexec/campus_factory/share/glidein_jobs/job.submit.template -a REMOTE_SCHEDULER="pbs" -a WN_TMP="/local_scratch" -a GLIDEIN_Site="osgconnect@user.palmetto.clemson.edu/pbs" -a GLIDEIN_DIR="/home/ruc.clemson/bosco/libexec/campus_factory/share/glidein_jobs" -a REMOTE_FACTORY="/scratch1/osgconnect/bosco/clemson/campus_factory" -a PASSWDFILE_LOCATION="/home/ruc.clemson/bosco/local.bosco/passwdfile" -a BOSCOCluster="osgconnect@user.palmetto.clemson.edu/pbs" -a GLIDEIN_HOST="rccf-osg.ci-connect.net:11012?sock=collector" -a REMOTE_CLUSTER="osgconnect@user.palmetto.clemson.edu"I suspect APF can completely replace this component.
The job submit template looks like this:
I think in my ideal world, I'd have unlimited hardware to install every quirky Linux or UNIX distro under the Sun. Since I don't live in that glorious fantasy realm, I've taken to virtualizing pretty much everything lately. One of the things that irritates me about that approach, though, is that I have to download a relatively fat ISO in order to get at a fairly small installer and bootstrap my shiny new OS.