Skip to content

Instantly share code, notes, and snippets.

#!/usr/bin/env python
"""
Login to DataONE using ORCID and retrieve token.
This script uses Selenium to control a Chrome web browser to log on to
DataONE using ORCID credentials.
Dependencies:
* Keyring: pip install keyring
@datadavev
datadavev / getsitemaps.sh
Last active February 12, 2020 17:08
Shell script to retrieve sitemaps from a CN to a search instance, correcting URLs and preserving file timestamps.
#/bin/bash
# Retrieve and transform metacat generated sitemaps
#
# Requires wget, xmlstarlet
#
# Run with:
# sudo bash getsitemap.sh
#sitemap namespace
SM_NS="sm=http://www.sitemaps.org/schemas/sitemap/0.9"
@datadavev
datadavev / certshow
Created March 7, 2019 21:02
Show status of a remote certificate
#!/bin/bash
# Dump a cert or show dates valid
#
# e.g. to show valid dates:
# ./certshow -d google.com:443
show_validity=0
function showHelp() {
cat << EOF
@datadavev
datadavev / amirror.py
Last active November 25, 2019 02:45
Headless partial render of a javascript app page
"""
Returns a partially rendered DataONE view, without the additional javascript
being executed.
This script is proof of concept only, and will only handle a single
request at a time since the single Chrome driver instance is reused.
Could recreate the driver for each request or use a pool of drivers.
Given a URL that starts with "https://search.", load the page using
Selenium and allow javascript to execute until the desired content
@datadavev
datadavev / d1_dataextract.py
Last active August 22, 2018 00:05
Extract structured data from DataONE search result page
import time
import urllib
from selenium import webdriver
import extruct
import pprint
import clipboard
pid = "https://pasta.lternet.edu/package/metadata/eml/knb-lter-fce/1224/1"
#The view service is broken by not accepting an escaped path element
#url = "https://search.dataone.org/view/" + urllib.parse.quote_plus(pid)
@datadavev
datadavev / certcheck
Created April 23, 2018 20:30
Quickly check certificate validity from the command line.
#!/bin/bash
show_validity=0
function showHelp() {
cat << EOF
Show a certificate from a remote service.
Usage:
@datadavev
datadavev / pptxslidesizes.py
Last active November 27, 2017 18:25
Show the sizes of objects in a pptx file
#!/usr/bin/env python
'''
Python script to show the size of images (bytes) on each slide
Requires pptx installed:
pip install -U python-pptx
'''
import os
#!/usr/bin/env python
'''
MOVED TO: https://github.com/DataONEorg/DataONE_Operations/blob/master/scripts/d1verifytoken
This copy no longer updated.
Verify a DataONE bearer token
'''
# Stdlib
@datadavev
datadavev / login-to-dataone
Last active June 14, 2017 15:06
Automate browser to log in to DataONE
#!/usr/bin/env python
'''
MOVED TO: https://github.com/DataONEorg/DataONE_Operations/blob/master/scripts/d1logintoken
This copy no longer updated.
Login to DataONE using ORCID and retrieve token.
This script uses Selenium to control a Chrome web browser to log on to

Keybase proof

I hereby claim:

  • I am datadavev on github.
  • I am davev (https://keybase.io/davev) on keybase.
  • I have a public key ASDHLQuHSfde1UsrYARjAKou8ZhihNznA1PqHoh1sSO4-wo

To claim this, I am signing this object: