Skip to content

Instantly share code, notes, and snippets.

@troyhill
troyhill / parseMonthlyLCD.R
Last active November 25, 2015 22:09
Code parses monthly National Climatic Data Center Local Climatological Data reports (downloaded as .txt files)
### Script to parse NCDC LCD comma-separated .txt files
parseMonthlyLCD <- function(txt_list) {
# function takes a vector of comma separated filenames for NCDC LCD data (https://www.ncdc.noaa.gov/IPS/lcd/lcd.html)
# returns a summary of monthly meteorological data
# this seems to work at least back to 2009, but NCDC formatting is inconsistent, so use this code with caution
# usage example:
# download .txt files from https://www.ncdc.noaa.gov/IPS/lcd/lcd.html
# input_files <- list.files(path = "C:/RDATA/NCDC/NewOrleans/2013", full.names = TRUE, pattern = ".txt") # find .txts in a directory
# parseMonthlyLCD(input_files)
@troyhill
troyhill / NorthwestInstruments_conversion.R
Last active March 27, 2018 17:00
Script converts total station data (Northwest Instruments NTS02) to lat/long or state plane (or other) coordinate systems using three control points.
# Script converts total station data (Northwest Instruments NTS02) to
# lat/long or state plane (or other) coordinate systems using three control points.
#####
# User inputs
#
# set input data: raw output from the NorthWest Instruments software
tsData <- read.csv("C:/RDATA/FelixNeck/data/data_elevation_20170504.csv", header=TRUE, na.strings=".")
# set the location of output file containing northing/easting and lat/long data
@troyhill
troyhill / BarPilotsDock_20170414.csv
Created April 14, 2017 18:22
NOAA data: Bar Pilots Dock
We can't make this file beautiful and searchable because it's too large.
"time (GMT)","verified water level at Mayport (Bar Pilots Dock), FL (meters rel. to NAVD)","station"
2013-01-01 00:00:00,-0.34,"Mayport (Bar Pilots Dock), FL"
2013-01-01 00:06:00,-0.3,"Mayport (Bar Pilots Dock), FL"
2013-01-01 00:12:00,-0.282,"Mayport (Bar Pilots Dock), FL"
2013-01-01 00:18:00,-0.235,"Mayport (Bar Pilots Dock), FL"
2013-01-01 00:24:00,-0.185,"Mayport (Bar Pilots Dock), FL"
2013-01-01 00:30:00,-0.164,"Mayport (Bar Pilots Dock), FL"
2013-01-01 00:36:00,-0.144,"Mayport (Bar Pilots Dock), FL"
2013-01-01 00:42:00,-0.123,"Mayport (Bar Pilots Dock), FL"
2013-01-01 00:48:00,-0.105,"Mayport (Bar Pilots Dock), FL"
@troyhill
troyhill / TridentPier_20170414.csv
Created April 14, 2017 21:14
Trident Pier NOAA data 2013-2015
We can't make this file beautiful and searchable because it's too large.
"time (GMT)","verified water level at Trident Pier, FL (meters rel. to NAVD)","station"
2013-01-01 00:00:00,-0.411,"Trident Pier, FL"
2013-01-01 00:06:00,-0.424,"Trident Pier, FL"
2013-01-01 00:12:00,-0.37,"Trident Pier, FL"
2013-01-01 00:18:00,-0.36,"Trident Pier, FL"
2013-01-01 00:24:00,-0.339,"Trident Pier, FL"
2013-01-01 00:30:00,-0.285,"Trident Pier, FL"
2013-01-01 00:36:00,-0.293,"Trident Pier, FL"
2013-01-01 00:42:00,-0.255,"Trident Pier, FL"
2013-01-01 00:48:00,-0.224,"Trident Pier, FL"
@troyhill
troyhill / dbhydro_functions.R
Last active October 12, 2018 17:15
An abbreviated R API for SFWMD's DBHYDRO: download station and project data
#!/usr/bin/Rscript
### functions to download DBHYDRO station and project data
# setwd("/home/thill/RDATA/dbhydro/data/20180419/")
library(httr)
# command line arguments: order matters (right now)
# arguments must be in order: [project/station name] [working_dir] [reportType: station or project]
name <- gsub("[^[:alnum:] ]", "", commandArgs()[6])
working_dir <- sapply(strsplit(commandArgs()[7], split = "\\["), "[", 1)
@troyhill
troyhill / upload_project_data_20180425.sh
Last active July 10, 2018 18:27
DBHYDRO-DataForEver data transfer
#!/bin/bash
# Script downloads station or project-level water quality data from DBHYDRO and uploads it to DataForEver
# Projects are specified in a file argument sent to script
# Usage: ./bash/upload_project_data_20180425.sh ~/RDATA/dataForEver/station_lists/20180514_deering.lst station
# create date-stamped folder for downloaded data
if [ "$#" < 1 ]
then
echo "Usage: $0 target_list report_type" 1>&2
@troyhill
troyhill / getNOAAdata.sh
Last active September 17, 2018 17:00
R-bash integration: bash script
#!/bin/bash
# Script downloads NOAA station data and saves it to a .csv file
# Usage: ./bash/getNOAAdata.sh ./bash/getNOAAdata.R 8656483 8665530
if [ "$#" -lt 2 ]; then
echo "Usage: $0 R_script NOAA_station1 NOAA_station2 ..." 1>&2
exit 1
fi
Rscript=$1
@troyhill
troyhill / getNOAAdata.R
Last active September 17, 2018 16:59
R-bash integration: R script
#!/usr/bin/Rscript
### Download NOAA station data
### First, check if VulnToolkit is installed, install if it's not
list.of.packages <- c("devtools", "VulnToolkit")
new.packages <- list.of.packages[!(list.of.packages %in%
installed.packages()[,"Package"])]
if ("devtools" %in% new.packages) {
@troyhill
troyhill / S333_NP206_stage.R
Created September 27, 2018 13:54
Script to plot water levels at S-333 and NP206
###
### Script to plot water levels at S-333 and NP206
###
stg <- read.csv("/data_S333_NP206_20180927.csv")
plot(stg$stg.NP206 ~ stg$hw.S333, pch = 19, cex = 0.5, las = 1, xlab = "Stage at S-333 (headwater; ft NGVD29)", ylab = "Stage at NP206 (ft NGVD29)", col = "darkgray")
abline(lm1 <- lm(stg$stg.NP206 ~ stg$hw.S333), col = "blue", lty = 1)
### estimate stage at NP206 corresponding to Walker's "low trigger" at S-333
@troyhill
troyhill / reformData.R
Last active October 11, 2018 16:22
Script to reformat water level predictions from Thai Navy monitoring stations
### script to reformat water level predictions from Thai Navy monitoring stations
reformatData <- function(filename, timezone = "GMT") {
### filename = location of tab-delimited text file with Thai Navy water level predictions
### timezone = I'm not sure how to refer to the local time zone, so I used GMT to at least avoid daylight savings time issues
### usage example
# UdayDat <- "C:/RDATA/Thailand/data/data_waterLevel2018.txt"
# dat1 <- reformatData(filename = UdayDat)
# tail(dat1)