data rescue

Instructions:

Been downloading for 3 days and only ~20% through 1 of the 6 FTP sites, so handing on instructions to someone with a dedicated and fast server for download.

See "FTP Access" links at https://eosweb.larc.nasa.gov/datapool

download each:
curl ftp://l5eil01.larc.nasa.gov/ACRIM_III/ACR3L2SC.001/ -o ACR3L2SC.001.txt

get the pathname:
awk '{ print $8 }' ACR3L2SC.001.txt > ACR3L2SC.001_cured.txt

clone harvesting-tools:
https://github.com/edgi-govdata-archiving/harvesting-tools

setup url.txt:
sysops-MBP-3% more url.txt
l5eil01.larc.nasa.gov /ACRIM_III/ACR3L2DM.001/

use this iterator, name it something like download.sh:
#!/bin/sh

url=$(awk -F = '{print $1}' url.txt)
mkdir -p data
for i in $(cat /a/path/ACR3L2DM.001_cured2.txt);
do 
        # echo "${url}${i}"
        python3 /another/path/ftp/download_ftp_tree.py ${url}${i} /path/to/zip/starter/7F02E2F0-45FF-405A-B439-25EE72C5BD5D/data
        sleep 2
done

run /download.sh for a very very long time