Skip to content

Instantly share code, notes, and snippets.

@rbavery
Last active September 30, 2020 00:25
Show Gist options
  • Save rbavery/223c00c2934e07ceb177787eadd7af60 to your computer and use it in GitHub Desktop.
Save rbavery/223c00c2934e07ceb177787eadd7af60 to your computer and use it in GitHub Desktop.
Search for, submit and download large, long running orders with porder
# export pl api key if porder doesn't work after using planet init. you can use porder quota to check if it is authenticated
export PL_API_KEY=######################
# create an idlist from an aoi and date range. there are other filters in the porder docs for more complicated orders
porder idlist --input "full path to .geojson" --start "2018-02-01" --end "2020-09-28" --item "PSScene4Band" --asset "analytic_sr" --outfile "full path to idlist.csv" --cmax ".10" --overlap "15" --number 10000000
# splits idlist into multiple csvs with max order size length. you can make 10 orders of size 500 at a time for a total of 5000 assets being processed at a time
porder idsplit --idlist "full path to idlist.csv" --lines "500" --local "full path to splitcsvs folder"
# get list of csv filenames
cd "full path to splitcsvs folder"
csvs=$(ls *.csv)
cd ..
# used to pause if too many orders running using while loop.
# sed -n 5,14p will cut out the top lines of the table without order urls up to the fifth line
# the 14 will cut out lines after line 14, which discludes orders in case you made any orders by mistake
# use more sed statements as needed to only use the right order urls within a date range that you want to download
# since quota is charged by what is downloaded
running=$(porder ostate --state running --start 2020-09-28 --end 2020-09-29 | sed -n 5,14p | awk -F "|" '{print $4}' | wc -l)
# this will make orders until 10 orders are being processed, then it will wait until some orders are done
for csv in $csvs; do while [ $running -gt 10 ]; do echo "can't order"; sleep 100; running=$(porder ostate --state running --start 2020-09-28 --end 2020-09-29 | sed -n 5,14p | awk -F "|" '{print $4}' | wc -l); done; porder order --name "niger {$csv}" --idlist "full path to splitcsv folder/$csv" --item "PSScene4Band" --boundary "full path to .geojson" --bundle "analytic_sr_udm2,analytic_sr" --op clip harmonize email; done
# get space delimited order links, pay attention to if the sed statement is what you want here and on line 16
orders=$(porder ostate --state success --start 2020-09-16 --end 2020-09-18 | sed -n 5,14p | awk -F "|" '{print $4}')
# download each order using the async/multiprocessing downloader, each order link is processed sequentially
for link in $orders; do porder multiproc --url $link --local "full path to download folder"; done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment