A shell script to "batch download" the NCCN guideline by Cookie in just one command

nccn.sh

#!/bin/bash
# title: download NCCN guideline
# date created: "2023-08-09"

green_color="\033[32m"
reset_color="\033[0m"
today_date=$(date +"%Y-%m-%d")
input_file="nccnlist.txt"

# Check if the input file exists
if [ -f nccnlist.txt ] && [ -f cookie.txt ]; then
    echo "Both files exist"
    while IFS= read -r line; do
        echo "Start to Download: ${green_color}$line${reset_color}"
        curl "https://www.nccn.org/professionals/physician_gls/pdf/$line.pdf" \
            -H 'authority: www.nccn.org' \
            -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
            -H 'accept-language: zh-TW,zh;q=0.9' \
            -H 'cache-control: max-age=0' \
            -H "Cookie: $(cat cookie.txt)" \
            --compressed --output "NCCN-$line-$today_date.pdf"
        sleep 3
    done <"$input_file"
else
    if [ ! -f nccnlist.txt ]; then
        touch file1.txt
        echo "Created nccnlist.txt"
    fi
    if [ ! -f cookie.txt ]; then
        touch cookie.txt
        echo "Created cookie.txt"
    fi
fi


exit 0

Batch Donwload the NCCN guideline

Method

• look at filename to findout the filename you want to donwload
• create a file called the nccnlist.txt, typing the file you want to donwload line by line, no space
• donwload the chrome/edge extension gaoliang/cookie-cook
• login into the NCCN website
• run the extension, you will get the window like below:

• click the bottom dropdown menu, select the Http Header value, then click the copy bottom
• create a file called cookie.txt paste the cookie in it, check if there's any empty line, remove it
• run

sh nccn.sh

• then you will get your file

filename

Cancer Type	filename.pdf
Acute Lymphoblastic Leukemia	all
Acute Myeloid Leukemia	aml
Ampullary Adenocarcinoma	ampullary
Anal Carcinoma	anal
Basal Cell Skin Cancer	nmsc
B-Cell Lymphomas	b-cell
Biliary Tract Cancers	btc
Bladder Cancer	bladder
Bone Cancer	bone
Breast Cancer	breast
Central Nervous System Cancers	cns
Cervical Cancer	cervical
Chronic Lymphocytic Leukemia/Small Lymphocytic Lymphoma	cll
Chronic Myeloid Leukemia	cml
Colon Cancer	colon
Dermatofibrosarcoma Protuberans	dfsp
Esophageal and Esophagogastric Junction Cancers	esophageal
Gastric Cancer	gastric
Gastrointestinal Stromal Tumors	gist
Gestational Trophoblastic Neoplasia	gtn
Hairy Cell Leukemia	hairy_cell
Head and Neck Cancers	head-and-neck
Hepatobiliary Cancers	hepatobiliary
Hepatocellular Carcinoma	hcc
Histiocytic Neoplasms	histiocytic_neoplasms
Hodgkin Lymphoma	hodgkins
Kaposi Sarcoma	kaposi
Kidney Cancer	kidney
Melanoma: Cutaneous	cutaneous_melanoma
Melanoma: Uveal	uveal
Merkel Cell Carcinoma	mcc
Mesothelioma: Peritoneal	meso_peritoneal
Mesothelioma: Pleural	meso_pleural
Multiple Myeloma	Myeloma
Myelodysplastic Syndromes	mds
Myeloid/Lymphoid Neoplasms with Eosinophilia and Tyrosine Kinase Gene Fusions	mlne
Myeloproliferative Neoplasms	mpn
Neuroendocrine and Adrenal Tumors	neuroendocrine
Non-Small Cell Lung Cancer	nscl
Occult Primary	occult
Ovarian Cancer/Fallopian Tube Cancer/Primary Peritoneal Cancer	ovarian
Pancreatic Adenocarcinoma	pancreatic
Pediatric Acute Lymphoblastic Leukemia	ped_all
Pediatric Aggressive Mature B-Cell Lymphomas	ped_b-cell
Pediatric Central Nervous System Cancers	ped_cns
Pediatric Hodgkin Lymphoma	ped_hodgkin
Penile Cancer	penile
Primary Cutaneous Lymphomas	primary_cutaneous
Prostate Cancer	prostate
Rectal Cancer	rectal
Small Bowel Adenocarcinoma	small_bowel
Small Cell Lung Cancer	sclc
Soft Tissue Sarcoma	sarcoma
Squamous Cell Skin Cancer	squamous
Systemic Light Chain Amyloidosis	amyloidosis
Systemic Mastocytosis	mastocytosis
T-Cell Lymphomas	t-cell
Testicular Cancer	testicular
Thymomas and Thymic Carcinomas	thymic
Thyroid Carcinoma	thyroid
Uterine Neoplasms	uterine
Vulvar Cancer	vulvar
Waldenström Macroglobulinemia / Lymphoplasmacytic Lymphoma	waldenstroms
Wilms Tumor (Nephroblastoma)	wilms_tumor