The following instructions are for OSX/Linux terminal. On Windows, you can use putty & filezilla, or install Windows Subsystem for Linux (details here).
## In terminal on local machine:
## Edit ~/.ssh/config and add the following
Host gacrc
User <yourname>
Hostname sapelo2.gacrc.uga.edu
## extra options, uncomment to activate
## see `man ssh_config` for details
# ForwardAgent yes
# ForwardX11 yes
Host xfer
User <yourname>
Hostname xfer.gacrc.uga.edu
Now make a test file and transfering it:
ls -alh >> tmp.file
scp tmp.file gacrc:
ssh gacrc
ls
## starting at local machine
ssh xfer
wget http://debian.osuosl.org/debian-cdimage/11.6.0/amd64/iso-dvd/debian-11.6.0-amd64-DVD-1.iso
## use Ctrl-C to stop
rm debian*.iso
## use Ctrl-D to exit
ssh gacrc
## Use your favorite editor
nano .bashrc
## start in your home directory
ls -alh
## Make symlink to your scratch directory
ln -s /scratch/$USER ./my.scratch
## Rohani lab directory: writeable by all
ls -alh /work/prlab
##
mkdir /work/prlab/<nickname>
ln -s /work/prlab/<nickname> ./my.work
ls -alh
ls -alh /work/prlab
## Some software is loaded but old:
git --version
## Search for newer:
module spider git
## Default: load newest available
module load git
git --version
module load R
module list
module unload R
- In your home directory, create a file named
.modulesand add the following lines:
module load git
## pin a version of R
module load R/4.2.1-foss-2020b
- Next, load them
source .modules
- First, create an interactive session
interact
source ~/.modules
R
- Next, run an R script
## Just once
install.packages('data.table')
source('/work/prlab/xian/lab-tutorial/load.safegraph.R', chdir=T)
exit
## back to login node
cd /work/prlab/xian/lab-tutorial
ls -alh
ls -alh ./data
less load.safegraph.R
less my.interact.sh
./my.interact.sh
./my.interact.sh 4 8 4
source ~/.modules
## config R multithreading
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
export MC_CORES=$SLURM_CPUS_PER_TASK
R
- Open a second terminal, connect to GACRC, and view
/work/prlab/xian/lab-tutorial/load.safegraph.R - On the first terminal (interactive session, in R):
system.time(source('load.safegraph.R'))
colnames(sg.visit)
dim(sg.visit)
## Memory used by data
object.size(sg.visit)
## easier to read
format(object.size(some.data), units='MB')
## how much memory did R use *total*?
gc()
## view subset
dim(sg.visit.sub)
head(sg.visit.sub)
## aggregate
summary(sg.visit.state)
head(sg.visit.state)
- In second terminal, view (compressed) saved file
cd ~
ls -alh
temp.data.csv.gz | less