ONT Guppy setup

ont_guppy_setup.md

Overview

This markdown file contains the steps involved in configuring a new computer, runnning Ubuntu 16.04, to run ONT Guppy GPU basecalling.

Prerequisites

• CUDA must be installed, which can be simple or extremely difficult, depending on if the CUDA gods smile on you.
• The computer must be running Ubuntu 16.04 'xenial', with all updates installed.

Steps

• The steps in the installation manual were followed as directed.

• For the graphics card that was installed, a RTX 2080ti, no additional configuration was necessary, similar to the recommendations for the GTX 1080ti.

• guppy_basecaller was tested with the following parameters and a simple bash for loop:

    # directory contains 0.tar.gz, 1.tar.gz, ...
    for i in *.tar.gz ; do
      V=$(echo "$i" | cut -d. -f1)
      OUTDIRPATH=/output/path
      INPUTDIRPATH=/input/path
      mkdir $OUTDIRPATH/${V}
      guppy_basecaller -x "cuda:0" --input_path $INPUTDIRPATH --output_path $OUTDIRPATH --flowcell $FLOWCELL --kit $KIT --records_per_fastq 0
      echo "basecalling for $i done"
    done
    # the option "-x" specifies a CUDA graphics card at slot 0
    # if you have only 1 card, or the card you want is at slot 0, this is not necessary
    # to get info on graphics cards, use this command:
    nvidia-smi
  

Stats/Benchmark

• The graphics card at cuda:0 was used:

    bash$ nvidia-smi 
    Fri May 24 18:31:05 2019       
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 418.67       Driver Version: 418.67       CUDA Version: 10.1     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |===============================+======================+======================|
    |   0  GeForce RTX 208...  Off  | 00000000:17:00.0 Off |                  N/A |
    | 52%   82C    P2   252W / 250W |   4218MiB / 10989MiB |     78%      Default |
    +-------------------------------+----------------------+----------------------+
    |   1  Quadro P400         Off  | 00000000:65:00.0  On |                  N/A |
    | 34%   43C    P8    N/A /  N/A |    233MiB /  1992MiB |      0%      Default |
    +-------------------------------+----------------------+----------------------+
                                                                                   
    +-----------------------------------------------------------------------------+
    | Processes:                                                       GPU Memory |
    |  GPU       PID   Type   Process name                             Usage      |
    |=============================================================================|
    |    0      7762      C   guppy_basecaller                            4207MiB |
    |    1      1309      G   /usr/lib/xorg/Xorg                           133MiB |
    |    1      2481      G   compiz                                        87MiB |
    +-----------------------------------------------------------------------------+
  

• Here is the output from a test basecalling run:

    ONT Guppy basecalling software version 3.1.5+781ed57
    config file:        /opt/ont/guppy/data/dna_r9.4.1_450bps_hac.cfg
    model file:         /opt/ont/guppy/data/template_r9.4.1_450bps_hac.jsn
    input path:         /media/drive2/ONTdata/50
    save path:          /media/databk1/ONTdata_05242019-output/50
    chunk size:         1000
    chunks per runner:  1000
    records per file:   0
    num basecallers:    4
    gpu device:         cuda:0
    kernel path:
    runners per device: 2
  
    Found 4000 fast5 files to process.
    Init time: 1547 ms
  
    0%   10   20   30   40   50   60   70   80   90   100%
    |----|----|----|----|----|----|----|----|----|----|
    ***************************************************
    Caller time: 56515 ms, Samples called: 558584255, samples/s: 9.88382e+06
    Finishing up any open output files.
    Basecalling completed successfully.
  

colindaven/guppy_on_slurm#1 and this https://github.com/colindaven/guppy_on_slurm/blob/master/runbatch_gpu_guppy.sh

Let us know if you have further improvements/ideas.