Overview of processing CycIF images with Ashlar, Stitcher and omero import

cycif processing notes.md

Ashlar: Stitching and registration

Ashlar is the tool for turning individual tile images from all of your imaging cycles into a single large multi-channel image:

mkdir my_project
cd my_project
ashlar /path/to/cycle1_image /path/to/cycle2_image ...

This will leave you with a series of full-size TIFF files named cycle_A_B.tif where A is the cycle number and B is the channel number (both starting with 0). The input image files can be any multi-series imaging file with stage positions that BioFormats can read. RareCyte .rcnpnl and InCell .xdce files have both been verified to work.

As a quality control step, you can then run make_alignment_movie which will animate all the DAPI channel TIFFs in a video file named alignment.mp4. View this video in looping mode to quickly spot any stitching or alignment errors.

Stitcher: Conversion to OME-TIFF image pyramid format

Stitcher and its create_pyramid command is used to convert the TIFF files produced by Ashlar into a single .ome.tiff file that can be imported into OMERO:

sticher create_pyramid --input ./scan*tif --output my_project.ome.tiff --mergeChannels --overwrite

Yes, the command is currently named sticher by accident -- that's not a typo in the instructions. This may be fixed in a future version. Also, eventually the pyramid creation process will be performed by Ashlar itself and Stitcher will no longer be necessary.

Loading the image into OMERO

You can use the OMERO.insight GUI program to import your .ome.tiff file, or you can use the omero import command if you are working on the O2 cluster.

To use omero import, first go to https://omero.hms.harvard.edu/ and log in. Select the dataset where you want to store your image, or create a new dataset. Copy the Dataset ID from the "General" tab on the right side of the screen for use in the following command:

omero sessions login [email protected]:4064
omero import -d DATASET_ID my_project.ome.tiff

Run omero import -h for more options.

Using Ashlar and Stitcher on the O2 cluster

As a one-time setup, add this line to the end of your ~/.bashrc file, then log out and log back in:

export MODULEPATH=$MODULEPATH:/n/groups/lsp/o2/modulefiles

Then for each O2 login session where you want to use these tools, run this command once:

module load ashlar stitcher omero

Also make sure you do not run these tools on the login nodes! You must run them on a compute node by launching either an interactive job or batch job. To start an interactive job, run this command:

srun -p interactive -t 12:00:00 --mem=8G --pty bash

Now your prompt should look something like this:

username@compute-a-16-89:~$ 

and you can continue running the compute-intensive tools. Note that the runtime limit for interactive jobs is 12 hours, so if you need more time than that you'll need to re-run the srun command above.

Batch jobs may also be used but setting them up is outside the scope of this document. See the O2 documentation for details.

Transferring and storing data

The Research Computing staff suggests storing temporary input and output data on O2 in /n/scratch2 which is very fast and accessible from all compute nodes. First create a subdirectory under scratch2 using your ecommons username. Then ssh to transfer.orchestra.med.harvard.edu and use rsync to copy files from /files/ImStor (or wherever your original image files are stored) to /n/scratch2/YOUR_USERNAME. Log out of transfer when finished copying.

Each user can use up to 10 terabytes of space in /n/scratch2. Note that files there are not backed up and will be deleted if they are not accessed for 30 days. This shouldn't be a problem if you are just copying images from ImStor to process and uploading the results directly to OMERO, but keep it in mind if you end up storing other things there.