SAND is Copyright (C) 2010- The University of Notre Dame. This software is distributed under the GNU General Public License. See the file COPYING for details.
SAND is a set of modules for accelerating genome assembly and other bioinformatics tasks. Using the Work Queue framework, SAND can distribute computational tasks to hundreds of nodes harnessed from clusters, clouds, grids, or just the idle machines you have in your office. SAND can be used as a drop-in replacement for the conventional overlapper in the Celera Assembler, or can be used as a standalone tool by advanced users.
SAND is part of the Cooperating Computing Tools. You can download the CCTools from this web page, follow the installation instructions, and you are ready to go.
Total | Workers | Tasks Avg | Candidates Time | Idle Busy | Submit Idle Run Done Time | Found 0 | 0 0 | 0 0 0 0 nan | 0 5 | 0 0 | 0 0 0 0 nan | 0 10 | 0 0 | 0 0 0 0 nan | 0 15 | 0 0 | 0 0 0 0 nan | 0 ...SAND is now waiting for you to start worker processes. Each worker that you start will connect back to the master process nd perform small pieces of the work at a time. In general, the more machines that you can harness, the faster the work will go.
For testing SAND, just open a new console window and start a single worker, specifying the hostname where the master runs and the port number it is listening on. (This can be changed with the sandPort option in the CA spec file.) For example:
% work_queue_worker master.somewhere.edu 9123With one worker, you will see some change in the output, like this:
Total | Workers | Tasks Avg | Candidates Time | Idle Busy | Submit Idle Run Done Time | Found 0 | 0 0 | 0 0 0 0 nan | 0 5 | 0 1 | 100 83 1 16 0.32 | 1858 10 | 0 1 | 100 69 1 30 0.33 | 3649 15 | 0 1 | 100 55 1 44 0.34 | 5464For a very small assembly, a single worker might be sufficient. However, to run a really large assembly at scale, you will need to run as many workers as possible. A simple (but tiresome) way of doing so is to ssh into lots of machines and manually run worker as above. But, if you have access to a batch system like Condor or SGE, you can use them to start many workers with a single submit command.
We have provided some simple scripts to make this easy. For example, to submit 10 workers to your local Condor pool:
% condor_submit_workers master.somewhere.edu 9123 10 Submitting job(s).......... Logging submit event(s).......... 10 job(s) submitted to cluster 298.Or, to submit 10 worker processes to your SGE cluster:
% sge_submit_workers master.somewhere.edu 9123 10 Your job 1054781 ("worker.sh") has been submitted Your job 1054782 ("worker.sh") has been submitted Your job 1054783 ("worker.sh") has been submitted ...Note that condor_submit_workers and sge_submit_workers are simple shell scripts, so you can edit them directly if you would like to change batch options or other details.
Once the workers begin running, the SAND modules can dispatch tasks to each one very quickly. It's ok if a machine running a worker crashes or is turned off; the work will be silently sent elsewhere to run.
When the SAND module's master process completes, your workers will still be available, so you can either run another master with the same workers, remove them from the batch system, or wait for them to expire. If you do nothing for 15 minutes, they will automatically exit.
% meryl -B -m 24 -C -L 100 -v -o small.meryl -s small.fa % meryl -Dt -s small.meryl -n 100 > small.repeatsThen use sand_compress_reads to compress the sequence data into a compressed FASTA (.cfa) file:
% sand_compress_reads small.fa small.cfaThe filtering step will read in the compressed sequence data (small.cfa) and quickly produce a list of candidate sequences (small.cand) for the following step to consider in detail. Start the filtering step as follows:
% sand_filter_master -r small.repeats small.cfa small.candWhile the filtering step runs, it will print some statistics to the console, showing the number of workers available, tasks running, and so forth:
Total | Workers | Tasks Avg | Candidates Time | Idle Busy | Submit Idle Run Done Time | Found 0 | 0 0 | 0 0 0 0 nan | 0 5 | 0 14 | 158 14 14 130 0.39 | 16452 10 | 0 15 | 356 15 15 326 0.38 | 42382 15 | 0 15 | 549 15 15 519 0.38 | 69055 20 | 0 15 | 744 15 15 714 0.38 | 96298 25 | 0 15 | 942 15 15 912 0.38 | 124284The alignment step will take the list of candidates generated in the previous step (small.cand), the compressed sequences (small.cfa) and produce a listing of how and where the sequences overlap (small.ovl). For example:
% sand_align_master sand_align_kernel -e "-q 0.04 -m 40" small.cand small.cfa small.ovlThe options -q 0.04 -m 40 passed to sand_align_kernel indicate a minimum alignment quality of 0.04 and a minimum alignment length of 40 bases. Again, a progress table will be printed to standard out:
Total | Workers | Tasks Avg | K-Cand K-Seqs | Total Time | Idle Busy | Submit Idle Run Done Time | Loaded Loaded | Speedup 0 | 0 0 | 0 0 0 0 0.00 | 0 0 | 0.00 8 | 0 48 | 100 52 48 0 0.00 | 1000 284 | 0.00 10 | 0 86 | 100 13 86 1 7.07 | 1000 284 | 0.71 36 | 1 83 | 181 14 83 2 19.47 | 1810 413 | 1.08 179 | 1 83 | 259 92 83 3 22.51 | 2590 1499 | 0.38 186 | 2 80 | 259 15 80 85 28.54 | 2590 1499 | 13.04 199 | 2 80 | 334 90 80 86 29.96 | 3340 1499 | 12.95 200 | 2 80 | 334 90 80 114 59.43 | 3340 1499 | 33.88 202 | 2 81 | 334 9 81 165 86.08 | 3340 1499 | 70.32After the sequence alignment step completes, you will have an overlap (.ovl) file that can be fed into the final stages of your assembler to complete the consensus step.