Sorting, Indexing, and Duplicate Marking

  • Sorting, indexing, and duplicate marking are essential operations in the processing of sequencing alignment files in bioinformatics. These operations are commonly performed on SAM, BAM, and CRAM files after sequencing reads have been aligned to a reference genome. They help organize alignment records, enable rapid access to specific genomic regions, identify potentially redundant sequencing reads, and prepare data for downstream analyses such as variant calling, genome coverage assessment, RNA sequencing, and structural-variant detection. Tools such as SAMtools, Picard, and GATK provide widely used methods for performing these operations.
  • When sequencing reads are aligned to a reference genome, the resulting alignment records may initially be arranged in the order in which reads were processed. This order is not necessarily suitable for genomic analysis. A file may contain alignments from different chromosomes and genomic positions in a mixed sequence. Many bioinformatics applications require alignment records to be organized according to their reference coordinates. Sorting arranges these records into a defined order, making it easier to retrieve reads from particular genomic regions and perform downstream processing.
  • There are several types of alignment sorting. Coordinate sorting arranges reads according to their reference sequence and genomic position. It is the most common sorting method for genomic analysis because it supports indexing and efficient regional access. Within a reference sequence, records are generally arranged according to their leftmost alignment positions. The ordering of reference sequences follows the sequence dictionary or reference order defined in the alignment header.
  • Query-name sorting arranges alignment records according to their read or template names. This groups records originating from the same sequencing template, which can be useful when processing paired-end reads or performing operations that require related reads to appear together. However, query-name-sorted files are not suitable for conventional coordinate-based indexing. Some tools also support different query-name sorting conventions, so users should follow the requirements of their chosen software.
  • Tag-based sorting is another possibility in which records are arranged according to selected alignment tags or other criteria. This is less common in routine genomic workflows but can be useful for specialized processing. The sorting method should always be selected according to the requirements of the downstream application. A file sorted by query name should not be treated as coordinate-sorted, and a file must generally be coordinate-sorted before conventional BAM or CRAM indexing.
  • SAMtools provides the sort command for sorting SAM, BAM, and CRAM alignment files. The following command sorts a BAM file by genomic coordinates:
  • samtools sort -@ 4 -o sample.sorted.bam sample.bam
  • The -@ 4 option specifies four additional computational threads, while -o defines the output filename. Coordinate sorting is the default behavior of samtools sort. The resulting file is suitable for indexing and many downstream genomic applications.
  • A SAM file can also be sorted directly and written as BAM:
  • samtools sort -@ 4 -O BAM -o sample.sorted.bam sample.sam
  • The -O BAM option specifies BAM output. SAMtools can also produce CRAM output when supplied with the appropriate reference:

samtools sort -@ 4 -O CRAM \

    -T reference_genome.fa \

    -o sample.sorted.cram sample.sam

  • The exact command options can vary with the SAMtools version. Users should consult the installed version’s documentation when working with CRAM output or reference-dependent operations.
  • Query-name sorting can be performed using:

samtools sort -n -@ 4 -o sample.name_sorted.bam sample.bam

  • The -n option sorts records by query name. SAMtools also supports a natural query-name sorting mode through -N, which handles numerical portions of names differently from ordinary lexical sorting. The appropriate option depends on the naming convention and the requirements of the downstream program.
  • Sorting may require substantial memory and temporary disk space, especially when processing whole-genome sequencing datasets. SAMtools can use temporary files during sorting when the data exceed available memory. The -m option controls the approximate maximum memory used by each sorting thread, while -T specifies a temporary-file prefix. For example:

samtools sort -@ 4 -m 2G \

    -T /tmp/sample_sort \

    -o sample.sorted.bam sample.bam

  • The memory option is applied per thread, so total memory use can be higher than the value specified with -m. Temporary storage should have sufficient available space to accommodate intermediate files. Sorting performance depends on file size, storage speed, available memory, number of threads, and compression settings.
  • Sorting is closely connected to indexing. An index is a data structure that allows bioinformatics software to retrieve alignment records from a selected genomic region without scanning the entire alignment file. Without an index, a program may need to read through a large proportion of the file to locate alignments at a particular chromosome and coordinate. Indexing therefore improves the speed of regional queries, genome-browser visualization, coverage calculations, and many variant-analysis operations.
  • BAM indexing commonly uses a BAI index file. The index stores information about the locations of alignment records in the compressed BAM file. It enables software to identify the file blocks that contain reads overlapping a requested genomic interval. A BAM file must generally be coordinate-sorted before a conventional BAI index can be created.
  • A BAM index can be generated using:

samtools index sample.sorted.bam

  • This command normally creates an index file named sample.sorted.bam.bai. The index is stored separately from the BAM file and should be kept alongside it. If the BAM file is renamed or moved, the index may also need to be renamed or moved so that software can locate it correctly.
  • A custom index filename can be specified using:

samtools index sample.sorted.bam sample.index.bai

  • For larger reference sequences or coordinate ranges that exceed the limits of the traditional BAI format, CSI indexing can be used:

samtools index -c sample.sorted.bam

  • This creates a CSI index, which supports larger coordinate ranges than BAI. The appropriate index type depends on the reference genome and the requirements of the software being used.
  • CRAM files can also be indexed after coordinate sorting. For example:

samtools index sample.sorted.cram

  • A CRAM index commonly uses the .crai extension. Like BAM indexing, CRAM indexing enables efficient access to selected genomic regions. The reference sequence may be needed when reading or indexing CRAM data, depending on the file and the operation. The correct reference should therefore be available and documented.
  • Once an alignment file has been sorted and indexed, reads can be retrieved from a specific genomic region. For example:
  • samtools view sample.sorted.bam chr1:100000-200000
  • This command displays alignment records overlapping the specified interval on chromosome 1. To save these records into another BAM file, use:

samtools view -b \

    -o chr1_region.bam \

    sample.sorted.bam chr1:100000-200000

  • Regional retrieval is particularly useful when inspecting candidate variants, examining gene-specific read coverage, evaluating splice junctions, and visualizing alignments in genome browsers. The index makes these operations substantially faster for large files.
  • Index files are linked to the alignment files from which they were generated. If the BAM or CRAM file is modified after indexing, the existing index may no longer represent the current file correctly. For this reason, sorting and other modifications should be completed before indexing. If an alignment file is changed, users should regenerate its index. An index should not be treated as an independent data file that can be reused with a different alignment file.
  • Duplicate marking is another important step in alignment processing. During sequencing library preparation, DNA fragments may be amplified using polymerase chain reaction (PCR). Multiple sequencing reads can therefore originate from copies of the same original DNA fragment. These reads are called PCR duplicates. If they are counted as independent observations, they may artificially increase apparent sequencing depth and create misleading evidence for genetic variants or other genomic features.
  • Duplicate reads are often identified by comparing their alignment positions and orientations. For paired-end sequencing, duplicate detection commonly considers the mapping positions and orientations of both reads in a pair. Reads with the same fragment boundaries and compatible orientations may be classified as duplicates. For single-end sequencing, duplicate identification generally relies on the alignment start position and strand, although the exact method depends on the software and available information.
  • Duplicate marking does not necessarily remove duplicate reads. Instead, it identifies them and records this information in the alignment file. In SAM/BAM/CRAM records, duplicate status is represented by the duplicate flag, 0x400 (decimal 1024). This allows downstream programs to exclude or handle marked reads without permanently deleting them. Marking is often preferred to immediate removal because the original sequencing evidence remains available for later inspection or alternative analysis.
  • PCR duplicates should be distinguished from optical duplicates. Optical duplicates can arise when the same cluster is detected more than once during imaging on certain sequencing platforms. Some duplicate-marking tools can identify optical duplicates using information encoded in read names and sequencing coordinates. The ability to identify them depends on the sequencing platform, read-name conventions, and software settings.
  • Picard MarkDuplicates is a widely used tool for identifying and marking duplicate reads in SAM, BAM, and CRAM workflows. It examines alignment records and identifies groups of reads that are likely to originate from the same original DNA fragment. A typical command for a coordinate-sorted BAM file is:

picard MarkDuplicates \

    I=sample.sorted.bam \

    O=sample.marked.bam \

    M=duplicate_metrics.txt

  • The I parameter specifies the input file, O defines the output file, and M specifies the duplicate metrics file. The output alignment file contains duplicate flags for records identified as duplicates. The metrics file provides information about the number and proportion of duplicate reads and other library-level statistics.
  • Picard can also remove duplicate records instead of marking them by using the REMOVE_DUPLICATES option:

picard MarkDuplicates \

    I=sample.sorted.bam \

    O=sample.deduplicated.bam \

    M=duplicate_metrics.txt \

    REMOVE_DUPLICATES=true

  • Removing duplicates is an irreversible change to the output file, although the original input file can be retained separately. It should not be performed automatically in every sequencing workflow. Some downstream analyses require duplicate information, while others need duplicate reads to remain available. The appropriate strategy depends on the experimental design and analysis objective.
  • SAMtools also provides duplicate-marking functionality through the markdup command. A common workflow first uses fixmate to prepare mate-related information in a name-sorted or name-collated file. The alignments are then coordinate-sorted before duplicate marking. For example:

samtools collate -@ 4 -o sample.collated.bam sample.bam

samtools fixmate -m sample.collated.bam sample.fixmate.bam

samtools sort -@ 4 -o sample.fixmate.sorted.bam sample.fixmate.bam

samtools markdup -@ 4 \

    sample.fixmate.sorted.bam \

    sample.marked.bam

  • The collate command groups records with the same query name without requiring a complete lexical name sort. The fixmate -m command updates mate-related information and adds the information required by markdup for duplicate detection. The file is then coordinate-sorted, and markdup identifies duplicate alignments. Exact input requirements should be checked against the SAMtools version being used.
  • SAMtools can also remove duplicates using the -r option:

samtools markdup -r \

    sample.fixmate.sorted.bam \

    sample.deduplicated.bam

  • This command removes records identified as duplicates rather than simply marking them. The original file should be retained if the possibility of revisiting the duplicate-handling decision is important.
  • Duplicate marking is especially important in whole-genome sequencing and whole-exome sequencing because PCR amplification can cause some DNA fragments to be represented many times. In variant calling, duplicate reads may create an artificial increase in read depth and can affect the confidence assigned to a candidate variant. Many variant-calling workflows therefore use duplicate-marked alignments and exclude or down-weight duplicate reads according to the caller’s settings.
  • In RNA-seq analysis, duplicate handling requires additional care. Highly expressed transcripts naturally generate many reads from the same genomic regions. These reads may share alignment positions even when they originate from different RNA molecules. Consequently, marking every positional duplicate as a PCR duplicate can incorrectly remove genuine biological signal. Unique molecular identifiers (UMIs), when incorporated into the library, provide additional information that can help distinguish independent molecules from amplification duplicates. UMI-aware tools such as UMI-tools use molecular barcodes and alignment information to identify likely duplicate molecules more accurately.
  • Duplicate marking is also relevant to targeted sequencing, amplicon sequencing, and single-cell sequencing, but the correct strategy varies. In amplicon-based workflows, many independent molecules may have identical start and end positions because primers define the fragment boundaries. Standard positional duplicate marking can therefore classify genuine reads as duplicates. UMI-based approaches or protocol-specific processing methods may be more appropriate. In single-cell sequencing, barcode and UMI information are often essential for distinguishing molecules and cells.
  • Duplicate metrics provide useful information about sequencing-library complexity. A high duplicate rate may indicate extensive PCR amplification, low library complexity, excessive sequencing depth relative to the number of unique molecules, or other technical issues. However, a high duplicate rate does not automatically mean that a library is poor quality. It can also occur in targeted sequencing, highly expressed RNA-seq samples, or experiments with a limited number of starting molecules. Duplicate metrics should therefore be interpreted in the context of the sequencing protocol and biological material.
  • After sorting, indexing, and duplicate marking, alignment files should be checked to ensure that the operations were completed successfully. SAMtools provides several useful commands for inspecting alignment files and their indexing status. For example:

samtools quickcheck -v sample.marked.bam

samtools flagstat sample.marked.bam

samtools stats sample.marked.bam > alignment_stats.txt

samtools view -c -f 1024 sample.marked.bam

  • The quickcheck command performs basic checks for file integrity, while flagstat summarizes alignment flags. The stats command produces more detailed alignment statistics. The final command counts records with the duplicate flag set. These checks can help confirm that the output file is readable and that duplicate marking has been recorded.
  • A typical alignment-processing workflow includes alignment, sorting, indexing, duplicate marking, and re-indexing. Because duplicate marking modifies the alignment file, the index should be generated after marking rather than relying on an index created for the unmodified file. A simplified workflow using SAMtools is:

# Sort the alignment file by genomic coordinates

samtools sort -@ 4 -o sample.sorted.bam sample.bam

# Group records by query name

samtools collate -@ 4 -o sample.collated.bam sample.sorted.bam

# Prepare mate information

samtools fixmate -m sample.collated.bam sample.fixmate.bam

# Sort the prepared alignment file by coordinates

samtools sort -@ 4 -o sample.fixmate.sorted.bam sample.fixmate.bam

# Mark duplicate reads

samtools markdup -@ 4 \

    sample.fixmate.sorted.bam \

    sample.marked.bam

# Index the final alignment file

samtools index sample.marked.bam

# Generate alignment statistics

samtools flagstat sample.marked.bam

samtools stats sample.marked.bam > sample.statistics.txt

  • This is an illustrative workflow rather than a complete pipeline for every sequencing experiment. The initial sorting and collation steps can be adapted to the input file and the SAMtools version. Some workflows also include read-group assignment, adapter trimming, base-quality recalibration, contamination screening, or other processing operations. The commands should be adjusted according to the experimental protocol and downstream software requirements.
  • Several technical considerations are important when performing these operations. First, coordinate sorting should use the same reference sequence dictionary and compatible reference naming conventions as the alignment file. Second, indexing should be performed only after the final coordinate-sorted alignment file has been created. Third, duplicate marking should be selected according to the library preparation method and sequencing technology. Fourth, duplicate metrics should be reviewed rather than interpreted as an automatic measure of sequencing quality. Finally, the original alignment file should be preserved when processing decisions may need to be revisited.
  • Reproducibility requires documenting the software names and versions, command-line parameters, reference genome assembly, sorting strategy, indexing format, and duplicate-marking settings. Researchers should also preserve the duplicate metrics and relevant library metadata. If UMI-based deduplication is used, the barcode extraction and error-correction procedures should be recorded. These details make it possible to understand how the final alignment files were generated and to reproduce the analysis.
  • Sorting, indexing, and duplicate marking are fundamental steps in the management of sequencing alignment files. Sorting organizes reads into an order suitable for genomic analysis, indexing enables rapid retrieval from selected genomic regions, and duplicate marking identifies reads that may originate from the same original DNA fragment. Together, these operations improve the efficiency, reliability, and reproducibility of bioinformatics workflows involving SAM, BAM, and CRAM files.
Author:

Leave a Reply

Your email address will not be published. Required fields are marked *