Skip to main content

cfdna ref-kmers

Count reference k-mer frequencies for genomic windows or groups.

Builds a reference-sequence background for downstream k-mer correction. It writes row-wise frequencies plus a row scaling factor for reconstructing counts downstream.

By default, the reference background represents both sequence orientations so it can correct end-motif labels from either fragment end. Each eligible reference k-mer contributes half its weight to the sequence read left-to-right from the reference and half to its reverse complement. A motif and its reverse complement therefore receive the same reference frequency. For a complete motif set, each genomic k-mer opportunity contributes a total weight of 1 across the output labels. With a motifs file, only motif labels present in the file receive their half contributions, so the row scaling factor is the total count assigned to the listed motifs or groups.

The selected window assignment controls whether overlapping k-mers contribute fractional counts (default) or full counts when overlapping with a certain proportion.

K-mers overlapping blacklisted bases or ambiguous N bases are not counted.

Limitations​

For end-motifs (see cfdna ends), beware that this command also includes k-mers at the edges of chromosomes and blacklisted regions, where one of the fragment's ends is not actually countable. The counts do thus not fully represent the "possible fragment ends", although the error is tiny in non-small windows compared to a per-fragment-length possible-fragments count.

Usage​

cfdna ref-kmers [OPTIONS] --ref-2bit <REF_2BIT> --output-dir <OUTPUT_DIR> --kmer-size <KMER_SIZE>

Options​

  • -h, --help

    Print help (see a summary with '-h')

Core​

  • -r, --ref-2bit <REF_2BIT>

    2bit reference genome file [path]

    E.g., "hg38.2bit" from UCSC ( https://hgdownload.cse.ucsc.edu/goldenpath/hg38/bigZips/hg38.2bit ).

  • -o, --output-dir <OUTPUT_DIR>

    Output directory for results [path]

  • --temp-dir <TEMP_DIR>

    Directory to use for temporary files [path]

    Moving the temporary file I/O to a local scratch directory can significantly reduce runtime. Commands that use --ref-2bit also copy the 2bit reference here to avoid shared-filesystem contention when running many jobs in parallel.

    Uniquely named temporary subdirectories are used, so the original contents are left alone.

    On HPC clusters, use a node-local scratch directory. Some clusters provide the path to node-local scratch through an environment variable such as $TMPDIR, $SLURM_TMPDIR, or $LOCAL_SCRATCH.

    When omitted, all temporary files are written under the output directory.

  • -x, --output-prefix <OUTPUT_PREFIX>

    Optional prefix for the output file (e.g., a reference genome name) [string]

    Leave empty to write the filename without a leading prefix.

    E.g., to allow storing packages for multiple reference genomes in the same directory.

    Produces the file as: <prefix>.ref_kmers.zarr

  • -t, --n-threads <N_THREADS>

    Number of threads to use (increases RAM usage) [integer]

    Defaults to the number of available CPU cores (-1).

    [default: auto]

  • -k, --kmer-size <KMER_SIZE>

    Size of the k-mers [integer]

    Without --motifs-file, the largest supported k-mer size is 27.

    With --motifs-file, larger k-mers can be counted because the command only tracks the selected k-mer subspace.

  • --orientation <ORIENTATION>

    Which sequence orientations contribute to each k-mer label [string]

    "both" gives half of each observation's weight to the reference-forward sequence and half to its reverse complement. This is required for fragment end-motif correction because end-motif labels read from each fragment end inward, making right-end labels reverse-complemented relative to reference coordinates.

    "reference-forward" counts only the sequence read left-to-right from the stored reference.

    Possible values:

    • both: Average the reference-forward k-mer and its reverse complement
    • reference-forward: Count only the k-mer read left-to-right from the stored reference

    [default: both]

  • --canonical

    Collapse each k-mer with its reverse complement [flag]

    Odd-sized k-mers are collapsed such that the middle base is A or C. Even-sized k-mers are collapsed to the lexicographically lowest motif.

  • --tile-size <TILE_SIZE>

    Size of tiles to process the reference in [integer]

    Chromosomes are processed in tiles of this size to reduce memory usage.

    [default: 10000000]

Windows (select max. one arg.)​

  • --by-size <BY_SIZE>

    Window definition: a fixed window size [integer]

    When no windowing is specified, the default is one global window.

  • --by-bed <BY_BED>

    Window definition: a BED file of windows [path]

  • --by-grouped-bed <BY_GROUPED_BED>

    Window definition: a BED file of grouped windows [path]

    Requires a fourth BED column with the group name.

    Windows with the same group name are aggregated together in the final output. The exact per-group output shape depends on the command.

Window Assignment​

  • --assign-by <ASSIGN_BY>

    How to assign k-mers to windows [string]

    Possible values: "count-overlap", "any", "all", "midpoint", or "proportion=<threshold>"

    "count-overlap": Count the fraction of k-mer bases overlapping each window.

    "any": Count the k-mer in every window overlapping at least one k-mer base.

    "all": Count the k-mer only in windows containing all k-mer bases.

    "midpoint": Count the k-mer in the window overlapping its center base. This mode requires an odd --kmer-size.

    "proportion=<threshold>": Count the k-mer when at least this fraction of its bases overlaps a window.

    [default: count-overlap]

Chromosome Selection (select max. one arg.)​

  • --chromosomes <CHROMOSOMES>...

    Names of chromosomes to process (comma-separated or repeated). E.g. 'chr1,chr2,chr3'.

    When no chromosomes are specified, it defaults to chr1..chr22.

    Specify "all" (as the only string) to use all chromosomes from the command's configured contig source.

  • --chromosomes-file <CHROMOSOMES_FILE>

    File with chromosome names to process (one per line)

Filtering​

  • -b, --blacklist <BLACKLIST>...

    Optional BED file(s) with blacklisted regions [path]

    K-mers overlapping a blacklisted base are not counted.

Motifs​

  • --all-motifs

    Include every possible motif in the output, even if its count is zero [flag]

    NOTE: When --motifs-file is specified, it defines the "possible" motifs.

  • --motifs-file <MOTIFS_FILE>

    File with motifs to include [path]

    TSV-like file (tab-separated, no header) with one motif per line. Add a second column with a group name to count multiple motifs together.

    Each motif must be an A/C/G/T k-mer of length --kmer-size. For compatibility with cfdna ends, a single _ separator is accepted and removed, so AC_GT is read as ACGT.

    In grouped mode, the output motif axis contains one entry per distinct group name, ordered alphabetically by group name.

    Group membership comes only from the motifs file. Reverse complements are not automatically added to the same group. With --orientation both, reverse-complement labels receive the same averaged count. For example, if the left-to-right reference counts are AACC=3 and GGTT=5: - If the file contains only AACC<TAB>group_a, group_a gets count 4. - If both motifs are assigned to group_a, group_a gets count 8. - If AACC is assigned to group_a and GGTT to group_b, each group gets count 4. A motif that is its own reverse complement keeps its full count.

    Frequencies are normalized over the motifs or groups listed in this file. Unlisted k-mers are not part of the denominator. The row scaling factor stores the total count assigned to the motifs or groups listed in the file, so reconstructed counts represent those motifs or groups.

    Specifying the allowed subset of motifs beforehand enables counting of much larger k-mers without enumerating every A/C/G/T k-mer of that length.

Logging​

  • --log <LOG>

    Logging destination [stdout|quiet|file|file=<path>]

    stdout keeps the normal run narrative on standard output.

    quiet suppresses the normal run narrative and progress bars, while warnings and errors still go to stderr.

    file writes the normal run narrative to an auto-generated log file under the command output directory.

    file=<path> writes the normal run narrative to the exact path you provide.

    [default: stdout]