seqkit sum should have a FASTQ sequencer header inclusive mode?

[tfallon-ionis]

Hi there,

Thanks for implementing seqkit sum some time back: #262

I've typically never needed to include the FASTQ sequence header in seqkit sum, but noticed that UMI extraction i.e. via UMI-tools, will shove that UMI sequence into the FASTQ sequence header. Thus, the sequence header contains useful sequence into in that sense, that should be optionally included in a seqkit sum content digest. What do you think?

Best,
-Tim

[shenwei356]

1. Sequence headers and qualities are skipped, only sequences matter.

It can optionally add the sequence ID to the sequence before computing the digest, and --id-regexp can be configured to capture anything you define, such as the UMI sequence, as the ID.

Then metadata needs to be changed; probably a <seq type> (such as S+I) should be added as well.

[tfallon-ionis]

Sounds good! I'm OK with the seq type being added to be explicit.

[shenwei356]

Added

We add meta information to the message digest, in the format of:

    seqkit.<version>_<seq type><seq structure><strand>[with seqid]_<kmer size>_<seq digest>

    <version>:       digest version
    <seq type>:      'D' for DNA, 'R' for RNA, 'P' for protein, 'N' for others
    <seq structure>: 'L' for linear sequence, 'C' for circular genome
    <strand>:        'D' for double-stranded, 'S' for single-stranded
    [with seqid]:	 'I' for including sequence ID, '' for not.
    <kmer size>:     0 for linear sequence, other values for circular genome

Flags:
  -i, --include-id           include the sequence ID (defined by --id-regexp) when computing the digest

Example

# Firstly, set a regular expression to capture the UMI in the sequence header
$ echo -ne ">r1_aaa\nactg\n" 
>r1_aaa
actg
$ echo -ne ">r1_aaa\nactg\n" | seqkit seq -i --id-regexp '^\S+_([ACTGactg]+)\s?'
>aaa
actg

# without id
$ echo -ne ">r1_aaa\nactg\n" | seqkit sum
seqkit.v0.1_DLS_k0_ebbf2222271114c348d63ba64f486e9e     -

# including ID (UMI here)
$ echo -ne ">r1_aaa\nactg\n" | seqkit sum -i --id-regexp '^\S+_([ACTGactg]+)\s?'
seqkit.v0.1_DLSI_k0_f6b68f6e41d91f0e93f900832304cdec    -

# including ID (UMI here)
# another read with a different sequence id, but the digest remains unchanged.
$ echo -ne ">r2_aaa\nactg\n" | seqkit sum -i --id-regexp '^\S+_([ACTGactg]+)\s?'
seqkit.v0.1_DLSI_k0_f6b68f6e41d91f0e93f900832304cdec    -

• seqkit_darwin_arm64.tar.gz
• seqkit_linux_amd64.tar.gz
• seqkit_windows_amd64.exe.tar.gz