Skip to content

Dedup/nondedup outputs, spike-in fail-fast fix, and docs audit - #140

Merged
kopardev merged 7 commits into
mainfrom
issue_138
Sep 17, 2026
Merged

kopardev merged 7 commits into
mainfrom
issue_138

Conversation

@kopardev

@kopardev kopardev commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Changes

  • Generate Tn5 nicking-site and read counts matrices, and the corresponding DiffATAC/DESeq2 results, from both dedup.bam (PCR/optical duplicates removed) and filtered.bam (duplicates retained, labeled nondedup), written to separate dedup/nondedup output subfolders. dedup is recommended for standard differential accessibility testing.
  • Fail fast with a clear, actionable error message when a replicate has 0 reads aligned to the spike-in genome, instead of crashing with an opaque ZeroDivisionError in _compute_downsampling_scaling_factors.py. The error is now also captured in a dedicated results/spikein/compute_scaling_factors.log.
  • Documentation audit across docs/overview.md, docs/outputs.md, docs/introduction.md, docs/deployment.md, and docs/index.md:
    • Fix stale claims left over from the dedup/nondedup change.
    • Clarify how consensus peaks and ROIs are generated (why two rounds, which output to use for what).
    • Add ENCODE-referenced QC rule-of-thumb callouts (library complexity, TSS enrichment, FRiP).
    • Consolidate scattered spike-in "when should I use this" guidance into a single decision-tree callout.
    • Note the embedded aspen --help output in deployment.md is an illustrative snapshot.
    • Remove the outdated, unreferenced docs/extra.md page.
  • CHANGELOG.md updated with entries for all of the above under the development version heading.

Issues

Closes #138
Fixes #139

PR Checklist

(Strikethrough any points that are not applicable.)

  • This comment contains a description of changes with justifications, with any relevant issues linked.
  • Update docs if there are any API changes.
  • Update CHANGELOG.md with a short description of any user-facing changes and reference the PR number. Guidelines: https://keepachangelog.com/en/1.1.0/
  • Test run completes successfully on biowulf.

Test run confirmed on biowulf (/vf/users/Boufraqech_group/analysis/.temp/aspen_run_for_Ying_test2, 12 replicates, hs1_chrR, macs2 + genrich) at this PR's HEAD (58469c9). One replicate's align job hit the pre-existing 24h walltime limit in cluster.json and was killed by SLURM; after bumping the walltime and resubmitting (--rerun-incomplete), the run completed successfully end-to-end (53 of 53 remaining steps, exit code 0). The dedup/nondedup and spike-in fail-fast changes in this PR both worked as expected during validation. A separate reliability gap unrelated to this PR — Snakemake never detects SLURM jobs killed by timeout/OOM because ASPEN's cluster submission has no --cluster-status script — was filed as #141.

⚡ Generated using AI ⚡

Add plans/ alongside existing plans_reviews/ for local, untracked
planning docs (e.g. dedup vs filtered counts matrix plan, tracked in
issue #138).

⚡ Generated using AI ⚡
Generate Tn5 nicking-site and read counts matrices, and the corresponding
DiffATAC/DESeq2 results, from both dedup.bam (PCR/optical duplicates
removed) and filtered.bam (duplicates retained, labeled nondedup), written
to separate dedup/nondedup output subfolders. Previously only the
duplicate-retaining filtered.bam was used. dedup is recommended for
standard differential accessibility testing.

Refs #138

⚡ Generated using AI ⚡
Add an explicit check in _compute_downsampling_scaling_factors.py that
exits with an actionable error message when a replicate has 0 reads
aligned to the spike-in genome, instead of crashing with an opaque
ZeroDivisionError. The compute_scaling_factors rule now also captures
the script's stderr in a dedicated results/spikein/compute_scaling_factors.log
so the message isn't buried in the main snakemake log.

Fixes #139

⚡ Generated using AI ⚡
Audit and fix documentation across docs/overview.md, docs/outputs.md,
docs/introduction.md, docs/deployment.md, and docs/index.md:

- Fix stale claims left over from the dedup/nondedup change.
- Clarify how consensus peaks (pooled/consensus.bed/fixed-width ROI) and
  regions of interest are generated, with a why-two-rounds explanation
  and a table of which output to use for which downstream task.
- Add ENCODE-referenced QC rule-of-thumb callouts for library complexity
  (NRF/PBC1/PBC2), TSS enrichment, and FRiP.
- Consolidate scattered spike-in "when should I use this" guidance
  (previously split across overview.md and deployment.md) into a single
  decision-tree callout in overview.md, cross-linked from deployment.md.
- Note that the embedded `aspen --help` output in deployment.md is an
  illustrative, point-in-time snapshot.
- Remove the outdated, unreferenced docs/extra.md page; its content is
  superseded by docs/outputs.md.

Update CHANGELOG.md with entries for this docs pass and the spike-in
zero-reads fix.

Refs #138, #139

⚡ Generated using AI ⚡
The 1.3.0 heading was premature -- per repo convention the CHANGELOG
heading stays as 'development version' while VERSION is a -dev suffix,
and only gets renamed to the actual release version (e.g. 1.3.0) by the
draft-release automation when preparing the release.

⚡ Generated using AI ⚡
@kopardev kopardev added the generated-by-AI Content generated with AI assistance label Sep 11, 2026
@github-actions github-actions Bot added the ASPEN RepoName label Sep 11, 2026
@kopardev
kopardev marked this pull request as ready for review September 16, 2026 20:09
@kopardev
kopardev merged commit c873843 into main Sep 17, 2026
5 checks passed
@kopardev
kopardev deleted the issue_138 branch September 17, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ASPEN RepoName generated-by-AI Content generated with AI assistance

Projects

None yet

1 participant