Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Initial release of nf-core/spatialaxe, created with the [nf-core](https://nf-co.

### `Added`

- Added the entire **spoQC tool and subworkflow**: `subworkflows/local/spoqc/main.nf` plus 31 new local modules under `modules/local/spoQC/` (`ambient`, `analysis_category`, `analysis_cluster`, `analysis_overview`, `annotation`, `bubble`, `cell`, `cellcycle`, `combine_masks`, `doublet`, `finalreport`, `general`, `hqcr_celltype`, `hqcr_ident`, `hqpr_bounding_box`, `hqpr_celltype`, `hqpr_clustering`, `hqpr_metrices`, `hqpr_refinement`, `hqtr_ac`, `hqtr_bounding_box`, `hqtr_celltype`, `hqtr_clustering`, `hqtr_metrices`, `hqtr_qv`, `hqtr_refinement`, `marker`, `model`, `transcript`, `void`, `whole_slide`), each with its own `main.nf`, `meta.yml`, and nf-test suite (`tests/main.nf.test` + snapshot).
- Added the entire **spoQC tool and subworkflow**: `subworkflows/local/spoqc/main.nf` plus 30 new local modules under `modules/local/spoQC/` (`ambient`, `analysis_category`, `analysis_cluster`, `analysis_overview`, `annotation`, `cell`, `cellcycle`, `combine_masks`, `doublet`, `finalreport`, `general`, `hqcr_celltype`, `hqcr_ident`, `hqpr_bounding_box`, `hqpr_celltype`, `hqpr_clustering`, `hqpr_metrices`, `hqpr_refinement`, `hqtr_ac`, `hqtr_bounding_box`, `hqtr_celltype`, `hqtr_clustering`, `hqtr_metrices`, `hqtr_qv`, `hqtr_refinement`, `marker`, `model`, `transcript`, `void`, `whole_slide`), each with its own `main.nf`, `meta.yml`, and nf-test suite (`tests/main.nf.test` + snapshot).
- Wired spoQC into `workflows/spatialaxe.nf`.
- Added `bin/spatialdata_write.py` support for spoQC's SpatialData output.
- New test configs: `conf/tests/test_spoqc.config` and `conf/tests/test_full_spoqc.config`
Expand Down
4 changes: 3 additions & 1 deletion conf/test.config
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,7 @@ params {

// Input data
input = "${projectDir}/assets/samplesheet.csv"
mode = 'coordinate'
//mode = 'coordinate'
mode = 'qc'
spoqc = true
}
16 changes: 8 additions & 8 deletions docs/output.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,6 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
- `unsupervised_cell_annotation.tsv` unsupervised cell-type annotation, generated only when no annotation file was supplied as input
- `whole_slide_qc/` whole-slide QC overview report
- `generalqc/` general-purpose QC report
- `bubbleqc/` bubble-artifact QC report
- `doubletqc/` doublet-detection QC report
- `voidqc/` void/empty-region QC report
- `cellqc/` cell-level QC report
Expand Down Expand Up @@ -170,16 +169,17 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
- `staining_log.txt` log of the stainings processed by spoQC
- `report.html` final self-contained HTML report aggregating every spoQC step above
- `spoQC_tmp/` intermediate data consumed by later spoQC steps
- `generalqc_output_hqcr.parquet`, `bubbleqc_output_hqcr.parquet`, `doubletqc_output_hqcr.parquet`, `voidqc_output_hqcr.parquet`, `cellqc_output_hqcr.parquet` per-step HQCR contribution scores
- `generalqc_output_hqcr.parquet`, `doubletqc_output_hqcr.parquet`, `voidqc_output_hqcr.parquet`, `cellqc_output_hqcr.parquet` per-step HQCR contribution scores
- `ambient_output_genes.parquet` ambient RNA gene-signal estimate
- `hqcr_output_mask_raw.parquet` / `hqcr_output_mask_smoothed_raw.parquet` combined raw/smoothed HQCR mask
- `hqcr_output_mask_smoothed_celltype_refined.parquet` cell-type-refined smoothed HQCR mask
- `mask_raw_output_hqcr.parquet` / `mask_smoothed_raw_output_hqcr.parquet` combined raw/smoothed HQCR mask
- `traffic_light_output_hqcr.parquet` HQCR traffic-light classification
- `mask_smoothed_celltype_refined_output_hqcr.parquet` cell-type-refined smoothed HQCR mask
- `metrices/hqpr/${staining}/` per-staining HQPR metrics
- `hqpr_${staining}_output_mask_raw/` / `hqpr_${staining}_output_mask_smoothed_raw/` per-staining raw/smoothed HQPR mask
- `mask_raw_output_hqpr_${staining}/` / `mask_smoothed_raw_output_hqpr_${staining}/` per-staining raw/smoothed HQPR mask
- `metrices/hqtr/` HQTR metrics
- `hqtr_output_ac_prob/` HQTR ambient-contamination probabilities
- `hqtr_output_qv_prob/` HQTR quality-value probabilities
- `hqtr_output_mask_raw/` / `hqtr_output_mask_smoothed_raw/` raw/smoothed HQTR mask
- `ac_density_output_hqtr/` HQTR ambient-contamination probabilities
- `qv_density_output_hqtr/` HQTR quality-value probabilities
- `mask_raw_output_hqtr/` / `mask_smoothed_raw_output_hqtr/` raw/smoothed HQTR mask
- `multiqc/`
- `multiqc_report.html`: a standalone HTML file that can be viewed in your web browser.
- `multiqc_data/`: directory containing parsed statistics from the different tools used in the pipeline.
Expand Down
2 changes: 1 addition & 1 deletion modules/local/spoQC/ambient/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ process SPOQC_AMBIENT {
label 'process_mid_time'
label 'spoqc'

container "heylf/spoqc_dev:0.1.0"
container "heylf/spoqc_dev:0.1.1"

input:
tuple val(meta), path(spatialdata, stageAs: "*")
Expand Down
16 changes: 7 additions & 9 deletions modules/local/spoQC/analysis_category/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,23 @@ process SPOQC_ANALYSIS_CATEGORY {
label 'spoqc'


container "heylf/spoqc_dev:0.1.0"
container "heylf/spoqc_dev:0.1.1"

input:
tuple val(meta), path(spatialdata, stageAs: "*")
path(annotation, stageAs: "*")
val(step)
path(tmp_general, stageAs: "spoQC_tmp/generalqc_output_hqcr.parquet")
path(tmp_bubble, stageAs: "spoQC_tmp/bubbleqc_output_hqcr.parquet")
path(tmp_doublet, stageAs: "spoQC_tmp/doubletqc_output_hqcr.parquet")
path(tmp_void, stageAs: "spoQC_tmp/voidqc_output_hqcr.parquet")
path(tmp_cell, stageAs: "spoQC_tmp/cellqc_output_hqcr.parquet")
path(mask_hqcr, stageAs: "spoQC_tmp/hqcr_output_mask_raw.parquet")
path(mask_smoothed_hqcr, stageAs: "spoQC_tmp/hqcr_output_mask_smoothed_raw.parquet")
path(traffic_light_hqcr, stageAs: "spoQC_tmp/traffic_light_output_hqcr.parquet")
path(qv, stageAs: "spoQC_tmp/hqtr_output_qv_prob")
path(ac, stageAs: "spoQC_tmp/hqtr_output_ac_prob")
path(mask_hqcr, stageAs: "spoQC_tmp/mask_raw_output_hqcr.parquet")
path(mask_smoothed_hqcr, stageAs: "spoQC_tmp/mask_smoothed_raw_output_hqcr.parquet")
path(qv, stageAs: "spoQC_tmp/qv_density_output_hqtr")
path(ac, stageAs: "spoQC_tmp/ac_density_output_hqtr")
path(metrices_hqtr, stageAs: "spoQC_tmp/metrices/hqtr")
path(mask_smoothed_hqtr, stageAs: "spoQC_tmp/hqtr_output_mask_smoothed_raw")
path(mask_hqtr, stageAs: "spoQC_tmp/hqtr_output_mask_raw")
path(mask_smoothed_hqtr, stageAs: "spoQC_tmp/mask_smoothed_raw_output_hqtr")
path(mask_hqtr, stageAs: "spoQC_tmp/mask_raw_output_hqtr")
path(metrices_hqpr, stageAs: "spoQC_tmp/metrices/hqpr/*")
path(mask_smoothed_hqpr, stageAs: "spoQC_tmp/*")
path(mask_hqpr, stageAs: "spoQC_tmp/*")
Expand Down
17 changes: 6 additions & 11 deletions modules/local/spoQC/analysis_category/meta.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,6 @@ input:
description: General QC parquet output from SPOQC_GENERAL
pattern: "generalqc_output_hqcr.parquet"
ontologies: []
- tmp_bubble:
type: file
description: Bubble QC parquet output from SPOQC_BUBBLE
pattern: "bubbleqc_output_hqcr.parquet"
ontologies: []
- tmp_doublet:
type: file
description: Doublet QC parquet output from SPOQC_DOUBLET
Expand All @@ -61,12 +56,12 @@ input:
- mask_hqcr:
type: file
description: HQCR raw mask from SPOQC_HQCR_IDENT
pattern: "hqcr_output_mask_raw.parquet"
pattern: "mask_raw_output_hqcr.parquet"
ontologies: []
- mask_smoothed_hqcr:
type: file
description: Smoothed HQCR mask from SPOQC_HQCR_IDENT
pattern: "hqcr_output_mask_smoothed_raw.parquet"
pattern: "mask_smoothed_raw_output_hqcr.parquet"
ontologies: []
- traffic_light_hqcr:
type: file
Expand All @@ -76,12 +71,12 @@ input:
- qv:
type: directory
description: HQTR quality-value probability output from SPOQC_HQTR_QV
pattern: "hqtr_output_qv_prob"
pattern: "qv_density_output_hqtr"
ontologies: []
- ac:
type: directory
description: HQTR ambient-contamination probability output from SPOQC_HQTR_AC
pattern: "hqtr_output_ac_prob"
pattern: "ac_density_output_hqtr"
ontologies: []
- metrices_hqtr:
type: directory
Expand All @@ -91,12 +86,12 @@ input:
- mask_smoothed_hqtr:
type: directory
description: Smoothed HQTR mask from SPOQC_HQTR_REFINEMENT
pattern: "hqtr_output_mask_smoothed_raw"
pattern: "mask_smoothed_raw_output_hqtr"
ontologies: []
- mask_hqtr:
type: directory
description: HQTR raw mask from SPOQC_HQTR_CLUSTERING
pattern: "hqtr_output_mask_raw"
pattern: "mask_raw_output_hqtr"
ontologies: []
- metrices_hqpr:
type: directory
Expand Down
73 changes: 29 additions & 44 deletions modules/local/spoQC/analysis_category/tests/main.nf.test
Original file line number Diff line number Diff line change
Expand Up @@ -56,16 +56,6 @@ nextflow_process {
}
}

run("SPOQC_BUBBLE") {
script "modules/local/spoQC/bubble/main.nf"
process {
"""
input[0] = SPATIALDATA_WRITE.out.spatialdata
input[1] = "bubbleqc"
"""
}
}

run("SPOQC_DOUBLET") {
script "modules/local/spoQC/doublet/main.nf"
process {
Expand Down Expand Up @@ -104,10 +94,9 @@ nextflow_process {
input[0] = SPATIALDATA_WRITE.out.spatialdata
input[1] = "hqcr_ident"
input[2] = SPOQC_GENERAL.out.tmp.map { it[1] }
input[3] = SPOQC_BUBBLE.out.tmp.map { it[1] }
input[4] = SPOQC_DOUBLET.out.tmp.map { it[1] }
input[5] = SPOQC_VOID.out.tmp.map { it[1] }
input[6] = SPOQC_CELL.out.tmp.map { it[1] }
input[3] = SPOQC_DOUBLET.out.tmp.map { it[1] }
input[4] = SPOQC_VOID.out.tmp.map { it[1] }
input[5] = SPOQC_CELL.out.tmp.map { it[1] }
"""
}
}
Expand Down Expand Up @@ -219,21 +208,19 @@ nextflow_process {
input[1] = SPOQC_ANNOTATION.out.annotation.map { it[1] }
input[2] = "analysis_category"
input[3] = SPOQC_GENERAL.out.tmp.map { it[1] }
input[4] = SPOQC_BUBBLE.out.tmp.map { it[1] }
input[5] = SPOQC_DOUBLET.out.tmp.map { it[1] }
input[6] = SPOQC_VOID.out.tmp.map { it[1] }
input[7] = SPOQC_CELL.out.tmp.map { it[1] }
input[8] = SPOQC_HQCR_IDENT.out.mask.map { it[1] }
input[9] = SPOQC_HQCR_IDENT.out.mask_smoothed.map { it[1] }
input[10] = SPOQC_HQCR_IDENT.out.traffic_light.map { it[1] }
input[11] = SPOQC_HQTR_QV.out.tmp.map { it[1] }
input[12] = SPOQC_HQTR_AC.out.tmp.map { it[1] }
input[13] = SPOQC_HQTR_METRICES.out.metrices.map { it[1] }
input[14] = SPOQC_HQTR_REFINEMENT.out.mask_smoothed.map { it[1] }
input[15] = SPOQC_HQTR_CLUSTERING.out.mask.map { it[1] }
input[16] = SPOQC_HQPR_METRICES.out.metrices.map { _meta, _staining, p -> p }.collect()
input[17] = SPOQC_HQPR_REFINEMENT.out.mask_smoothed.map { _meta, _staining, p -> p }.collect()
input[18] = SPOQC_HQPR_CLUSTERING.out.mask.map { _meta, _staining, p -> p }.collect()
input[4] = SPOQC_DOUBLET.out.tmp.map { it[1] }
input[5] = SPOQC_VOID.out.tmp.map { it[1] }
input[6] = SPOQC_CELL.out.tmp.map { it[1] }
input[7] = SPOQC_HQCR_IDENT.out.mask.map { it[1] }
input[8] = SPOQC_HQCR_IDENT.out.mask_smoothed.map { it[1] }
input[9] = SPOQC_HQTR_QV.out.tmp.map { it[1] }
input[10] = SPOQC_HQTR_AC.out.tmp.map { it[1] }
input[11] = SPOQC_HQTR_METRICES.out.metrices.map { it[1] }
input[12] = SPOQC_HQTR_REFINEMENT.out.mask_smoothed.map { it[1] }
input[13] = SPOQC_HQTR_CLUSTERING.out.mask.map { it[1] }
input[14] = SPOQC_HQPR_METRICES.out.metrices.map { _meta, _staining, p -> p }.collect()
input[15] = SPOQC_HQPR_REFINEMENT.out.mask_smoothed.map { _meta, _staining, p -> p }.collect()
input[16] = SPOQC_HQPR_CLUSTERING.out.mask.map { _meta, _staining, p -> p }.collect()
"""
}
}
Expand All @@ -258,21 +245,19 @@ nextflow_process {
input[1] = SPOQC_ANNOTATION.out.annotation.map { it[1] }
input[2] = "analysis_category"
input[3] = SPOQC_GENERAL.out.tmp.map { it[1] }
input[4] = SPOQC_BUBBLE.out.tmp.map { it[1] }
input[5] = SPOQC_DOUBLET.out.tmp.map { it[1] }
input[6] = SPOQC_VOID.out.tmp.map { it[1] }
input[7] = SPOQC_CELL.out.tmp.map { it[1] }
input[8] = SPOQC_HQCR_IDENT.out.mask.map { it[1] }
input[9] = SPOQC_HQCR_IDENT.out.mask_smoothed.map { it[1] }
input[10] = SPOQC_HQCR_IDENT.out.traffic_light.map { it[1] }
input[11] = SPOQC_HQTR_QV.out.tmp.map { it[1] }
input[12] = SPOQC_HQTR_AC.out.tmp.map { it[1] }
input[13] = SPOQC_HQTR_METRICES.out.metrices.map { it[1] }
input[14] = SPOQC_HQTR_REFINEMENT.out.mask_smoothed.map { it[1] }
input[15] = SPOQC_HQTR_CLUSTERING.out.mask.map { it[1] }
input[16] = SPOQC_HQPR_METRICES.out.metrices.map { _meta, _staining, p -> p }.collect()
input[17] = SPOQC_HQPR_REFINEMENT.out.mask_smoothed.map { _meta, _staining, p -> p }.collect()
input[18] = SPOQC_HQPR_CLUSTERING.out.mask.map { _meta, _staining, p -> p }.collect()
input[4] = SPOQC_DOUBLET.out.tmp.map { it[1] }
input[5] = SPOQC_VOID.out.tmp.map { it[1] }
input[6] = SPOQC_CELL.out.tmp.map { it[1] }
input[7] = SPOQC_HQCR_IDENT.out.mask.map { it[1] }
input[8] = SPOQC_HQCR_IDENT.out.mask_smoothed.map { it[1] }
input[9] = SPOQC_HQTR_QV.out.tmp.map { it[1] }
input[10] = SPOQC_HQTR_AC.out.tmp.map { it[1] }
input[11] = SPOQC_HQTR_METRICES.out.metrices.map { it[1] }
input[12] = SPOQC_HQTR_REFINEMENT.out.mask_smoothed.map { it[1] }
input[13] = SPOQC_HQTR_CLUSTERING.out.mask.map { it[1] }
input[14] = SPOQC_HQPR_METRICES.out.metrices.map { _meta, _staining, p -> p }.collect()
input[15] = SPOQC_HQPR_REFINEMENT.out.mask_smoothed.map { _meta, _staining, p -> p }.collect()
input[16] = SPOQC_HQPR_CLUSTERING.out.mask.map { _meta, _staining, p -> p }.collect()
"""
}
}
Expand Down
15 changes: 7 additions & 8 deletions modules/local/spoQC/analysis_cluster/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,24 @@ process SPOQC_ANALYSIS_CLUSTER {
label 'spoqc'


container "heylf/spoqc_dev:0.1.0"
container "heylf/spoqc_dev:0.1.1"

input:
tuple val(meta), path(spatialdata, stageAs: "*")
path(annotation, stageAs: "*")
val(step)
path(tmp_general, stageAs: "spoQC_tmp/generalqc_output_hqcr.parquet")
path(tmp_bubble, stageAs: "spoQC_tmp/bubbleqc_output_hqcr.parquet")
path(tmp_doublet, stageAs: "spoQC_tmp/doubletqc_output_hqcr.parquet")
path(tmp_void, stageAs: "spoQC_tmp/voidqc_output_hqcr.parquet")
path(tmp_cell, stageAs: "spoQC_tmp/cellqc_output_hqcr.parquet")
path(mask_hqcr, stageAs: "spoQC_tmp/hqcr_output_mask_raw.parquet")
path(mask_smoothed_hqcr, stageAs: "spoQC_tmp/hqcr_output_mask_smoothed_raw.parquet")
path(mask_hqcr, stageAs: "spoQC_tmp/mask_raw_output_hqcr.parquet")
path(mask_smoothed_hqcr, stageAs: "spoQC_tmp/mask_smoothed_raw_output_hqcr.parquet")
path(traffic_light_hqcr, stageAs: "spoQC_tmp/traffic_light_output_hqcr.parquet")
path(qv, stageAs: "spoQC_tmp/hqtr_output_qv_prob")
path(ac, stageAs: "spoQC_tmp/hqtr_output_ac_prob")
path(qv, stageAs: "spoQC_tmp/qv_density_output_hqtr")
path(ac, stageAs: "spoQC_tmp/ac_density_output_hqtr")
path(metrices_hqtr, stageAs: "spoQC_tmp/metrices/hqtr")
path(mask_smoothed_hqtr, stageAs: "spoQC_tmp/hqtr_output_mask_smoothed_raw")
path(mask_hqtr, stageAs: "spoQC_tmp/hqtr_output_mask_raw")
path(mask_smoothed_hqtr, stageAs: "spoQC_tmp/mask_smoothed_raw_output_hqtr")
path(mask_hqtr, stageAs: "spoQC_tmp/mask_raw_output_hqtr")
path(metrices_hqpr, stageAs: "spoQC_tmp/metrices/hqpr/*")
path(mask_smoothed_hqpr, stageAs: "spoQC_tmp/*")
path(mask_hqpr, stageAs: "spoQC_tmp/*")
Expand Down
22 changes: 11 additions & 11 deletions modules/local/spoQC/analysis_cluster/meta.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,6 @@ input:
description: General QC parquet output from SPOQC_GENERAL
pattern: "generalqc_output_hqcr.parquet"
ontologies: []
- tmp_bubble:
type: file
description: Bubble QC parquet output from SPOQC_BUBBLE
pattern: "bubbleqc_output_hqcr.parquet"
ontologies: []
- tmp_doublet:
type: file
description: Doublet QC parquet output from SPOQC_DOUBLET
Expand All @@ -63,12 +58,17 @@ input:
- mask_hqcr:
type: file
description: HQCR raw mask from SPOQC_HQCR_IDENT
pattern: "hqcr_output_mask_raw.parquet"
pattern: "mask_raw_output_hqcr.parquet"
ontologies: []
- mask_smoothed_hqcr:
type: file
description: Smoothed HQCR mask from SPOQC_HQCR_IDENT
pattern: "hqcr_output_mask_smoothed_raw.parquet"
pattern: "mask_smoothed_raw_output_hqcr.parquet"
ontologies: []
- traffic_light_hqcr:
type: file
description: HQCR traffic-light classification from SPOQC_HQCR_IDENT
pattern: "traffic_light_output_hqcr.parquet"
ontologies: []
- traffic_light_hqcr:
type: file
Expand All @@ -78,12 +78,12 @@ input:
- qv:
type: directory
description: HQTR quality-value probability output from SPOQC_HQTR_QV
pattern: "hqtr_output_qv_prob"
pattern: "qv_density_output_hqtr"
ontologies: []
- ac:
type: directory
description: HQTR ambient-contamination probability output from SPOQC_HQTR_AC
pattern: "hqtr_output_ac_prob"
pattern: "ac_density_output_hqtr"
ontologies: []
- metrices_hqtr:
type: directory
Expand All @@ -93,12 +93,12 @@ input:
- mask_smoothed_hqtr:
type: directory
description: Smoothed HQTR mask from SPOQC_HQTR_REFINEMENT
pattern: "hqtr_output_mask_smoothed_raw"
pattern: "mask_smoothed_raw_output_hqtr"
ontologies: []
- mask_hqtr:
type: directory
description: HQTR raw mask from SPOQC_HQTR_CLUSTERING
pattern: "hqtr_output_mask_raw"
pattern: "mask_raw_output_hqtr"
ontologies: []
- metrices_hqpr:
type: directory
Expand Down
Loading
Loading