Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DSL2: Make publication of large intermediate files opt-in #968

Open
scarlhoff opened this issue Mar 31, 2023 · 1 comment
Open

DSL2: Make publication of large intermediate files opt-in #968

scarlhoff opened this issue Mar 31, 2023 · 1 comment
Labels
DSL2 enhancement New feature or request

Comments

@scarlhoff
Copy link
Contributor

On March 31st, we decided to make the publication of all large output files that are used for down-stream processing, particularly bam files after mapping, etc., opt-in.

@scarlhoff scarlhoff added the enhancement New feature or request label Mar 31, 2023
@TCLamnidis TCLamnidis changed the title Make publication of large intermediate files opt-in DSL2: Make publication of large intermediate files opt-in Aug 15, 2023
@TCLamnidis
Copy link
Collaborator

The default output should only include BAMs for the following levels:

  • Deduplication output
  • Sample-level BAM (which goes in the final_bams dir)
  • (optional) Genotyping bams, if different from sample-level ones.

Opt-in processes (e.g. MAPDAMAGE_RESCALING) do not need opt-in to publish, since they are themselves opt-in.

Had a quick look through modules.config. I believe these are some pending changes for this:

  • SAMTOOLS_LENGTH_FILTER_INDEX|SAMTOOLS_FILTER_INDEX published indexes without checking params.bamfiltering_savefilteredbams
  • SAMTOOLS_{SORT,INDEX}_MERGE_LANES needs option to opt-in for keeping the lanemerged bams.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DSL2 enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants