Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MarkDuplicates Input specificity and input refactor. #8118

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

johnoooh
Copy link
Contributor

@johnoooh johnoooh commented Mar 26, 2025

When using MarkDuplicates on an HPC with scratch enabled, the globbed *.bam copys both the input and output bam from scratch to the work directory. The output needed to be more specific. It is now ${prefix}.bam for the output so it now targets the output bam/cram instead of all files with the right file extension. For more details see the issue.

This should fix the issue for MarkDuplicates but be on the lookout for other modules where this occurs.

The second part of this may be more controversial but I found the bam/cram declaration to be a bit difficult to read. Prefix did not actually mean prefix here, it meant ${prefix}.bam. Because of this it required some logic later on to change this to cram. I cleared this up and hopefully made it easier to read.

PR checklist

Addresses ##7792, but more modules could have issues.

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

@johnoooh johnoooh requested a review from maxulysse as a code owner March 26, 2025 14:52
Copy link
Contributor

@SPPearce SPPearce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is another open PR to pipe the cram output straight from mark duplicates too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants