v1.0.0 (#23)

* Adjusted the samplesheet check for the pipeline

* Added splitting of the BED files

* Added haplotypecaller and bcftools concat

* Rearranged some processes

* Added family id to the input samplesheet

* Added joint-genotyping

* added ensembl vep (WIP)

* some template adjustments

* adjusted input parameters and added createsequencedictionary

* adjusted input parameters and added createsequencedictionary

* updated templates

* Updated all documentation

* Preparation for adding dragstr models

* small redundancy fix

* added optional fasta index generation

* added optional fasta index generation

* added dragstrmodels

* added use_dragstr_model to nextflow_schema.json

* small simplification

* small simplification

* Added versions output and added every module to modules.config

* added an HPC samplesheet

* Added VCF QC

* Added multiQC

* Added multiQC

* Added parameters to all tools

* Added reblockGVCF

* Added VEP plugins

* Added VEP plugins

* Added EOG to ensemblVEP

* Added the option to use a local merged cache with VEP

* small parameter fix

* small parameter fix

* modules config fix

* adjusted the CPUs for calibratedragstrmodel

* Added PEDIGREE and SAMPLE metadata to the VCFs

* removed a view()

* fix merge_vcf_headers output

* uncomment some code

* Family ID is now fetched from the PED file

* Added some missing version annotations

* prettier

* prettier

* changed input parsing to multiMap

* changed input parsing to multiMap

* renamed genotype to postprocess
fixed an issue with ensemblvep when no plugins were used

* removed some print statements

* added post-processing for seqplorer

* Added output folders for some steps

* Added vcf2db for seqplorer compatibility

* added temporary seqr support (to be discussed further)

* changed the name of the pipeline to nf-cmgg-germline

* typo

* updated logos for the new name

* added some error messages
refactored the code

* fixes for github actions

* prettier

* linting fix?

* linting fix?

* removed editorconfig

* prettier

* fixed the CI action

* fixed the CI action

* fixed ped path

* fixed ped path

* fixed ped path?

* fix for the PED file not being recognized by samplesheet_check

* small update to the samplesheet

* updated nextflow version of the CI action

* fix for CI disk space?

* fix for CI disk space?

* Updated GitHub Actions

* used python 3.9 everywhere

* fixed a tab being 2 spaces in the PED file

* updated the documentation

* linting

* Added a test_local config profile

* updates to local conf

* updated the validity checks for PED files

* small update to the PED validation

* added optional vcfanno annotation

* prettier

* updated the test profiles

* updated the strtablefile location in test.config

* updated vcfanno

* linting

* Updated the publishDir (WIP)

* Fixed the multiqc reports location bug

* Added a simple version of a metro graph

* Added the functionality for multiple crams per sample (#7)

* Added the functionality for multiple crams per sample

* linting

* small update to the test

* updated the HPC samplesheets

* Added an optional parameter to convert the bam to cram after merge

* Optional skip genotyping parameter (#8)

* added optional parameter to skip genotyping

* changed the test data

* updated metro map and test config

* linting

* updated the local tests

* removed the textual summary

* Bcftools merge vs combinegvcfs (#11)

* small update to the README

* made a test version

* added optional cram indexing if the index is not supplied

* updated gitignore

* removed a view statement

* small fix to the error handling of merge vcf headers

* properly added bclconvert

* linting

* changed the filter expression for bcftools view

* some minor overall improvements

* some minor overall improvements

* some minor overall improvements

* temporarily commented automatic vep version assignment

* added the new parameters to the schema

* updated the metro map for the new changes

* General improvements (#12)

* add a script to quickly remove output files (for easier hpc transfers)

* updated the parameter parsing

* removed the script again cuz ugh

* updated the test.config

* added vcfanno channel support

* refactored the missing reference files handling

* test updates

* parameter updates

* simplified optional input file creation

* test update

* test update

* updated optional parameter handling

* Updated parameter descriptions

* updated the help command

* moved a duplicate process

* refactoring

* updated the documentation

* linting

* added comments to modules.config (#15)

* added a family field + rework of input check (#17)

* added a family field + rework of input check

* prettier

* nf-test  (#20)

* Template update for nf-core/tools version 2.5.1

* file save updates

* changes to the template

* Template update for nf-core/tools version 2.5.1

* updates to template

* added index for seqr output

* updated the modules

* first try on adding nf-test

* Try out for nf-test part 2

* Template update for nf-core/tools version 2.5.1

* file save updates

* changes to the template

* Template update for nf-core/tools version 2.5.1

* updates to template

* added index for seqr output

* updated the modules

* first try on adding nf-test

* CI tests update

* linting test fix

* ci tests update

* ci tests update

* Last improvements (#22)

* Template update for nf-core/tools version 2.5.1

* file save updates

* changes to the template

* Template update for nf-core/tools version 2.5.1

* updates to template

* added index for seqr output

* updated the modules

* first try on adding nf-test

* CI tests update

* linting test fix

* ci tests update

* ci tests update

* push to test nf-test

* CI update

* CI update

* CI update

* CI update

* CI update

* CI update

* CI update

* CI update

* CI update

* CI update

* CI update

* CI update

* improving nf-test part 1

* test for setup-nextflow fix

* reset last commit

* added teams hooks

* fix for teams hooks

* CI fix

* linting fix

* modules config fix

* turned debug on for actions

* turned debug on for actions

* updated all tests

* changed nextflow install

* CI update

* CI update

* fix for vcfanno test

* prettier

* black reformat

* several small improvements

* final improvements (hopefully)

* black

* tests fix

* tests fix

* tests fix?

* fix default test

* fix default test

* fix default test

* fix tests being done twice

* fix default test

* tests update

* tests update

* fix default test

* fix tests (fingers crossed)

* bumped version + changelog

* prettier

.editorconfig

@@ -8,7 +8,7 @@ trim_trailing_whitespace = true
 indent_size = 4
 indent_style = space
 
-[*.{md,yml,yaml,html,css,scss,js}]
+[*.{md,yml,yaml,html,css,scss,js,cff}]
 indent_size = 2
 
 # These files are edited and tested upstream in nf-core/modules

.github/CONTRIBUTING.md

@@ -1,22 +1,20 @@
-# nf-core/tva: Contributing Guidelines
+# CenterForMedicalGeneticsGhent/nf-cmgg-germline: Contributing Guidelines
 
 Hi there!
-Many thanks for taking an interest in improving nf-core/tva.
+Many thanks for taking an interest in improving CenterForMedicalGeneticsGhent/nf-cmgg-germline.
 
-We try to manage the required tasks for nf-core/tva using GitHub issues, you probably came to this page when creating one.
+We try to manage the required tasks for CenterForMedicalGeneticsGhent/nf-cmgg-germline using GitHub issues, you probably came to this page when creating one.
 Please use the pre-filled template to save time.
 
 However, don't be put off by this template - other more general issues and suggestions are welcome!
 Contributions to the code are even more welcome ;)
 
-> If you need help using or modifying nf-core/tva then the best place to ask is on the nf-core Slack [#tva](https://nfcore.slack.com/channels/tva) channel ([join our Slack here](https://nf-co.re/join/slack)).
-
 ## Contribution workflow
 
-If you'd like to write some code for nf-core/tva, the standard workflow is as follows:
+If you'd like to write some code for CenterForMedicalGeneticsGhent/nf-cmgg-germline, the standard workflow is as follows:
 
-1. Check that there isn't already an issue about your idea in the [nf-core/tva issues](https://github.com/nf-core/tva/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this
-2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [nf-core/tva repository](https://github.com/nf-core/tva) to your GitHub account
+1. Check that there isn't already an issue about your idea in the [CenterForMedicalGeneticsGhent/nf-cmgg-germline issues](https://github.com/CenterForMedicalGeneticsGhent/nf-cmgg-germline/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this
+2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [CenterForMedicalGeneticsGhent/nf-cmgg-germline repository](https://github.com/CenterForMedicalGeneticsGhent/nf-cmgg-germline) to your GitHub account
 3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
 4. Use `nf-core schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
 5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged
@@ -39,7 +37,7 @@ If any failures or warnings are encountered, please follow the listed URL for mo
 
 ### Pipeline tests
 
-Each `nf-core` pipeline should be set up with a minimal set of test-data.
+Each pipeline should be set up with a minimal set of test-data.
 `GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully.
 If there are any failures then the automated tests fail.
 These tests are run both with the latest available version of `Nextflow` and also the minimum required version that is stated in the pipeline code.
@@ -52,13 +50,9 @@ These tests are run both with the latest available version of `Nextflow` and als
 - Fix the bug, and bump version (X.Y.Z+1).
 - A PR should be made on `master` from patch to directly this particular bug.
 
-## Getting help
-
-For further information/help, please consult the [nf-core/tva documentation](https://nf-co.re/tva/usage) and don't hesitate to get in touch on the nf-core Slack [#tva](https://nfcore.slack.com/channels/tva) channel ([join our Slack here](https://nf-co.re/join/slack)).
-
 ## Pipeline contribution conventions
 
-To make the nf-core/tva code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
+To make the CenterForMedicalGeneticsGhent/nf-cmgg-germline code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
 
 ### Adding a new step
 
@@ -85,7 +79,7 @@ Once there, use `nf-core schema build` to add to `nextflow_schema.json`.
 
 Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
 
-The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
+The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.
 
 ### Naming schemes
 

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -2,14 +2,6 @@ name: Bug report
 description: Report something that is broken or incorrect
 labels: bug
 body:
-  - type: markdown
-    attributes:
-      value: |
-        Before you post this issue, please check the documentation:
-
-        - [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
-        - [nf-core/tva pipeline documentation](https://nf-co.re/tva/usage)
-
   - type: textarea
     id: description
     attributes:
@@ -47,4 +39,4 @@ body:
         * Executor _(eg. slurm, local, awsbatch)_
         * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
         * OS _(eg. CentOS Linux, macOS, Linux Mint)_
-        * Version of nf-core/tva _(eg. 1.1, 1.5, 1.8.2)_
+        * Version of CenterForMedicalGeneticsGhent/nf-cmgg-germline _(eg. 1.1, 1.5, 1.8.2)_

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -1,5 +1,5 @@
 name: Feature request
-description: Suggest an idea for the nf-core/tva pipeline
+description: Suggest an idea for the CenterForMedicalGeneticsGhent/nf-cmgg-germline pipeline
 labels: enhancement
 body:
   - type: textarea

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,22 +1,21 @@
 <!--
-# nf-core/tva pull request
+# CenterForMedicalGeneticsGhent/nf-cmgg-germline pull request
 
-Many thanks for contributing to nf-core/tva!
+Many thanks for contributing to CenterForMedicalGeneticsGhent/nf-cmgg-germline!
 
 Please fill in the appropriate checklist below (delete whatever is not relevant).
 These are the most common things requested on pull requests (PRs).
 
 Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.
 
-Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/tva/tree/master/.github/CONTRIBUTING.md)
+Learn more about contributing: [CONTRIBUTING.md](https://github.com/CenterForMedicalGeneticsGhent/nf-cmgg-germline/blob/master/.github/CONTRIBUTING.md)
 -->
 
 ## PR checklist
 
 - [ ] This comment contains a description of changes (with reason).
 - [ ] If you've fixed a bug or added code that should be tested, add tests!
-  - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/tva/tree/master/.github/CONTRIBUTING.md)
-  - [ ] If necessary, also make a PR on the nf-core/tva _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
+  - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/CenterForMedicalGeneticsGhent/nf-cmgg-germline/blob/master/.github/CONTRIBUTING.md)
 - [ ] Make sure your code lints (`nf-core lint`).
 - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
 - [ ] Usage Documentation in `docs/usage.md` is updated.

.github/workflows/branch.yml

@@ -11,9 +11,9 @@ jobs:
     steps:
       # PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
       - name: Check PRs
-        if: github.repository == 'nf-core/tva'
+        if: github.repository == 'CenterForMedicalGeneticsGhent/nf-cmgg-germline'
         run: |
-          { [[ ${{github.event.pull_request.head.repo.full_name }} == nf-core/tva ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
+          { [[ ${{github.event.pull_request.head.repo.full_name }} == CenterForMedicalGeneticsGhent/nf-cmgg-germline ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
 
       # If the above check failed, post a comment on the PR explaining the failure
       # NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets

.github/workflows/ci.yml

@@ -10,41 +10,65 @@ on:
 
 env:
   NXF_ANSI_LOG: false
-  CAPSULE_LOG: none
 
 jobs:
-  test:
-    name: Run pipeline with test data
+  test_all:
+    name: Run pipeline with test data (complete)
     # Only run on push if this is the nf-core dev branch (merged PRs)
-    if: "${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/tva') }}"
+    if: "${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'CenterForMedicalGeneticsGhent/nf-cmgg-germline') }}"
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        # Nextflow versions
-        include:
-          # Test pipeline minimum Nextflow version
-          - NXF_VER: "21.10.3"
-            NXF_EDGE: ""
-          # Test latest edge release of Nextflow
-          - NXF_VER: ""
-            NXF_EDGE: "1"
+        nxf_ver: ["21.10.3", "22.04.0"]
+        test:
+          - "default"
+          - "fails"
+          - "seqplorer_min"
+          - "seqplorer_vcfanno"
+          - "seqr_full"
+          - "seqr_no_genotyping"
     steps:
       - name: Check out pipeline code
         uses: actions/checkout@v2
 
-      - name: Install Nextflow
-        env:
-          NXF_VER: ${{ matrix.NXF_VER }}
-          # Uncomment only if the edge release is more recent than the latest stable release
-          # See https://github.com/nextflow-io/nextflow/issues/2467
-          # NXF_EDGE: ${{ matrix.NXF_EDGE }}
+      - name: Install nextflow
         run: |
-          wget -qO- get.nextflow.io | bash
-          sudo mv nextflow /usr/local/bin/
+          sudo bash; mkdir /opt/nextflow; cd /opt/nextflow; wget https://github.com/nextflow-io/nextflow/releases/download/v${{ matrix.nxf_ver }}/nextflow; chmod +x nextflow;
+          echo "/opt/nextflow" >> $GITHUB_PATH;
+
+      # - name: Install Nextflow
+      #   uses: nf-core/setup-nextflow@v1
+      #   with:
+      #     version: "${{ matrix.nxf_ver }}"
+
+      - name: Install nf-test
+        run: |
+          sudo bash; mkdir /opt/nf-test; cd /opt/nf-test; wget https://github.com/askimed/nf-test/releases/download/v0.6.0/nf-test-0.6.0.tar.gz; tar xvfz nf-test-0.6.0.tar.gz; chmod +x nf-test;
+          echo "/opt/nf-test" >> $GITHUB_PATH;
+
+      - name: Free some space
+        run: |
+          sudo rm -rf "/usr/local/share/boost"
+          sudo rm -rf "$AGENT_TOOLSDIRECTORY"
 
       - name: Run pipeline with test data
-        # TODO nf-core: You can customise CI pipeline run tests as required
-        # For example: adding multiple test runs with different parameters
-        # Remember that you can parallelise this by using strategy.matrix
         run: |
-          nextflow run ${GITHUB_WORKSPACE} -profile test,docker --outdir ./results
+          nf-test test tests/${{ matrix.test }}.test
+
+      - name: Output log on failure
+        if: failure()
+        run: |
+          sudo apt install bat > /dev/null
+          batcat --decorations=always --color=always .nf-test/tests/*/meta/std.{out,err}
+      - name: Upload logs on failure
+        if: failure()
+        uses: actions/upload-artifact@v2
+        with:
+          name: nf-test-logs
+          path: |
+            .nf-test/tests/*/meta/nextflow.log
+            .nf-test/tests/*/meta/std.out
+            .nf-test/tests/*/meta/std.err
+            .nf-test/tests/*/meta/trace.csv
+            .nf-test/tests/*/work
+            .nf-test/*/output