Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR for release 1.0.0 #5

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
2065fbb
update repo qbic-pipelines
ggabernet Aug 24, 2020
3b10124
update changelog
ggabernet Aug 24, 2020
059e58d
updated docs
ggabernet Aug 24, 2020
97995e2
Allow 3 retries in fasterq-dump process
steffenlem Nov 9, 2020
0c978c7
Merge branch 'dev' into dev
ggabernet Nov 27, 2020
d14b7da
Merge pull request #1 from ggabernet/dev
ggabernet Dec 9, 2020
5b07c5c
repo name update
ggabernet Dec 9, 2020
e07c2cb
fix mkdown errors
ggabernet Dec 9, 2020
24c60ec
more mkdown errors
ggabernet Dec 9, 2020
85bd6a7
Initial template commit
ggabernet Dec 9, 2020
a5632aa
Merge branch 'TEMPLATE' into template_merge
ggabernet Dec 9, 2020
2cec86f
repo name change
ggabernet Dec 9, 2020
7756f9d
rm igenomes params
ggabernet Dec 9, 2020
bd84cc8
fix multiqc
ggabernet Dec 9, 2020
951963a
fix publish dir
ggabernet Dec 9, 2020
fb589d1
add multiqc
ggabernet Dec 9, 2020
e1f9fff
update github actions reponame
ggabernet Dec 9, 2020
27cdd6b
Merge pull request #3 from ggabernet/template_merge
ggabernet Dec 10, 2020
d28d9ef
added process for metadata downloading in xml format
SusiJo Dec 10, 2020
563926d
added metadata process to output description
SusiJo Dec 10, 2020
a2f19db
minor corrections and adding changes to changelog
SusiJo Dec 10, 2020
6d5fd5f
reformatting blank lines
SusiJo Dec 10, 2020
676b76d
reformatting trailing spaces
SusiJo Dec 10, 2020
c45b55b
removed blank line
SusiJo Dec 10, 2020
f30f873
removed date from Changelog and added flag --link to docs/usage
SusiJo Dec 12, 2020
8f15629
Merge pull request #4 from SusiJo/dev
ggabernet Dec 15, 2020
af17aa9
remove logo
ggabernet Dec 15, 2020
d2a9437
update readme
ggabernet Dec 15, 2020
0ebc6ef
update readme
ggabernet Dec 15, 2020
f8cad16
bump versions
ggabernet Dec 15, 2020
dd9aaab
update changelog
ggabernet Dec 15, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/.dockstore.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Dockstore config version, not pipeline version
version: 1.2
workflows:
- subclass: nfl
primaryDescriptorPath: /nextflow.config
18 changes: 11 additions & 7 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,23 @@
# steffenlem/sradownloader: Contributing Guidelines
# qbic-pipelines/sradownloader: Contributing Guidelines

Hi there!
Many thanks for taking an interest in improving steffenlem/sradownloader.
Many thanks for taking an interest in improving nf-core/sradownloader.

We try to manage the required tasks for steffenlem/sradownloader using GitHub issues, you probably came to this page when creating one.
We try to manage the required tasks for nf-core/sradownloader using GitHub issues, you probably came to this page when creating one.
Please use the pre-filled template to save time.

However, don't be put off by this template - other more general issues and suggestions are welcome!
Contributions to the code are even more welcome ;)

> If you need help using or modifying nf-core/sradownloader then the best place to ask is on the nf-core Slack [#sradownloader](https://nfcore.slack.com/channels/sradownloader) channel ([join our Slack here](https://nf-co.re/join/slack)).

## Contribution workflow

If you'd like to write some code for steffenlem/sradownloader, the standard workflow is as follows:
If you'd like to write some code for nf-core/sradownloader, the standard workflow is as follows:

1. Check that there isn't already an issue about your idea in the [steffenlem/sradownloader issues](https://github.com/steffenlem/sradownloader/issues) to avoid duplicating work
1. Check that there isn't already an issue about your idea in the [nf-core/sradownloader issues](https://github.com/nf-core/sradownloader/issues) to avoid duplicating work
* If there isn't one already, please create one so that others know you're working on this
2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [steffenlem/sradownloader repository](https://github.com/steffenlem/sradownloader) to your GitHub account
2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [nf-core/sradownloader repository](https://github.com/nf-core/sradownloader) to your GitHub account
3. Make the necessary changes / additions within your forked repository
4. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged

Expand Down Expand Up @@ -45,9 +46,12 @@ These tests are run both with the latest available version of `Nextflow` and als

## Patch

: warning: Only in the unlikely and regretful event of a release happening with a bug.
:warning: Only in the unlikely and regretful event of a release happening with a bug.

* On your own fork, make a new branch `patch` based on `upstream/master`.
* Fix the bug, and bump version (X.Y.Z+1).
* A PR should be made on `master` from patch to directly this particular bug.

## Getting help

For further information/help, please consult the [nf-core/sradownloader documentation](https://nf-co.re/sradownloader/usage) and don't hesitate to get in touch on the nf-core Slack [#sradownloader](https://nfcore.slack.com/channels/sradownloader) channel ([join our Slack here](https://nf-co.re/join/slack)).
24 changes: 16 additions & 8 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,32 @@
# steffenlem/sradownloader bug report
---
name: Bug report
about: Report something that is broken or incorrect
labels: bug
---

<!--
# qbic-pipelines/sradownloader bug report

Hi there!

Thanks for telling us about a problem with the pipeline.
Please delete this text and anything that's not relevant from the template below:
-->

## Describe the bug
## Description of the bug

A clear and concise description of what the bug is.
<!-- A clear and concise description of what the bug is. -->

## Steps to reproduce

Steps to reproduce the behaviour:

1. Command line: `nextflow run ...`
2. See error: _Please provide your error message_
1. Command line: <!-- [e.g. `nextflow run ...`] -->
2. See error: <!-- [Please provide your error message] -->

## Expected behaviour

A clear and concise description of what you expected to happen.
<!-- A clear and concise description of what you expected to happen. -->

## System

Expand All @@ -33,10 +41,10 @@ A clear and concise description of what you expected to happen.

## Container engine

- Engine: <!-- [e.g. Conda, Docker or Singularity] -->
- Engine: <!-- [e.g. Conda, Docker, Singularity or Podman] -->
- version: <!-- [e.g. 1.0.0] -->
- Image tag: <!-- [e.g. nfcore/sradownloader:1.0.0] -->

## Additional context

Add any other context about the problem here.
<!-- Add any other context about the problem here. -->
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Join nf-core
url: https://nf-co.re/join
about: Please join the nf-core community here
- name: "Slack #sradownloader channel"
url: https://nfcore.slack.com/channels/sradownloader
about: Discussion about the nf-core/sradownloader pipeline
20 changes: 14 additions & 6 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,32 @@
# steffenlem/sradownloader feature request
---
name: Feature request
about: Suggest an idea for the qbic-pipelines website
labels: enhancement
---

<!--
# qbic-pipelines/sradownloader feature request

Hi there!

Thanks for suggesting a new feature for the pipeline!
Please delete this text and anything that's not relevant from the template below:
-->

## Is your feature request related to a problem? Please describe

A clear and concise description of what the problem is.
<!-- A clear and concise description of what the problem is. -->

Ex. I'm always frustrated when [...]
<!-- e.g. [I'm always frustrated when ...] -->

## Describe the solution you'd like

A clear and concise description of what you want to happen.
<!-- A clear and concise description of what you want to happen. -->

## Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.
<!-- A clear and concise description of any alternative solutions or features you've considered. -->

## Additional context

Add any other context about the feature request here.
<!-- Add any other context about the feature request here. -->
20 changes: 10 additions & 10 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# steffenlem/sradownloader pull request
<!--
# qbic-pipelines/sradownloader pull request

Many thanks for contributing to steffenlem/sradownloader!
Many thanks for contributing to qbic-pipelines/sradownloader!

Please fill in the appropriate checklist below (delete whatever is not relevant).
These are the most common things requested on pull requests (PRs).

## PR checklist
Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.

Learn more about contributing: [CONTRIBUTING.md](https://github.com/qbic-pipelines/sradownloader/tree/master/.github/CONTRIBUTING.md)
-->

# PR checklist

- [ ] This comment contains a description of changes (with reason)
- [ ] `CHANGELOG.md` is updated
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If necessary, also make a PR on the [steffenlem/sradownloader branch on the nf-core/test-datasets repo](https://github.com/nf-core/test-datasets/pull/new/steffenlem/sradownloader)
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker`).
- [ ] Make sure your code lints (`nf-core lint .`).
- [ ] Documentation in `docs` is updated
- [ ] `CHANGELOG.md` is updated
- [ ] `README.md` is updated

**Learn more about contributing:** [CONTRIBUTING.md](https://github.com/steffenlem/sradownloader/tree/master/.github/CONTRIBUTING.md)
43 changes: 43 additions & 0 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: nf-core AWS full size tests
# This workflow is triggered on published releases.
# It can be additionally triggered manually with GitHub actions workflow dispatch.
# It runs the -profile 'test_full' on AWS batch

on:
workflow_run:
workflows: ["nf-core Docker push (release)"]
types: [completed]
workflow_dispatch:

jobs:
run-awstest:
name: Run AWS full tests
if: github.repository == 'nf-core/sradownloader'
runs-on: ubuntu-latest
steps:
- name: Setup Miniconda
uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
python-version: 3.7
- name: Install awscli
run: conda install -c conda-forge awscli
- name: Start AWS batch job
# TODO nf-core: You can customise AWS full pipeline tests as required
# Add full size test data (but still relatively small datasets for few samples)
# on the `test_full.config` test runs with only one set of parameters
# Then specify `-profile test_full` instead of `-profile test` on the AWS batch command
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
run: |
aws batch submit-job \
--region eu-west-1 \
--job-name nf-core-sradownloader \
--job-queue $AWS_JOB_QUEUE \
--job-definition $AWS_JOB_DEFINITION \
--container-overrides '{"command": ["nf-core/sradownloader", "-r '"${GITHUB_SHA}"' -profile test --outdir s3://'"${AWS_S3_BUCKET}"'/sradownloader/results-'"${GITHUB_SHA}"' -w s3://'"${AWS_S3_BUCKET}"'/sradownloader/work-'"${GITHUB_SHA}"' -with-tower"], "environment": [{"name": "TOWER_ACCESS_TOKEN", "value": "'"$TOWER_ACCESS_TOKEN"'"}]}'
39 changes: 39 additions & 0 deletions .github/workflows/awstest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: nf-core AWS test
# This workflow is triggered on push to the master branch.
# It can be additionally triggered manually with GitHub actions workflow dispatch.
# It runs the -profile 'test' on AWS batch.

on:
workflow_dispatch:

jobs:
run-awstest:
name: Run AWS tests
if: github.repository == 'nf-core/sradownloader'
runs-on: ubuntu-latest
steps:
- name: Setup Miniconda
uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
python-version: 3.7
- name: Install awscli
run: conda install -c conda-forge awscli
- name: Start AWS batch job
# TODO nf-core: You can customise CI pipeline run tests as required
# For example: adding multiple test runs with different parameters
# Remember that you can parallelise this by using strategy.matrix
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
run: |
aws batch submit-job \
--region eu-west-1 \
--job-name nf-core-sradownloader \
--job-queue $AWS_JOB_QUEUE \
--job-definition $AWS_JOB_DEFINITION \
--container-overrides '{"command": ["nf-core/sradownloader", "-r '"${GITHUB_SHA}"' -profile test --outdir s3://'"${AWS_S3_BUCKET}"'/sradownloader/results-'"${GITHUB_SHA}"' -w s3://'"${AWS_S3_BUCKET}"'/sradownloader/work-'"${GITHUB_SHA}"' -with-tower"], "environment": [{"name": "TOWER_ACCESS_TOKEN", "value": "'"$TOWER_ACCESS_TOKEN"'"}]}'
33 changes: 27 additions & 6 deletions .github/workflows/branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,36 @@ name: nf-core branch protection
# This workflow is triggered on PRs to master branch on the repository
# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
on:
pull_request:
branches:
- master
pull_request_target:
branches: [master]

jobs:
test:
runs-on: ubuntu-18.04
runs-on: ubuntu-latest
steps:
# PRs are only ok if coming from an nf-core `dev` branch or a fork `patch` branch
# PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
- name: Check PRs
if: github.repository == 'qbic-pipelines/sradownloader'
run: |
{ [[ $(git remote get-url origin) == *steffenlem/sradownloader ]] && [[ ${GITHUB_HEAD_REF} = "dev" ]]; } || [[ ${GITHUB_HEAD_REF} == "patch" ]]
{ [[ ${{github.event.pull_request.head.repo.full_name}} == qbic-pipelines/sradownloader ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]


# If the above check failed, post a comment on the PR explaining the failure
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
with:
message: |
Hi @${{ github.event.pull_request.user.login }},

It looks like this pull-request is has been made against the ${{github.event.pull_request.head.repo.full_name}} `master` branch.
The `master` branch on qbic-pipelines repositories should always contain code from the latest release.
Because of this, PRs to `master` are only allowed if they come from the ${{github.event.pull_request.head.repo.full_name}} `dev` branch.

You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.

Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false

50 changes: 38 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,30 +1,56 @@
name: nf-core CI
# This workflow is triggered on pushes and PRs to the repository.
# It runs the pipeline with the minimal test dataset to check that it completes without any syntax errors
on: [push, pull_request]
# This workflow runs the pipeline with the minimal test dataset to check that it completes without any syntax errors
on:
push:
branches:
- dev
pull_request:
release:
types: [published]

jobs:
test:
name: Run workflow tests
# Only run on push if this is the qbic-pipelines dev branch (merged PRs)
if: ${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'qbic-pipelines/sradownloader') }}
runs-on: ubuntu-latest
env:
NXF_VER: ${{ matrix.nxf_ver }}
NXF_ANSI_LOG: false
runs-on: ubuntu-latest
strategy:
matrix:
# Nextflow versions: check pipeline minimum and current latest
nxf_ver: ['19.10.0', '']
nxf_ver: ['20.04.0', '']
steps:
- uses: actions/checkout@v2
- name: Check out pipeline code
uses: actions/checkout@v2

- name: Check if Dockerfile or Conda environment changed
uses: technote-space/get-diff-action@v4
with:
FILES: |
Dockerfile
environment.yml

- name: Build new docker image
if: env.MATCHED_FILES
run: docker build --no-cache . -t qbicpipelines/sradownloader:1.0.0

- name: Pull docker image
if: ${{ !env.MATCHED_FILES }}
run: |
docker pull qbicpipelines/sradownloader:dev
docker tag qbicpipelines/sradownloader:dev qbicpipelines/sradownloader:1.0.0

- name: Install Nextflow
env:
CAPSULE_LOG: none
run: |
wget -qO- get.nextflow.io | bash
sudo mv nextflow /usr/local/bin/
- name: Pull docker image
run: |
docker pull steffenlem/sradownloader:latest
docker tag steffenlem/sradownloader:latest steffenlem/sradownloader:latest

- name: Run pipeline with test data
# For example: adding multiple test runs with different parameters
# Remember that you can parallelise this by using strategy.matrix
run: |
# TODO nf-core: You can customise CI pipeline run tests as required
# (eg. adding multiple test runs with different parameters)
nextflow run ${GITHUB_WORKSPACE} -profile test,docker
Loading