Skip to content

Commit

Permalink
Merge branch 'master' into env-function
Browse files Browse the repository at this point in the history
  • Loading branch information
bentsherman authored Nov 15, 2024
2 parents 3ca75fd + f0a4c52 commit c0dd0ff
Show file tree
Hide file tree
Showing 3 changed files with 35 additions and 10 deletions.
28 changes: 19 additions & 9 deletions docs/process.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,9 +161,9 @@ In the above example, the process will execute one of several scripts depending

### Template

Process scripts can be externalised to **template** files, which can be reused across different processes and tested independently from the overall pipeline execution.
Process scripts can be externalized to **template** files, which allows them to be reused across different processes and tested independently from the pipeline execution.

A template is simply a shell script file that Nextflow is able to execute by using the `template` function as shown below:
A template can be used in place of an embedded script using the `template` function in the script section:

```nextflow
process templateExample {
Expand All @@ -179,9 +179,9 @@ workflow {
}
```

By default, Nextflow looks for the `my_script.sh` template file in the `templates` directory located alongside the Nextflow script and/or the module script in which the process is defined. Any other location can be specified by using an absolute template path.
By default, Nextflow looks for the template script in the `templates` directory located alongside the Nextflow script in which the process is defined. An absolute path can be used to specify a different location. However, this practice is discouraged because it hinders pipeline portability.

The template script may contain any code that can be executed by the underlying environment. For example:
An example template script is provided below:

```bash
#!/bin/bash
Expand All @@ -190,12 +190,22 @@ echo $STR
echo "process completed"
```

:::{tip}
The dollar character (`$`) is interpreted as a Nextflow variable when the script is run as a Nextflow template, whereas it is evaluated as a Bash variable when run as a Bash script. This can be very useful for testing your script independently from Nextflow execution. You only need to provide a Bash environment variable for each of the Nextflow variables that are referenced in your script. For example, it would be possible to execute the above script with the following command in the terminal: `STR='foo' bash templates/my_script.sh`
:::
Variables prefixed with the dollar character (`$`) are interpreted as Nextflow variables when the template script is executed by Nextflow and Bash variables when executed directly. For example, the above script can be executed from the command line by providing each input as an environment variable:

```bash
STR='foo' bash templates/my_script.sh
```

The following caveats should be considered:

- Template scripts are recommended only for Bash scripts. Languages that do not prefix variables with `$` (e.g. Python and R) can't be executed directly as a template script.

- Variables escaped with `\$` will be interpreted as Bash variables when executed by Nextflow, but will not be interpreted as variables when executed from the command line. This practice should be avoided to ensure that the template script behaves consistently.

- Template variables are evaluated even if they are commented out in the template script. If a template variable is missing, it will cause the pipeline to fail regardless of where it occurs in the template.

:::{tip}
As a best practice, the template script should not contain any `\$` escaped variables, because these variables will not be evaluated properly when the script is executed directly.
Template scripts are generally discouraged due to the caveats described above. The best practice for using a custom script is to embed it in the process definition at first and move it to a separate file with its own command line interface once the code matures.
:::

(process-shell)=
Expand Down Expand Up @@ -227,7 +237,7 @@ In the above example, `$USER` is treated as a Bash variable, while `!{str}` is t
:::{note}
- Shell script definitions require the use of single-quote `'` delimited strings. When using double-quote `"` delimited strings, dollar variables are interpreted as Nextflow variables as usual. See {ref}`string-interpolation`.
- Variables prefixed with `!` must always be enclosed in curly brackets, i.e. `!{str}` is a valid variable whereas `!str` is ignored.
- Shell scripts support the use of the {ref}`process-template` mechanism. The same rules are applied to the variables defined in the script template.
- Shell scripts support the use of the {ref}`process-template` mechanism. The same rules are applied to the variables defined in the template script.
:::

(process-native)=
Expand Down
5 changes: 4 additions & 1 deletion docs/reference/process.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,11 @@ The following task properties are defined in the process body:
: *Available only in `exec:` blocks*
: The task unique hash ID.

`task.id`
: The pipeline-level task index. Corresponds to `task_id` in the {ref}`execution trace <trace-report>`.

`task.index`
: The task index (corresponds to `task_id` in the {ref}`execution trace <trace-report>`).
: The process-level task index.

`task.name`
: *Available only in `exec:` blocks*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,11 +73,23 @@ abstract class RepositoryProvider {
return this
}

String getRevision() {
return this.revision
}

RepositoryProvider setRevision(String revision) {
this.revision = revision
return this
}

String getProject() {
return this.project
}

ProviderConfig getConfig() {
return this.config
}

boolean hasCredentials() {
getUser() && getPassword()
}
Expand Down

0 comments on commit c0dd0ff

Please sign in to comment.