Writing a Tool Wrapper
Overview
Teaching: 20 min
Exercises: 30 minQuestions
What are the key components of a tool wrapper?
How do I use software containers to supply the software I want to run?
Objectives
Write a tool wrapper for the featureCounts tool.
Find an software container that has the software we want to use.
Add the tool wrapper to our main workflow.
This lesson has migrated to https://doc.arvados.org/rnaseq-cwl-training/04-commandlinetool/index.html
It is time to add the last step in the analysis.
# Count mapped reads
featureCounts -T $cores -s 2 -a $gtf -o $counts $counts_input_bam
This will use the “featureCounts” tool from the “subread” package.
File header
A CommandLineTool describes a single invocation of a command line program. It consumes some input parameters, runs a program, and captures output, mainly in in the form of files produced by the program.
Create a new file “featureCounts.cwl”
Let’s start with the header. This is very similar to the workflow, except that we use class: CommandLineTool
.
cwlVersion: v1.2
class: CommandLineTool
label: featureCounts tool
Command line tool inputs
The inputs
section describes input parameters with the same form as
the Workflow inputs
section.
Exercise
The variables used in the bash script are
$cores
,$gtf
,$counts
and$counts_input_bam
.
- $cores is the number of CPU cores to use.
- $gtf is the input .gtf file
- $counts is the name we will give to the output file
- $counts_input_bam is the input .bam file
Write the
inputs
section for the File inputsgtf
andcounts_input_bam
.Solution
inputs: gtf: File counts_input_bam: File
Specifying the program to run
Give the name of the program to run in baseCommand
.
baseCommand: featureCounts
Command arguments
The easiest way to describe the command line is with an arguments
section. This takes a comma-separated list of command line arguments.
arguments: [-T, $(runtime.cores),
-a, $(inputs.gtf),
-o, featurecounts.tsv,
$(inputs.counts_input_bam)]
Input variables are included on the command line as
$(inputs.name_of_parameter)
. When the tool is executed, the
variables will be replaced with the input parameter values.
There are also some special variables. The runtime
object describes
the resources allocated to running the program. Here we use
$(runtime.cores)
to decide how many threads to request.
arguments
vsinputBinding
You may recall from examining existing the fastqc and STAR tools wrappers in lesson 2, another way to express command line parameters is with
inputBinding
andprefix
on individual input parameters.inputs: parametername: type: parametertype inputBinding: prefix: --some-option
We use
arguments
in the example simply because it is easier to see how it lines up with the source shell script.You can use both
inputBinding
andarguments
in the same CommandLineTool document. There is no “right” or “wrong” way, and one does not override the other, they are combined to produce the final command line invocation.
Outputs section
In CWL, you must explicitly identify the outputs of a program. This associates output parameters with specific files, and enables the workflow runner to know which files must be saved and which files can be discarded.
In the previous section, we told the featureCounts program the name of
our output files should be featurecounts.tsv
.
We can declare an output parameter called featurecounts
that will
have that output file as its value.
The outputBinding
section describes how to determine the value of
the parameter. The glob
field tells it to search for a file in the
output directory called featurecounts.tsv
outputs:
featurecounts:
type: File
outputBinding:
glob: featurecounts.tsv
Running in a container
In order to run the tool, it needs to be installed. Using software containers, a tool can be pre-installed into a compatible runtime environment, and that runtime environment (called a container image) can be downloaded and run on demand.
Although plain CWL does not require the use of containers, many popular platforms that run CWL do require the software be supplied in the form of a container image.
Finding container images
Many bioinformatics tools are already available as containers. One resource is the BioContainers project. Let’s find the “subread” software:
- Visit https://biocontainers.pro/
- Click on “Registry”
- Search for “subread”
- Click on the search result for “subread”
- Click on the tab “Packages and Containers”
- Choose a row with type “docker”, then on the right side of the “Full Tag” column for that row, click the “copy to clipboard” button.
To declare that you want to run inside a container, add a section called
hints
to your tool document. Underhints
add a subsectionDockerRequirement
. UnderDockerRequirement
, paste the text your copied in the above step. Replace the textdocker pull
todockerPull:
ensure it is indented twice so it is a field ofDockerRequirement
.Answer
hints: DockerRequirement: dockerPull: quay.io/biocontainers/subread:1.5.0p3--0
Running a tool on its own
When creating a tool wrapper, it is helpful to run it on its own to test it.
The input to a single tool is the same kind of input parameters file that we used as input to a workflow in the previous lesson.
featureCounts.yaml
counts_input_bam:
class: File
location: Aligned.sortedByCoord.out.bam
gtf:
class: File
location: rnaseq/reference_data/chr1-hg19_genes.gtf
Running the tool
Run the tool on its own to confirm it has correct behavior:
cwl-runner featureCounts.cwl featureCounts.yaml
Adding it to the workflow
Now that we have confirmed that the tool wrapper works, it is time to add it to our workflow.
Exercise
- Add a new step called
featureCounts
that runs our tool wrapper. The new step should take input fromsamtools/bam_sorted_indexed
, and should be allocated a minimum of 500 MB of RAM- Add a new output parameter for the workflow called
featurecounts
The output source should come from the output of the newfeatureCounts
step.- When you have an answer, run the updated workflow, which should run the “featureCounts” step and produce “featurecounts” output parameter.
Answer
steps: ... featureCounts: requirements: ResourceRequirement: ramMin: 500 run: featureCounts.cwl in: counts_input_bam: samtools/bam_sorted_indexed gtf: gtf out: [featurecounts] outputs: ... featurecounts: type: File outputSource: featureCounts/featurecounts
Episode solution
Key Points
The key components of a command line tool wrapper are the header, inputs, baseCommand, arguments, and outputs.
Like workflows, CommandLineTools have
inputs
andoutputs
.Use
baseCommand
andarguments
to provide the program to run and the command line arguments to run it with.Use
glob
to capture output files and assign them to output parameters.Use DockerRequirement to supply the name of the Docker image that contains the software to run.