Pipeline jobs

Pipelines are a relatively new addition to Jenkins. The Jenkins project maintains a set of detailed documentation for this job type on their website.

For new projects, or projects where multiple branches are in use, we would recommend the use of a Pipeline job type, as it is more powerful than freestyle jobs for most use cases.

For a Pipeline job, the entire workflow of the job is defined in a Groovy script, called a Jenkinsfile. It is best to store this in your repository, but it can also be written in the job itself. Details of how to write this script are in the Pipeline Script/Jenkinsfile section of this page.

There is some overlap between the options available for freestyle and Pipeline jobs, but not all freestyle options are available for Pipelines, and vice versa.

General options

There are a number of basic options which you may want to consider when configuring a Pipeline job.

Top-level options on the configuration page

Description

The description can be used to store extra information about the build. This can be particularly useful if the job has multiple administrators.

Parameterised builds

By selecting This project is parameterised you can add a wide range of parameters to the build job. For example a string parameter could be used to provide a version number for a build script, or a boolean parameter could specify whether to upload the build output. These parameters are then specified at build time through the web interface.

Concurrent builds

In its default state Anvil will allow concurrent builds of a particular Pipeline job - this is the opposite to freestyle jobs. Choosing Do not allow concurrent builds will prevent this from occurring.

Build Triggers

By default all jobs can be run manually through the web interface. It is also possible to set jobs up to trigger at regular intervals, or upon code changes.

Build trigger options on the configuration page

Both the Build periodically and Poll SCM options take cron-style syntax to describe the times at which they will run. The build system also supports the following simpler options:

Option	Description
@hourly	Every hour
@daily	Every day
@weekly	Every week

You can also use standard cron-syntax although the symbol H is recommended to help spread the running of jobs.

If Poll SCM is selected but no schedule is entered, the build can be triggered using a post-commit hook - this must be set up in the SCM tool, not Anvil.

If regular polling is set up, but you want to temporarily prevent new builds from executing (e.g. if you know some required infrastructure will be unavailable for a period of time), selecting Disable this project will prevent new builds without losing the polling schedule.

Advanced Project Options

You may wish to set a display name for the job. No other advanced options are currently available for Pipeline jobs.

Pipeline Script/Jenkinsfile

The instructions for compiling a Pipeline are written as a script, called a 'Jenkinsfile'. It is best practice to store this in your repository and test it along with else, but the drop-down menu also gives you the option to write the script directly inside the job.

Pipeline script section

In your repository, you should call your script Jenkinsfile and put it at the top level. If you prefer to store it in a different location or under a different name, edit the Script Path to match this. You should also start the script with the line #!groovy, to allow external tools to recognise it as a Groovy script.

The Jenkinsfile contains one or more 'stages', which are groups of related tasks - for example, testing tasks could be contained in a stage called 'Test'. In each stage, one or more 'steps' are carried out. Steps are single tasks, such as shell commands. Below is an example of a Jenkinsfile.

#!groovy
pipeline {
    agent any
    stages {
        stage('Preparation') {
            steps {
                git <repo-url>
            }
        }
        stage('Build') {
            steps {
                sh 'make'
            }
        }
        stage('Test') {
            steps {
                sh 'make check'
                echo 'Testing complete.'
            }
        }
    }
}

This example uses 'Declarative Pipeline' syntax. In this syntax, pipeline defines the block containing all instructions for the Pipeline, and agent instructs Jenkins to allocate an executor for the Pipeline. The any keyword specifies that any available executor can be used; if you have specific operating system requirements, you can use the label option, e.g. agent { label 'sl7'} (possible labels can be found here).

A more advanced syntax exists, called 'Scripted Pipeline' syntax. This syntax omits declarations such as pipeline, stages and steps. This simplifies the appearance of the file, but reduces the options for customisation of the agent and environment. The keyword node instructs Jenkins to allocate the Pipeline on any available agent, and is effectively equivalent to agent any in Declarative Pipeline syntax (to use a specific operating system, add the relevant label as an argument of the node, e.g. node('sl7')). The Jenkinsfile from above is rewritten in Scripted Pipeline syntax below. Declarative Syntax is recommended if you are new to Pipelines due to its greater clarity.

#!groovy
node {
    stage('Preparation') {
        git <repo-url>
    }
    stage('Build') {
        sh 'make'
    }
    stage('Test') {
        sh 'make check'
        echo 'Testing complete.'
    }
}

Other Pipeline steps

There are many different Pipeline steps available; the default ones are documented on the Pipeline Syntax page of the Jenkins website. Additional custom steps have been added to Anvil, such as the git step in the examples above, which clones a Git repository. At the bottom of the configuration page, you can click Pipeline Syntax to access the Snippet Generator, which can generate the Pipeline script code for some of the additional steps that are available on Anvil.

Snippet Generator page

Further Pipeline scripting resources

These resources provide further information about Pipelines and how to apply them. Pipelines are widely used, so many other resources can be found online to help you write a Pipeline script for your specific workflow.

Jenkins - Pipeline - basic documentation

Jenkins - Pipeline Syntax - details of different Pipeline steps

CloudBees - Top 10 Best Practices for Jenkins Pipeline Plugin

CloudBees Network - Main differences between Freestyle - Scripted Pipeline Job - Declarative Pipeline Job