Continuous integration: Jenkins Pipeline syntax introduction (2023)

PIPELINE is often used in UNIX / Linux systems, and PIPELINE sends an output of a command / program / process to another command to perform further processing. for example:cat test.txt | grep test1. Pipeline in Jenkins borrows the Pipeline idea in UNIX / Linux, and implements the Jenkins task as the pipeline, and describes the entire continuous integration process through jenkinsfile.

content

A simple Pipeline project
- Newly built PIPELINE tasks
- Pipeline Script
- Pipeline Script from SCM
- Construct
Jenkinsfile syntax
- Declarative pipeline
  - Sections
    - agent
    - stages
    - steps
    - post
  - Directives
    - environment
    - options
    - parameters
    - triggers
    - stage
    - tools
    - input
    - when
  - Parallel execution
- Scripted pipeline
DSL method steps
summary

Let's first look at how to create a simple pipeline on Jenkins.

Newly built PIPELINE tasks

Newly built a Pipeline task

You can set the build trigger mode.

Next start writing a Pipeline script, select Pipline Script or Pipline Script from SCM.

The PIPline Script mode is written in the input box below the script.
The Pipline Script from SCM is to put the PIPELINE SCRIPT on the git or svn, pull down from above when executed.

Pipeline Script

Select Pipeline Script to prepare the following code:

pipeline { agent any stages { stage('begin') { steps { echo 'Hello pipeline' } } } post { always { echo 'say goodbay' } }}

Stages: Add execution steps
POST: Operations performed after the task is executed

Pipeline Script from SCM

In addition to writing in Jenkins interface, Pipeline Script can also be placed in the GIT source code library.

Select Pipeline Script from SCM in Jenkins Pipeline Task

Add a git source code address, fill in the address where the JenkinsFile file you want to run in Script Path is completed.

Construct

Save after the creation is complete, click Build Now, we can see the progress of different build phases.

Build a log:

Jenkinsfile supports two grammar forms:

Scripted Pipeline - Script lines, based onGroovyUniversal DSL (Domain-Specific Language, Domain-Specific Language, Domain Specific Language)
Declarative Pipeline - Declarative pipeline syntax, introduced after V2.5, support structured, provides richer syntax characteristics.

DSL is a computer language focused on an application area. Different from Python, Java, etc., the GPL is different, DSL is a development language designed for specific areas, such as HTML, Scalable tag language XML, SQL language, etc. .

Typically, the declaration and script syntax in the pipeline are used in conjunction.

Declarative pipeline

The declarative pipeline syntax must be included in a pipeline block:

pipeline { /* Declarative Pipeline */}

The PIPELINE block is mainlySections, Directives, StepsOr the assignment statement consists of.

pipeline { agent any stages { stage('begin') { steps { echo 'Hello pipeline' } } } post { always { echo 'say goodbay' } }}

Sections

Sections includes Agent, Stages, Steps, and POST.

agent

Agent defines the PIPELINE execution node, must bepipeline The top layer definition of the block.

The main parameters:

any: Can perform PIPELINE on any available Agent
none: Pipeline will not assign global agents, each Stage assigns its own Agent
label: Specifies the Label of the Run Node Agent
node: Custom Run Node Configuration,
- Specify label
- Specify CustomWorkspace
docker: Perform the pipeline using a given container.
dockerfile: Use the container built in the Source Library to perform PIPELINE.
kubernetes: Perform pipeline in the Kubernetes cluster

The above parameters can also be used in the Stage.

Example script:

pipline { agent { node { label "myslave" customWorkspace "myWorkspace" } }}

stages

Contains one or more stages, most of the works of Pipeline are executed here. Stages are also the instructions that must be specified, no parameters. Alternatively, there must be only one Stages in each PipeLine block.

Stage must also be specified, you need to define the name of the Stage:

pipeline { agent any stages { stage('init') { steps { echo 'Hello World' } } }}

steps

Steps is located in the Stage block, but also the instructions that must be set, no parameters.

STEPS blocks can include Script blocks, available for storageScripted Pipeline script:

pipeline { agent any stages { stage('Example') { steps { echo 'Hello World' script { def browsers = ['chrome', 'firefox'] for (int i = 0; i < browsers.size(); ++i) { echo "Testing the ${browsers[i]} browser" } } } } }}

post

POST is the operation after the PIPELINE or Stage executes, not the instructions that must appear, can set the following trigger conditions:

always: Whether the status of PIPELINE or STAGE runs will run
changed: Only the status of the current Pipeline or Stage runs with the previous running status to run
fixed: Only when the current build fails or unstable, and the current PIPELINE or STAGE runs successfully
regression: The previous run is successful, and the current pipeline or stage runs to run when Failure, UnStable or Aborted.
aborted: You can run only when the current Pipeline or Stage is in the "Aborted" state. Usually manual termination.
failure: Run when the current pipeline or stage is in the "failed" state
success: Run when the PIPELINE or Stage has "Success" status
unstable: The current pipeline or stage has "unsTable" status
unsuccessful: The current Pipeline or Stage is not running when "success" status
cleanup: No matter how the PIPELINE or Stage is executed, run after each Post condition is executed.

Example script:

pipeline { agent any stages { stage('init') { steps { echo 'Hello World' } } } post { success { echo 'success!' sleep 2 } always { echo 'goodbye' } }}

Directives

Declarative Pipeline also includes various instructions, such as Environment, Options, Parameters, Triggers, etc., instructions are not required to appear.

environment

Define the environment variables at the PIPELINE or Stage runtime, located at the top of the Pipeline blockenvironment The defined environment variable is suitable for all steps in the pipeline,stage Definedenvironment Can only be applied tostage The steps are in the steps.

pipeline { agent any environment { CC = 'clang' } stages { stage('Example') { environment { hlw = 'hello world' } steps { echo hlw } } }}

options

The Options instruction allows the PIPELINE proprietary property to be defined within the PIPELINE.

PIPELINE Available Options:

buildDiscarder: Keep the maximum number of builds, which will discard the previous build after the maximum number of buildings.
checkoutToSubdirectory: Perform auto source code control Checkout in the subdirectory of the workspace.
disableConcurrentBuilds: Prohibition of parallel implementation of the PIPELINE task can be used to prevent simultaneous access to share resources.
disableResume: If the controller is restarted, the pipeline is not allowed to recover.
newContainerPerStage: Working with the top agents of the Docker or Dockerfile. When specified, each phase will run in the new container instance on the same node, not all phases run in the same container instance.
overrideIndexTriggers: Allow the default processing method of rewrite the branch index trigger. If the branch index trigger is disabled at a multi-branch or organization tag,options { overrideIndexTriggers(true) }Will only be enabled for this job. otherwise,options { overrideIndexTriggers(false) }The branch index trigger of this Job will only disable.
preserveStashes: Keep the build-up STASHES to use when the Stage is restarted. E.g: options {preserveStashes()}STASHES for saving recently completed builds, oroptions {preserveStashes(buildCount: 5)}The five stacks used to save the recently completed build.
quietPeriod: Set the silent (in seconds) for Pipeline, overwrite the global default. E.g: options {quietPeriod(30)}
skipDefaultCheckout: In the Agent command, skip from the source code Checkout code by default. E.g: options {skipDefaultCheckout()}
skipStagesAfterUnstable: Once the build status becomes "unsteable" status, STAGES is skipped. E.g: options {skipstagesafterstable ()}
retry: After failing, try the number of times the entire PIPELINE, for example:options { retry(3) }
timeout: Setting a timeout for the PIPELINE running, after which Jenkins will stop PIPELINE. E.g: options {timeout(time: 1, unit: 'HOURS')}
timestamps: Plus between all console output generated by the Pipeline running. E.g: options {timestamp ()}
parallelsAlwaysFailFast: Sets Failfast in the subsequent parallel stage in the pipeline to TRUE for parallel to execute the Stage. E.g: options {parallelsAlwaysFailFast()}

The Stage Options directive is similar to the PIPELINE Options. However, the Stage option can only containretry, timeout, or timestamps Steps, or related declarative options related to Stage, such asskipDefaultCheckout。

In Stage, the steps in the Options instruction are called before entering the agent or checks any WHEN.

Stage Available Options:

skipDefaultCheckout
timeout: Set a timeout time for the current stage
retry
timestamps

Example script:

pipeline { agent any options { timeout (time: 1, unit: 'HOURS') buildDiscarder(logRotator(numToKeepStr: '2') retry(5)} stages { stage('init') { options { timeout(time: 30, unit: 'SECONDS') } steps { echo 'Hello World' } } }}

parameters

parameters Directive defines a list of PIPELINEs, support parameter types:

string: String Type
text: Text, can include multi-line
booleanParam: Boolean parameters
choice: Choice parameters
password: Password parameters

Example script:

pipeline { agent any parameters { string(name: 'DEPLOY_ENV', defaultValue: 'staging', description: '') text(name: 'DEPLOY_TEXT', defaultValue: 'One\nTwo\nThree\n', description: '') booleanParam(name: 'DEBUG_BUILD', defaultValue: true, description: '') choice(name: 'CHOICES', choices: ['one', 'two', 'three'], description: '') password(name: 'PASSWORD', defaultValue: 'SECRET', description: 'A secret password') } stages { stage('Example') { steps { echo "${params.DEPLOY_ENV}" echo "${params.PASSWORD}" } } }}

triggers

triggers The instruction defines the way the PIPELINE automation trigger, mainly includes three triggers:

cron: Accept a string of a cron style to define the interval cycle triggered by the PIPELINE, for example:triggers { cron('H */4 * * 1-5') }
pollSCM: Accepting a string string to define Jenkins to check the regular intervals of the SCM source update; if there is a new change, PipeLine will be re-triggered. E.g: triggers { pollSCM('H */4 * * 1-5') }
upstream: Accepting a comma-separated Job string and threshold. When any job in the string is completed at a minimum threshold, the PIPELINE will be re-triggered. E.g: triggers { upstream(upstreamProjects: 'job1,job2', threshold: hudson.model.Result.SUCCESS) }

More cron expression syntax introductionLinux cron Timed Introduction。

Example script:

pipeline { agent any triggers { cron('H */4 * * 1-5') } stages { stage('init') { steps { echo 'Hello World' } } }}

stage

As mentioned earlier, the Stage command is in the Stages block, and the instructions that must be set should include at least one.

tools

Define automatic installation andPATHThe tool. This parameter is ignored if the Agent is not specified.

Support as follows:

maven
jdk
gradle

Example script:

pipeline { agent any tools { maven 'apache-maven-3.0.1' } stages { stage('Example') { steps { sh 'mvn --version' } } }}

Among themapache-maven-3.0.1 Must configure in Jenkins: Manage Jenkins -> Global Tool Configuration.

input

stage of input Directive allows usageinput stepTip input. In the applicationoptions After enteringstage of agent Or assessmentwhen Before conditions,stage Will be paused. if input Be agreed,stage Will continue.

The configurable options are as follows:

message: Must be specified, information presented to the user.
id: Optional identifier, defaultstage name.
ok: Optional text for the "OK" button.
submitter: Separated by commas or allowed submissionsinput The external group name. Any user is allowed by default.
submitterParameter: An optional name for environment variables. If present, usesubmitter Name setting.
parameters: Tips a list of optional parameters provided by the submitter. And foregoingparameters The command definition method is the same.

Example script:

pipeline { agent any stages { stage('Example') { input { message "Should we continue?" ok "Yes, we should." submitter "alice,bob" parameters { string(name: 'PERSON', defaultValue: 'Mr Jenkins', description: 'Who should I say hello to?') } } steps { echo "Hello, ${PERSON}, nice to meet you." } } }}

when

when The instruction is located in the Stage instruction, allowing the pipeline to determine whether or not the stage should be executed according to a given condition, at least one condition must be included.

Support the following nested conditions:

Branch: This stage is performed when the branch being built is matched with branch mode, for example:when { branch 'master' }. Note that only use of multi-branch water wire.
buildingTag: This phase is performed when building a tag is building a tag. E.g:when {buildingTag()}
Environment: When the specified environment variable is a given value, this step is performed, for example:when { environment name: 'DEPLOY_TO', value: 'production' }
changelog: If the build SCM change log contains a given regular expression mode, execute this phase, for example:when { changelog '.*^\\[DEPENDENCY\\] .+$' }
changeset: If the build SCM change set contains one or more files that match the given mode, the phase is executed. E.g: when { changeset "**/*.js" }
changeRequest: If the current build is for a "Change Request" (for example, GitHub's Pull Request, Gitlab's Merge Request, Change in Gerrit), this phase is performed. If the parameters are not passed, the Stage runs on each change request, for example:when { changeRequest() }
environment: When the specified environment variable is set to a given value, for example:when { environment name: 'DEPLOY_TO', value: 'production' }
equals: When the desired value is equal to the actual value, for example:when { equals expected: 2, actual: currentBuild.number }
expression: Performs when the specified Groovy expression is TRUE, for example:when { expression { return params.DEBUG_BUILD } }. Note that when an expression returns a string, they must be converted to a Boolean value, or return null to calculate the false. Simply returns "0" or "false" will still be calculated as "true".
tag:ifTAG_NAMEThis phase is performed with a given mode match. E.g: when { tag "release-*" }. If an empty mode is provided, if there is a tag_name variable, this phase will be executed (withbuildingTag() same).
not: When the nested condition isfalse At this stage, it must contain a condition, for example:when { not { branch 'master' } }
allOf: When all nested conditions aretrue When executed, at least one condition must be included, for example:when { allOf { branch 'master'; environment name: 'DEPLOY_TO', value: 'production' } }
anyOf: When at least one nested condition istrue When executed, at least one condition must be included, for example:when { anyOf { branch 'master'; branch 'staging' } }
triggeredBy: Perform it when the current constructed is triggered. E.g:
- when { triggeredBy 'SCMTrigger' }
- when { triggeredBy 'TimerTrigger' }
- when { triggeredBy 'BuildUpstreamCause' }
- when { triggeredBy cause: "UserIdCause", detail: "vlinde" }

Enterstage of agent Before evaluationwhen

By default, if it is definedstage of agent ,butstageof when Conditions will enterstageofagentThe calculation was then calculated. However, this can be passedwhen Block specifiedbeforeAgent Options to change. if beforeAgent Set as true First calculatewhen Condition, only when the conditions are calculatedtrue Talentedagent。

exist input Before the command evaluationwhen

By default, if definedstage of when Will not be in terms of conditionsinput Previous evaluation. However, this can be passedwhen Block specifiedbeforeInput Options to change. if beforeInput Set as true , when The condition will first calculate, only when the conditions are calculated as True, will enterinput 。
beforeInput trueHigh prioritybeforeAgent true。

exist options Before the command evaluationwhen

By default, ifstageDefine anyoptions ，stage of when Conditions will enterstageofoptionsThe calculation was then calculated. However, this can be passedwhen Block specifiedbeforeOptions Options to change. if beforeOptions Set to True, first calculatewhen Condition, only when the conditions are calculatedtrue I will enter againoptions。
beforeOptions trueHigh prioritybeforeInput trueandbeforeAgent true。

Example script:

pipeline { agent any stages { stage('Example Build') { steps { echo 'Hello World' } } stage('Deploy1') { when { branch 'production1' } steps { echo 'Deploying1' } } stage('Deploy2') { when { allOf { branch 'production2' environment name: 'DEPLOY_TO', value: 'production2' } } steps { echo 'Deploying2' } } stage('Deploy3') { when { beforeInput true branch 'production3' } input { message "Deploy to production3?" id "simple-input" } steps { echo 'Deploying3' } } }}

Parallel execution

The declarative pipeline supports multi-stage parallel implementation,Parallel In block, when any of the stages fails, you can addfailFast trueTo force all parallel stages to abort. You can also add one in the pipeline definition.options : options {parallelsAlwaysFailFast() }

pipeline { agent any stages { stage('Non-Parallel Stage') { steps { echo 'This stage will be executed first.' } } stage('Parallel Stage') { when { branch 'master' } failFast true parallel { stage('Branch A') { agent { label "for-branch-a" } steps { echo "On Branch A" } } stage('Branch B') { agent { label "for-branch-b" } steps { echo "On Branch B" } }}}}}

The declarative pipeline can place the Stage into the matrix unit, and Matrix can define a multi-dimensional Name-Value combination matrix and operate it in parallel. Specific use method can be referencedOfficial document。

Scripted pipeline

Scripted PipeLine is a DSL language based on Groovy syntax, which has higher flexibility and better scalability.

Using it needs to understand a certain Groovy syntax knowledge, you can refer to the following documentation:

Please see the scripted pipeline script below, using the IF / ELSE and TRY / CATCH / FINALLY process control method, no matter what the build status is sent:

node('win_agent') { try { // automated test stage('Test') { TEST_STATU = bat ( script: '''echo autotest''', returnStatus: true ) == 0 if ("${TEST_STATU}" == "false") { catchError(stageResult: 'FAILURE') { echo "test did not pass" env.log = "Test is not passed" } } else { echo "test passed" env.log = "Test" } } } catch (exc) { currentBuild.result = 'FAILURE' echo "Something failed, I'm in the catch block." } finally { stage("email") { emailext ( Subject: '\' Construction Notice: $ {project_name} - Build # $ {build_number} - $ {build_status} \ ', to: "[emailprotected]", body: '${FILE,path="email.html"}', ) } }}

In the first introduction to the instance of the Declarative Pipeline and Scripted Pipeline syntax, some method steps are used, such assh、echo、emailextWait, they are some method steps provided by the Jenkins plugin, and 2 examples are mentioned below:

1, BAT / SH method:

STATU = bat ( script: '''echo 666''', returnStatus: true ) == 0echo ${STATU}

2, CATCHERROR Method: After capturing exceptions, set the current build or phase status set to failure and continue to perform the steps behind Pipeline:

stage('Test') { STATU = bat ( script: '''echo 666''', returnStatus: true ) == 0 if ("${TEST}" == "false") { catchError(stageResult: 'FAILURE') { echo "test did not pass" } } else { echo "test passed" }}

You can also usetry-catchto realise.

More steps for the current Jenkins platform support can be accessedhttp://:/pipeline-syntax/html Check.

This article describes two PIPELINE script syntax, which can make task scheduling more flexible, especially for complicated items.

In addition, these two PIPELINE script syntax usually use, in order to make the code more simple, it is recommended to use the PIPELINE shared library, put the Groovy script in the shared library, so many PIPELINE projects can share these methods, greatly reduce code redundancy .

Reference documentation:

--THE END--

The husband is only disagrable, so the world is Mo Neng and dispute. - "Morality"