Trigger Jenkins pipeline for new tags in Github Repo

Recently I wanted to create a Jenkins pipeline which builds only when a developer tags a commit → build it; containerize the artifacts; then push to GKE.

Although it was pretty straight forward to set this up using Jenkins GUI with certain plugins, I particularly wanted to try an approach which satisfies “separation of duties” / automation / portability/ efficiency etc., etc.,.

So, we have the following structure in Github, Project repo, Devops Repo (infra/manifests etc.,) and clusters in GKE.

Requirements:

1. Trigger build only when a new tag is created.
2. Separate Jenkinsfile (pipeline) from the project repo in order to allow developers to concentrate only on development and also for reusability.
3. My personal favourite, keep all pipelines in one place for portability. Hint: Job DSL.

I will explain the approach for one solution, there may be many more, happy to hear them in the comments :).

Step 1: Manual GUI pipeline

Created a pipeline job from Jenkins GUI and developed the pipeline script that is required, with all the stages etc., For example:

Please note the checkout part here, which contains the following parameters:

Refspec: ‘+refs/tags/*’:’refs/remotes/origin/tags/*’

Branch Specifier: **/tags/**

This ensures that the build will trigger only for new tags that are created.

I ran good number of times by creating multiple tags and commits. Confirmed the code is good for next steps.

Step 2: Enable webhook

Here is the link for — how to enable webhook and consume with Jenkins:

https://dzone.com/articles/adding-a-github-webhook-in-your-jenkins-pipeline .

Step 3: Create DSL job file

The DSL job file contains all the jobs (pipeline, freestyle, multibranch etc.,) that we want to create. The efficiency/usefulness of this file is that, we can check-in this file into any SCM (git) and maintain versions. We can reuse existing jobs to create new jobs and with a simple click on a button we can create the new job.

We decided to use all jobs as pipeline jobs, so that we can have the flexibility of scripting these jobs using groovy. To create the pipeline jobs using the DSL job file, we follow the template below.

pipelineJob(“<JOB NAME>”) {
  properties { 
    pipelineTriggers {
      triggers {
        githubPush()
      }
    }
  }
  definition {
    cps {
      script(‘’’ 
      <Pipeline script from Step 1 goes here>
      ‘’’.stripIndent())
      sandbox()
    }
  }
}

Please look at the document on how to create jobs using DSL pipeline. https://jenkinsci.github.io/job-dsl-plugin/#path/pipelineJob

Step 4: Jenkins DSL job

A Jenkins DSL job is called the SEED job which would in turn create multiple jobs. To setup SEED job, we create a “free style project” job, specify the SCM (git) and in the build step, select “process job DSLs” and provide the relative path to the above file that we have created and SAVE.

After SAVE, click on the build and if all goes well, you can see the jobs are created in the main Jenkins page.

The main challenge I faced was how to integrate the gitHubPush() to the actual pipeline script using DSL Job. Because, my first attempt was to place the pipeline script from step 1, in a file in the Devops repo and then using the DSL job file, create a job which will call the script using the SCM from the Devops repo. This applied the gitHubPush() on the Devops repo, meaning when I created a tag on Devops repo, the build is trigged and not the project repo which was mentioned in the pipeline script.

Therefore, I have created the job template in Step 3. Now, because the script is placed as a pipeline script, the gitHubPush is applied to the checkout stage. We have basically re-created the job in step 1 using DSL Job. So, whenever a developer tags a new commit on the project repo, the build is triggered. Also, the pipeline script is still located in our Devops repo and thereby satisfying the “separation of duties” requirement.

Steps for creating the tag:

git add <changed files>
git commit -am “msg”
git tag v0.0.0 
git push origin v0.0.0

Ensure the webhook has sent the push notification to the Jenkins job. Go to settings on your Github repo, select webhooks and in the bottom of the link you specified for your Jenkins, you should see something like below.

On Jenkins, click on your job, you should see a new build, if qualified (meaning new tag is pushed).

If not, then, on the left hand side, click on “GitHub Hook Log”, you should see something like, to see that an unqualified (meaning no new tags but a commit was pushed) push has been made,

Started on Apr 13, 2020 4:08:44 AM
Started by event from <ip> -> your Jenkins URL on <time>

Hope this helps, let me know if you have any questions on this topic.

Thank you.