avatarEric Wei

Summary

The web content provides a guide on setting up a master-slave architecture in Jenkins to improve CI/CD pipeline availability and scalability.

Abstract

Jenkins is a critical tool for managing continuous integration and deployment (CI/CD) pipelines, but running multiple pipelines on a single Jenkins instance can lead to availability and scalability issues. The solution proposed is the implementation of a master-slave architecture, where the master Jenkins schedules and distributes tasks to various slave Jenkins instances. This setup allows for high availability, as the failure of a slave does not impact the overall system, and easy scalability by adding more slaves. The article outlines the benefits of this architecture, including the ability to run specific pipelines on different operating systems and the reduction of build times. It also provides a step-by-step guide on how to add a Jenkins slave to the master and how to specify slave labels in the pipeline configuration to ensure jobs are executed on the appropriate slave.

Opinions

  • The author suggests that a master-slave architecture is superior for managing multiple CI/CD pipelines in Jenkins, emphasizing its benefits over a single Jenkins instance.
  • It is recommended to have two master nodes (a master-standby setup) for high availability, ensuring that the system can handle the failure of the primary master node.
  • The article implies that using labels to assign jobs to specific slaves is a best practice for organizing and optimizing the workflow in a Jenkins master-slave setup.
  • The author believes that scaling slave nodes is crucial for running multiple jobs efficiently, hinting at further reading on the subject for advanced setups.
  • The article encourages reader engagement by suggesting that the provided information is part of a series, with additional tutorials available for those interested in further enhancing their Jenkins CI/CD infrastructure.

How to Setup Master and Slave Architecture using Jenkins? (CI/CD: Part 1)

Jenkins is an essential tool to run CI/CD pipelines for your projects.

However, if you have more and more pipelines running in single Jenkins, you may encounter some problems in availability and scalability.

Given that one Jenkins can cause a single point of failure problem and longer building time when executing the increasing number of CI/CD pipelines.

How to set up a superior architecture to resolve these problems?

You can set up master and slave architecture for Jenkins.

In this post, we will explore

  • Master-Slave architecture of Jenkins
  • Advantages of using Master-Slave architecture in Jenkins
  • 2 Steps to setup Slaves with Jenkins Master

Master-Slave architecture of Jenkins

This architecture is composed of two components — one master and multiple slaves below.

Master Jenkins:

  • Can define the jobs and assign which slave machine to actually execute the pipelines.
  • Requires to set up the connection to a slave beforehand via SSH in TCP.
  • Needs to save pipelines.

Slave Jenkins:

  • does not need to save jobs or pipelines.
  • requires to keep the same environment configuration as the master.
  • can be hosted on a different OS for your needs.

For instance, a developer makes any code change in PR to Github, Github will notify Jenkins Master via webhook.

Then Jenkins Master can assign a slave Jenkins to run a specific pipeline such as building games for Android on Slave-Win and iOS on Slave-Mac to speed up your workflow.

Note:

  • If you don't know how to set up a pipeline from the ground up in Jenkins, refer to the tutorial below.

Advantages of Master-Slave architecture in Jenkins

Availability:

  • Jenkins Slave can achieve high availability while Jenkins Master can’t.
  • If one of Jenkins's slave machines goes down, Jenkins master will automatically find an available one.
  • Be careful that Jenkins Master may go down if having no standby server to take over.
  • Of course, it’s recommended to set up two masters (master-standby)and multiple salves when needed.

Scalability:

  • It’s simple to scale capacity to ensure the same performance by adding a new Jenkins slave.

2 Steps to setup Slaves with Jenkins Master

Step1: Add a Jenkin Slave to Jenkins Master

  • Log in to Jenkins Web
  • Go to Manage Jenkins → Manage Nodes and clouds → Click New Nodes
  • Fill in the following

Name:

Description:

Remote root directory:

Label:

Usage: Select Only build jobs with label expression matching this node

Launch method: Choose Launch agent via SSH

Host:

Host: Credentials:

  • Then click the Save button.

Note:

  • Labels are used for Jenkins Master to assign which Jenkins Slave to run CI/CD.
  • Remember you need to check the slave node’s connection status is fine or not on the dashboard.
  • If not, check the logs to debug.

Step2: Specify Slave Label in Pipeline

Once you have a slave node ready, you can specify Jenkins Slave’s label below to allow your pipeline to execute CI/CD on a specific Jenkins Slave.

If you use a free-style project to create jobs (not pipeline), you can check in Restrict where this project can be run and specify your Jenkins Slave Label in Label Expression.

That’s all. With only two steps you can set up a master and slave architecture for Jenkins.

Conclusion

In this post, you should have learned how to set up the master-slave architecture for your Jenkins and know the benefits of utilizing this architecture.

I hope this post can be helpful for you to ensure the high availability and scalability of your CI/CD backends.

But there are still some points you can improve

  • Because the Master node may go down at some time due to a single point of failure.
  • How do we scale slave nodes to run multiple jobs?

You can refer to how to scale Master and Slave architecture using Jenkins? for details to make your CI/CD backends even better.

If you liked this story, please show your support by 👏 this story or subscribe to our email to get the latest tutorials in the future.

Thank you for reading!

Related posts:

You might be interested in

Jenkins
Unity
Cicd
Ci Cd Pipeline
DevOps
Recommended from ReadMedium
avatarVisal Tyagi
Trigger a Pipeline Using Git When Push on Develop Branch & Also Pull Git Content on Folder —…

Tasks to Be Performed:

15 min read
avatarJoel Wembo
Technical Guide: End-to-End CI/CD DevOps with Jenkins, Docker, Kubernetes, ArgoCD, Github Actions …

Building an end-to-end CI/CD pipeline for Django applications using Jenkins, Docker, Kubernetes, ArgoCD, AWS EKS, AWS EC2

53 min read
avatarAlexander Nguyen
The resume that got a software engineer a $300,000 job at Google.

1-page. Well-formatted.

4 min read
avatarBoqiang & Henry
Prometheus: Guide to Modern Monitoring Magic

Summary:

3 min read
avatarSurabhi Gupta
Why 500 LeetCode Problems Changed My Life

How I Prepared for DSA and Secured a Role at Microsoft

5 min read
avatarVictor wasonga onyango
Building a Robust CI/CD Pipeline with Jenkins, SonarQube, Docker, and GitHub on AWS.

Introduction

18 min read