avatarEric Wei

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

2753

Abstract

andby so that master nodes can failover each other to ensure high availability by the following settings in Pipeline.</li></ul><div id="6a25"><pre>agent { <span class="hljs-keyword">node</span> <span class="hljs-title">{ label</span> ‘<span class="hljs-literal">master</span>-active||<span class="hljs-literal">master</span>-standby’ } }</pre></div><p id="87e3">With this setting, Jenkins’s Pipeline can select available nodes on Master-Active or Master-Standby to execute jobs.</p><h1 id="5383">How to Scale Capacity on Slave Node?</h1><p id="e59d">This is tricky I would like to share some tips with you because this is not easy to find.</p><p id="ff3d">You may know master node in the Master-Slave architecture of Jenkins can assign a label to determine which slave node can run specific jobs.</p><p id="dc93"><b>However, how to smoothly dispatch multiple build evens to slave nodes?</b></p><p id="8bcd">The trick is you can assign multiple slave nodes with the same label in the pipeline.</p><div id="3f70"><pre>agent { <span class="hljs-keyword">node</span> <span class="hljs-title">{ label</span> ‘<span class="hljs-literal">slave</span>-group’ } }</pre></div><p id="888d">Or you can assign different labels for different slave nodes.</p><div id="0033"><pre>agent { <span class="hljs-keyword">node</span> <span class="hljs-title">{ label</span> ‘<span class="hljs-literal">slave</span>-Mac||<span class="hljs-literal">slave</span>-Win1||<span class="hljs-literal">Slave</span>-Win2’ } }</pre></div><p id="24c4"><b>Is it possible to make the slave node run multiple jobs at the same time?</b></p><p id="10d9">Yes, remember you need to uncheck below to allow your pipeline to run parallel jobs.</p><figure id="3dbb"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*8oM5Iew_XIAPt9QhX5xamQ.png"><figcaption></figcaption></figure><h1 id="bccb">Summary</h1><p id="0507">In this post, you have learned</p><ul><li>how to make master nodes of Jenkins in high availability via Amazon Route 53.</li><li>how to scale capability by running current jobs on a single node or dispatch jobs to slave nodes by assigning the same label.</li></ul><p id="7071">This can easily achieve horizontal scaling by adding new slave nodes with the same label in Jenkins’s pipeline for your CI/CD backend.</p><p id="a7a0">Hope this tutorial will be helpful for you.</p><p id="c9b9">Thank you for reading!</p><p id="c11d">If you liked this story, please show your support by 👏 this story or subscribe to our email to get the latest tutorials in the future.</p><p id="3aa4">You might be interested in</p><div id="b45d" class="link-block"> <a href="https://tyricwei.medium.com/how-to-reduce

Options

-unity-build-times-for-jenkins-part-3-6df25104b2ec"> <div> <div> <h2>How to Reduce Unity build times for Jenkins? (Part 3)</h2> <div><h3>A complete tutorial to improve Unity build times for your game projects and CI/CD on Jenkins.</h3></div> <div><p>tyricwei.medium.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/1*X_12RuZwnBGB9DjxS9w8yg.jpeg)"></div> </div> </div> </a> </div><div id="ab18" class="link-block"> <a href="https://readmedium.com/how-to-setup-master-and-slave-architecture-using-jenkins-ci-cd-afe69124a07e"> <div> <div> <h2>How to Setup Master and Slave Architecture using Jenkins? (CI/CD)</h2> <div><h3>Jenkins is an essential tool for running CI/CD pipelines for your projects.</h3></div> <div><p>medium.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/1*9HKdyoDcJjcbU6GvYUZsGg.png)"></div> </div> </div> </a> </div><div id="16a8" class="link-block"> <a href="https://readmedium.com/how-to-set-up-ci-cd-pipeline-with-jenkins-and-unity-test-runner-complete-tutorial-518d1ffa289b"> <div> <div> <h2>How to Setup CI/CD Pipeline with Jenkins and Unity Test Runner? (Complete Tutorial)</h2> <div><h3>Continuous Integration and Deployment allow you to have fewer bugs in games to increase your productivity.</h3></div> <div><p>medium.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/1*oA9NWH_74eISFvdEU7EczQ.png)"></div> </div> </div> </a> </div><div id="431c" class="link-block"> <a href="https://readmedium.com/how-to-create-a-serverless-cron-job-in-aws-batch-beb2d4077d8e"> <div> <div> <h2>How to create a Serverless Cron Job in AWS Batch?</h2> <div><h3>AWS Batch is a managed service to allow you to run batch jobs on AWS like big data analyzing or updating data in MYSQL.</h3></div> <div><p>medium.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/1*Jvf9sZ7rfQMGHkX-Qjxb-w.jpeg)"></div> </div> </div> </a> </div></article></body>

How to Scale Master and Slave Architecture using Jenkins? (CI/CD: Part 2)

This article will detail how to set up Master-Slave Architecture on Jenkins for high availability and scalability.

In the Master-Slave architecture of Jenkins, it is very common to have one master connected to several slaves.

Suppose a master node crashes due to an unexpected hardware failure. Your CI/CD service may be interrupted at some time.

In this post, you will learn

  • How to make the master node highly available for Master-Slave architecture?
  • How to scale the capacity of slave nodes?

How to make the master node highly available for Master-Slave architecture?

You can set up DNS services like Amazon Route 53 which can easily achieve failover architecture for Jenkins and ensure high availability.

It’s recommended to design the following Master- Slave architect to integrate Github and Jenkins with Amazon Route 53.

You can refer below to understand Master-Slave architecture for details.

How to Setup Master and Slave Architecture using Jenkins? (CI/CD)

Flow:

Developer → GitHub → Amazon Route 53 → Master-Active and Master- Standby node → Multiple Slave nodes

  • When a developer pushes to Github, Github will send Push events to Master-Active or Master-Standby machine via Amazon Route 53.
  • Amazon Route 53 can determine available nodes to direct traffic to either master-active or master-standby.
  • If Master-Activeve dies, Route 53 can automatically switch traffic to the available node for Master-Standby.
  • To scale the capacity, you can add slave nodes behind the master node when needed via SSH connection. ( Refer to How to Setup Master and Slave Architecture using Jenkins? for details )
  • You need to set up the master-active to connect master-standby so that master nodes can failover each other to ensure high availability by the following settings in Pipeline.
agent {
     node {
         label ‘master-active||master-standby’
      }
 }

With this setting, Jenkins’s Pipeline can select available nodes on Master-Active or Master-Standby to execute jobs.

How to Scale Capacity on Slave Node?

This is tricky I would like to share some tips with you because this is not easy to find.

You may know master node in the Master-Slave architecture of Jenkins can assign a label to determine which slave node can run specific jobs.

However, how to smoothly dispatch multiple build evens to slave nodes?

The trick is you can assign multiple slave nodes with the same label in the pipeline.

agent {
     node {
         label ‘slave-group’
      }
 }

Or you can assign different labels for different slave nodes.

agent {
     node {
         label ‘slave-Mac||slave-Win1||Slave-Win2’
      }
 }

Is it possible to make the slave node run multiple jobs at the same time?

Yes, remember you need to uncheck below to allow your pipeline to run parallel jobs.

Summary

In this post, you have learned

  • how to make master nodes of Jenkins in high availability via Amazon Route 53.
  • how to scale capability by running current jobs on a single node or dispatch jobs to slave nodes by assigning the same label.

This can easily achieve horizontal scaling by adding new slave nodes with the same label in Jenkins’s pipeline for your CI/CD backend.

Hope this tutorial will be helpful for you.

Thank you for reading!

If you liked this story, please show your support by 👏 this story or subscribe to our email to get the latest tutorials in the future.

You might be interested in

Jenkins
Jenkins Pipeline
AWS
Route 53
Cicd
Recommended from ReadMedium