Edge CICD Pipeline – Part A: Using Jenkins

Nowadays, Running AI on edge is getting quite popular, and there are many SDK’s that help us achieve it. Such as Intel OpenVino, NVIDIA Deep Stream, and Google Edge TPU. The bottleneck here is that, since AI runs on edge when it gets deployed on the client-side, there are only limited ways of sending firmware updates to the software or troubleshooting any bug fixes, using third-party software such as Team-viewer or any-desk.

How can we automate this process without the need of trusting any third-party software?

This article is the first one of the 2 explaining different approaches of how anyone can implement CI/CD on an edge.
Now will cover the 1st approach of edge CI/CD pipeline using Jenkins.
Note: To implement it, one should have some basic networking knowledge, Jenkins and python programming.

Jenkins – Overview

The leading open-source automation server, Jenkins provides hundreds of plugins to support building, deploying, and automating any project.

Jenkins is a self-contained, open-source automation server that can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software.

Jenkins can be installed through native system packages, Docker, or even run standalone by any machine with a Java Runtime Environment (JRE) installed.

Jenkins supports version control tools, including AccuRev, CVS, Subversion, Git, Mercurial, Perforce, ClearCase, and RTC, and can execute Apache Ant, Apache Maven, and sbt based projects as well as arbitrary shell scripts and Windows batch commands.

Jenkins CI/CD – The core technology driving that initiative was Jenkins, an open-source tool to build continuous integration and continuous delivery (CI/CD) pipelines.

Advantages of Jenkins are its flexibility, openness, powerful plugin-capabilities, and ease of use.

Jenkins – Workflow

How does it work?

• Jenkins is configured to support master and slave communications. Here master node act as server and slave nodes are edge device
• The communications between master and slave happen through SSH connection. The SSH-keys are set and shared across edge devices to get the communication between Jenkins and edge devices to be more secure.
• Once connected, Jenkins is capable of following things,

===> Setup new edge device with latest code base
===> Check for new firmware updates
===> Troubleshoot

Setup new edge device:

At the initial phase setting up a new edge, the device is a crucial part; we can write a job file in Jenkins with a set of instructions that it needs to follow. These jobs can cover from satisfying the requirements needed for the project to deploying the latest to code the device.

Check for new firmware updates:

Once an edge device is deployed on the client-side. How can an edge device automatically check for a new firmware update? To resolve this caveat, Jenkins provides us with different RESTFUL APIs. These api allow us to trigger firmware update jobs in the Jenkins server, from edge devices themselves.

Check out this link that briefly describes different Jenkins API
Jenkins API Doc

Hands-On: Building CI/CD Pipeline on Edge Using Jenkins

Setting Up Edge Device – Instructions:

Step1: Install the required OS for your edge Device.

Step2: Once the device is booted up. During the first boot, create a new user. follow this username format: “ device_x (Here x is a number)”

Step3: After the user has been created use this command to set the user as default user “ubuntu config — default-user username”.

Step4: Once the default user is set, open Activities and type users and Click Users to open the panel.

Step5: Select the user account that you want to log in to automatically at start-up.

Step6: Press Unlock in the top right corner and type in your password when prompted click the Automatic Login switch to on.

Step7: Once the above setup is complete, Follow the instruction below to help Jenkins run Sudo commands without password,

• Sudo gedit /etc/sudoers, Find the below line.
• Here replace ubuntu with device_x
• After the above line added, try sudo it should not ask for the password.

Step8: Jenkins has java dependency, Follow these instructions below,

sudo apt install software-properties-common apt-transport-https -y
sudo add-apt-repository ppa:openjdk-r/ppa -y
sudo apt install openjdk-8-jdk -y

• After the installation is complete, check the installed java version.

 java --version

Configuring Edge Device with Jenkins – Instructions:

Step1: Initiate SSH connection between Jenkins and edge device, follow the steps below to do so.

Login to Jenkins server using the command below,

ssh Username@hostaddress

After successful login, Type in the following commands.

sudo su jenkins
cat ~/.ssh/id_rsa.pub | ssh username@hostaddress “mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys”
ssh-keyscan -t rsa “hostaddress” >> ~/.ssh/known_hosts

To test if the above commands worked. Copy-paste the below ssh command, it won’t prompt for the password.

ssh username@hostaddress

Note: Replace username and host address, respective to your edge device

Step2: Once the ssh connection is successful, Add the device as a slave node to Jenkins. Follow the steps below to add the device as a slave node,

• Login to Jenkins dashboard.
• Go to Manage Jenkins ==> Manage Nodes and Clouds ==> New Node,
• Give node name, here the node name and edge device OS Username should be same and click on permanent agent and click ok.

• Once clicked on ok, it will redirect to another page where it will ask for additional node information. Follow the steps below to give the required information’s as shown in the below screenshot.

• To test if the edge device node is working. Click on the node and click relaunch agent. In the console it will print agent is online.

Creating a Job in Jenkins for ‘Newly Created Agent’

• Step1: Go to Jenkins Dashboard and click on New Item

• Step2: Select freestyle project and click on OK.

• Step3: Select General tab and set the label expression respective to jenkins node name

• Step4: Select Source Code Management and provide Git repository, also specify the git tag under branches to build.

Here following tags that can be specified which helps to target the build.
1. */tags/dev_* ==> Development build
2. */tags/test_* ==> Testing build
3. */tags/prod_* ==> Production build

• Step5(optional): Artificial Intelligence AI-based project will always have a config file, which contains hyper-parameters that need to be tweaked periodically and also have a requirements file that needs to be executed. To do so Jenkins provides shell scripting that can be done under the build tab.

Click on save, we have successfully implemented Edge CI/CD pipeline using Jenkins.

Benefits:

• Less cost when compared to AWS IoT.
• Helps us to interconnect with different cloud services.
• Easily Manageable.

Limitations:

• Public IP of the edge device is required if the device is hosted outside the Jenkins network.
• We can map an edge device to a public IP using ngrok (port forwarding) to overcome the public IP scenario.