# 6. CICD using Jenkins

[toc]

## Topics
:::success
- [x] Introduction to CICD
- [x] Introduction to Jenkins
- [x] Installation and Configuration
- [x] Create a sample job in jenkins
- [x] Jenkins Security and Access Management
- [x] Jenkins Plugins
- [x] Sample maven job
- [x] Parameterized builds
- [x] Build triggers
- [x] Build Periodically
- [x] Poll SCM
- [x] Build after other projects are built
- [x] Trigger builds remotely
- [x] Git Webhooks (post-commit hook) (notes to be updated)
- [x] Distributed Builds
- [x] Pipeline-as-Code (Jenkinsfile)(Only if time permits)
:::

### Introduction to CICD
:::warning
- Continuous Integration
- Continuous Delivery
- Continuous Deployment
:::

#### Continuous Integration Workflow

#### Continuous Delivery

#### Continuous Deployment

#### Full CICD Workflow

### Introduction to Jenkins
:::warning
- one of the most popular CI servers
- Written in Java
- more than 2000 plugins
:::

### CI server examples
:::warning
- Jenkins
- Circle CI
- teamcity
- Azure Pipelines
- AWS CodePipeline
- GitLab CI
- GitHub Actions
- Bitbucket Pipeline
:::

### Jenkins Terminology
:::warning
- Workspace
- Job
- Build
- Parameters
- Build Server / Agents
-
:::
### Installation and Configuration

#### Step 1 Install Java(Pre-requisite for Jenkins)

sudo apt update
sudo apt install openjdk-11-jre
java -version

#### Step 2 - Install Jenkins

curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null

echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null

sudo apt-get update

sudo apt-get install fontconfig openjdk-11-jre

sudo apt-get install jenkins

#### Step 3 - Validate Jenkins installation

which jenkins
ls /var/lib/jenkins
sudo systemctl status jenkins

#### Step 4 - Access Jenkins server on a web browser
:::success
**Simplilearn Lab**
http://localhost:8080/

Once you are on login page:
username --> admin
password --> admin

**DigitalOcean / AWS / any other Lab environment**
http://<public-ip>:8080/

unlock jenkins password
- cat /var/lib/jenkins/secrets/initialAdminPassword
- copy the alphanumeric string and paste it in jenkins browser window to unlock jenkins
- Install suggested plugins
- Create an admin user and remember it for future use

**Links:**
- https://www.jenkins.io/download/
- https://www.jenkins.io/doc/book/installing/linux/#debianubuntu
:::

### Creating a Simple Job in Jenkins

#### Step 1 - Create a Freestyle Job
:::warning
- Dashboard --> New Item
- Provide a name for your job
- Choose Freestyle Project
- Click OK
- Select Git as your SCM tool
- Provide the following as your source code:
https://github.com/Coveros/helloworld.git
or
https://github.com/sk12k/addressbook.git
- Save the project
:::

#### Step 2 - Run the Job
:::warning
Go to the Job --> Click "Build now"
:::

### Jenkins Security and Access Management
**3 A's of Security**

A - Authentication (2FA / 3FA / MFA)
    - What you know --> Credentials
    - What you have --> Soft and hard tokens, OTP
    - What you are  --> biometrics
A - Authorization
A - Accounting (Auditing / investigation)    
    

#### **Configure Jenkins Authentication and authorization**
:::warning
Jenkins --> Manage Jenkins --> Configure Global Security
:::

#### Workaround if you have forgotten Jenkins Password

Disable Jenkins security following below steps:
sudo vi /var/lib/jenkins/config.xml
edit the following line:
<useSecurity>true</useSecurity>
to
<useSecurity>false</useSecurity>

Restart Jenkins service:
systemctl restart jenkins

Access your jenkins server on a web browser. it should not ask for any credentials.

Once you are able to access your Jenkins server:
    
1. Revert your security settings (Authentication) to "Jenkins' own user database"
2. Create a new user (if needed) by going to:
Dashboard --> Manage Jenkins --> Manage users

:mag: ***Note:** This is a security vulnarability so once practiced and retrieved your credentials you should revert the security setting to True.*

### Plugins

#### Working with Plugins

Dashboard --> Manage Jenkins --> Manage Plugins

#### Install/uninstall Jenkins plugins
:::warning
Go to:
Dashboard --> Manage Jenkins --> Manage Plugins
Go to:
Available plugins

Search with kewords (e.g. cvs, svn, docker, aws, azure etc.)

Install selected plugins

Restart Jenkins server, if needed.
1. via browser page
2. via CLI (systemctl restart jenkins)

Create a new jenkins job to see if the plugin has been installed successfully.
:::

### Working with Maven for a Java build
:::warning
**Java build tools**
- Ant | build.xml (https://ant.apache.org/)
- Maven | pom.xml (https://maven.apache.org/)
- Gradle | build.xml (https://gradle.org/)

**.Net build tools**
- MSbuild (https://learn.microsoft.com/en-us/visualstudio/msbuild/msbuild)
:::

#### Maven Goals

:::info
mvn compile
test
install
clean install
package
deploy

:::

#### **Class activity - Configure and run a maven job**
**Step 1 - Maven Plugin configuration**
:::success
Dashboard --> Manage Jenkins --> Global Tool configuration
Scroll to the bottom of the page
Look for Maven section
Click Maven installation
Give your maven config a suitable name
Make sure "install automatically" is checked
save the changes
:::
**Step 2 - Configure the Maven job**
:::success
Create a new freestyle job

use the following for your source code repo
https://github.com/sk12k/addressbook.git

Go to "Build steps" section:
select "invoke top-level maven targets" from the dropdown
Choose the appropriate maven version from the drop down
mention "package" as goal
Click on "advanced"
Mention "pom.xml" under "POM"

save the job
run the job
verify the console ouput
:::

### Parameterized builds in Jenkins
**Problem statement**
:::warning
- Create a parameterized build Job in jenkins which contains:
- Creates a folder in /tmp and asks for folder name at the run time. Use string parameter.
- Clones a git repo and asks the repo name at the run time. use choice parameter
Use the following git repos:
- https://github.com/sk12k/addressbook.git
- https://github.com/jenkins-docs/simple-java-maven-app.git
:::
**Solution**
**Step 1 - Create a Parameterized Job**
:::warning
- Create a new freestyle project and give it an appropriate name
- make sure "This project is parameterized" under "General" section is checked
- Click "Add Parameter"
- Choose "String Parameter"
- Name --> foldername
- DefaultValue --> testdir (Optional)
- Description --> as required (Optional)
- Click "Add Parameter" again
- Choose "Choice Parameter"
- Name --> gitrepo
- Choices:
https://github.com/sk12k/addressbook.git
https://github.com/jenkins-docs/simple-java-maven-app.git
- Description --> as required (Optional)
- Scroll down to "source code" section
- Choose git as the SCM
- put ""$gitrepo" as the repository URL
- scroll down to "Build Steps" section
- Click on "Add build step" --> "Execute Shell"
- Put the following in the Execute box:

cd /tmp
mkdir $foldername
echo "This file was created using Jenkins Parameterized build" >> param.info

- Save the job
:::

**Step 2 - Initialize and run the Parameterized Job**
:::warning
- Go to the Job homepage of the job you just created.
- Click on "Build with Parameters"
- Provide an appropriate "foldername"
- Select the intented repository from "gitrepo" dropdown.
- Click "Build"
- Validate the outcome via any of the following:
- Console output (GUI)
- /tmp directory on Jenkins server
:::

### Build Triggers
:::warning
Build triggers are the mechanisms to initiate or trigger your Jenkins job. There are different ways you can trigger a Jenkins job, most common of which are:

1. Build Periodically
2. Poll SCM
3. Build after other projects are built
4. Trigger builds remotely
5. Git Webhooks (post-commit hook)
6. GitHub hook trigger
:::

#### Working with Periodic Builds
:::warning
- Open an existing Jenkins job and scroll down to "build triggers" section
- Select "Build periodically"
- Put the following as schedule:

* * * * *

This will schedule the job to run every minute.
- Save the job and wait for upto a minute.
- The next build of the job should start automatically.
:::

#### Working with Poll SCM
:::warning
- Open an existing Jenkins job and scroll down to "build triggers" section
- Select "Poll SCM"
- Put the following as schedule:

* * * * *

*Note: This will schedule Jenkins to Poll the source code on GitHub (or wherever your Source code is) to be polled every minute.*

- Go to your Github repo and do a new commit with some random changes.
- Come back to the Jenkins Job and check "Git Polling Log" section. It should trigger a new build as a result of the new commit you have made.
:::

#### Upstream / Downstream Projects
:::warning
If you need to trigger / initiate multiple projects in a sequential manner on completion of a previous project. More importantly you want the trigger to be completely automated.
:::

#### Working with multiple jenkins projects
:::warning
- At this point you should have multiple Jenkins jobs configured. If not, go ahead and create at least 3 jobs.

**Configure Upstream Job**
- Open the Second Jenkins job and scroll down to "build triggers" section
- Select "Build after other projects are built"
- Type first few charactors of the 1st job and select from the suggested options.
- Select "Trigger only if build is stable"

**Configure Downstream Job**
- scroll down to "Post build actions"
- Click on the dropdown and select "Build other Projects"
- Type first few charactors of the 1st job and select from the suggested options.
- Select "Trigger only if build is stable"

**Run the Jobs**
- Go to Jenkins Dashboard.
- Build the First Job
- Wait for a few minutes and refresh the page. 2nd and 3rd Jenkins jobs should have run automatically. You can verify it by checking the build numbers and timestamp of last successful build.
:::

#### Trigger builds remotely
:::warning

- Open an existing Jenkins job and scroll down to "build triggers" section
- select "Trigger builds remotely"
- provide a token which you remember
- copy the URL syntax provided just below the box and change the values for "JENKINS_URL" and "TOKEN_NAME"


Syntax:    
JENKINS_URL/job/firstjob/build?token=TOKEN_NAME

Example:
http://localhost:8080/job/firstjob/build?token=sldemotoken123

- go to terminal and use curl to trigger the next build.
- use (-u) flag in curl command if it fails due to user authentication.

Authentication in curl:
Syntax:
curl -u username:password JENKINS_URL/job/firstjob/build?token=TOKEN_NAME

Example:
curl -u admin:admin http://localhost:8080/job/firstjob/build?token=sldemotoken123

curl --> Linux/unix
invoke-webrequest --> Powershell
:::

#### Configure a Git post-commit hook for a Jenkins job
:::warning
**Get the Endpoint**
- You should have the curl endpoint from the previous step (trigger job remotely) which should look like below:

Authentication in curl:
Syntax:
curl -u username:password JENKINS_URL/job/firstjob/build?token=TOKEN_NAME

Example:
curl -u admin:admin http://localhost:8080/job/firstjob/build?token=sldemotoken123

**Configure post-commit hook in your Git Repo**

- Go to your git repo and cd into the *"hooks"* directory under *".git"* It should look like the following:

- create a file named post-commit and put the following content:

## vi post-commit

#!/bin/sh
curl -u admin:admin http://localhost:8080/job/firstjob/build?token=sldemotoken123   

Make sure the file has execute permissions. You might have to run the following to achieve it:

chmod +x post-commit


Now you ready to test the webhook.
1. Go to your jenkins job and make a note of the last Build number and time stamp.
2. Come back to your Git repo, make a small change in code files and commit the changes.
3. Check the build log for your Jenkins job. You should see the new build number with current time stamp.
:::

### Distributed builds in Jenkins (Build Agents)


#### Why Distributed builds
:::warning
- Distribute/manage the load
- High Availability
- Parallel builds (by increasing the number of Executors)
- Creating and working with multiple isolated environments (such as for maven, ant, .net, python etc.)
:::

:::warning
The Jenkins architecture is designed for distributed build environments. It allows us to use different environments for each build project balancing the workload among multiple agents running jobs in parallel.

The Jenkins controller is the original node in the Jenkins installation. The Jenkins controller administers the Jenkins agents and orchestrates their work, including scheduling jobs on agents and monitoring agents. Agents may be connected to the Jenkins controller using either local or cloud computers.

The agents require a Java installation and a network connection to the Jenkins controller. View the 3 minute video below for a brief explanation of Jenkins agents.

:mag: [Working with Jenkins Agents](https://www.jenkins.io/doc/book/using/using-agents/)
:::

#### Setting up a Jenkins Worker node / agent

:::warning
**Step 1 - Setup an ubuntu node**
- Create/setup a new ubuntu linux node and create a new user on it which would be by Jenkins controller to ssh into this machine.
- Test your ssh connection from Jenkins controller to Jenkins agent/worker.
:::

:mag: *For convenience use username as **jenkinsadmin** and provide a password which is easy for you remember*

:::warning
**Step 2 - Configure Worker node from Jenksin UI**
- Login to your Jenkins CI server and navigate to the following path:
*Dasboard --> Manage Jenkins --> Manage Nodes and Clouds*
- Click on "New Node"
- Give your node a name (e.g. Maven Node)
- Fill in all the details as follows:

Name - Maven Node
Description - Jenkins Worker Node
Number of Executors - 2
Remote root directory - /tmp
Label - maven java worker1 ##Space delimited
Usage - Use this node as much as possible
Launch method - Launch agents via ssh
Host - <ipaddress of your worker node>
Credentials - add ssh credentials for your worker node and select from the dropdown.
Host Key Verification Strategy - Select "Non  Verifying Verification Strategy"

- Leave the rest to default and save changes.
- Wait for a few seconds and refresh the node list. You should see the new node listed.
:::

:mag: Check the node log (Manage Jenkins --> Manage nodes and clouds --> <nodename> --> log) if node fails to initialize

:::warning
**Step 3 - Configure a job to run only on a specific agent**

- Open an existing job or create a new freestyle job explained before.
- Go to Dashboard --> Jobname --> Configuration
- Click on General (1)
- Make sure "Restrict where this project can be run" is checked (2)
- Provide the label name of the agent in the box (3)
- Save the job

:::

:::warning
**Step 4 - Run the Job and verify the build agent**
- Once configured, run the job and see "Console Output" to verify that the job was built on the specified agent.


:::
### Assignment (20/21 May)
:::info
1. Change the Jenkins port from default (8080) to 8888

**Solution:**

Edit the file /usr/lib/systemd/system/jenkins.service
Update the line 
Environment="JENKINS_PORT=8080"
to
Environment="JENKINS_PORT=8888"

Restart the service and systemd:
systemctl daemon-reload
systemctl restart jenkins.service

or

vi /etc/default/jenkins 

Change 
HTTP_PORT=8080
to 
HTTP_PORT=8888

Restart the jenkins service:
sudo systemctl restart jenkins

or

sudo systemctl edit jenkins.service

added this 2 lines:
[Service]
Environment="JENKINS_PORT=8088"
systemctl daemon-reload
sudo systemctl restart jenkins

2. Configure a GitHub web Hook for your own repository which will trigger a Jenkins job on every Push to your repository. (Please note you would need a public jenkins server for this task such as on DigitalOcean or AWS etc.)

**Solution:**
1. https://www.blazemeter.com/blog/how-to-integrate-your-github-repository-to-your-jenkins-project
2. https://medium.com/@marc_best/trigger-a-jenkins-build-from-a-github-push-b922468ef1ae
:::

---
## [Session recording for 13th May](https://zoom.us/rec/play/HdVsiX5OuhkSyxZnwzkRQh3MkTDJiWrOfXuXi7Ckia8xyUmjE0SqDdEtSO1-LhEmwgUvvnA3qNVlolOj.sMqyWe_0fd3rds6T?autoplay=true&startTime=1683977353000)

---
### References
:::success
- https://www.jenkins.io/doc/tutorials/
- https://www.cloudbees.com/jenkins/certification
- [Maven Lifecycle](https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html)
- https://crontab.guru/
- https://typicode.github.io/husky/
:::