---
tags: interview
---
# Bash
#### echo

# print in new line
$ echo -e "hello \nworld"
# print with a tab
$ echo -e "hello \tworld"
# Omit a character (prefix)
$ echo -e "hello\bbuvanesh"
hellbuvanesh
# omit a character (suffix)
$ echo -e "hello\ebuvanesh"
hellouvanesh

#### I/O redirection & special characters

# redirect STDOUT
$ ps aux > output.txt
# redirect STDOUT & STDERR
$ find / -name *foo* &> output.txt
# redirect STDERR
$ find / -name *foo* 2> output.txt

File descriptor 0 is STDIN
File descriptor 1 is STDOUT
File descriptor 2 is STDERR

Usage of ||, &&, ;, &:

"A ; B" Run A and then B, regardless of success of A
"A && B" Run B if A succeeded
"A || B" Run B if A failed
"A &" Run A in background.

Empty a file `> important_file`
#### if elif else

if [[ "${UID}" -eq 0 ]]
 then
  echo "you're root"
 else
  echo "you're NOT root"
  exit 1
 fi

Check if a variable is defined or not

if [[ -z "$1" ]]
then
  echo "variable not defined"
 else
  echo "variable defined"
 fi

Find if the given input is file or directory

if [[ -d $1 ]]
then
  echo "$1 is a directory"
  
elif [[ -f $1 ]]
then
  echo "$1 is a file"
fi

check if you've a write permission

if [[ -w $1 ]]
then
  echo "You have a write permission to this file"
else
  echo "You don't have write permission to this file"
fi

check if the string is empty

if [[ -n $string ]]
then
  echo "The string is not empty"
fi

"and" and "or" operator

a=1
b=10
if [[ $a -gt 2 ]] && [[ $b -gt 6 ]]
then
        echo "numbers are within range"
elif [[ $a -lt 2 ]] || [[ $b -gt 6 ]]
then
        echo "either one is true"
fi

if elif else

if [[ $1 = "start" ]]
then
  echo "starting..."
  
elif [[ $1 = "not" ]]
then
  echo "not starting"
  exit 1
  
else
  echo "not a valid option"
fi

#### Expressions
eq, ge, gt, le, lt, ne
Refer: `man test`

Special characters
&&
`$ ls |grep foo && echo "success"`
echo will be executed only if the first one returns exit 0.
#### Cron

*   *    *             *     *           = 5 stars
min 
hour 
day of month  
month 
day of week

Examples

# for every 15 mins
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=[e-mail]
*/15 * * * *

# for every hr
0 * * * *

# at 2 pm everyday
0 2 * * *

# everyday at 12 am
0 0 * * *

# update system on 1st day of each month 4:25 am 
25 04 1 * * /usr/bin/dnf -y update

#25 =min
#04 = hr (4am)
#01 = on 1st day of every month

# send email of the output
MAILTO=[e-mail]

Anacron
Anacron program handles the cron that didn't run.
#### Calculation

1. Using **let**

value=5
let value=value + 1

2. Using $(( ))

echo $(( 5 + 5))

3. Using `expr`

expr 1 + 5

4. Using `bc`

bc <<< 5 + 5
# OR
echo "5 + 5" |bc
# <<< is here string

#### Here document
1. Write to a file

$ cat >> test.txt << EOF
> hello
> hi
> EOF

2. Echo multiple lines

$ cat << EOF
  this is line number 1
  this is line number 2
  EOF
 # this prints above lines

3. Very useful one for working with ftp**

lftp localhost <<EOL
ls
get file.txt
bye
EOL

echo "The file is downloaded"

#### Substitution Operators
Substitution operators allows to manipulate values of variables in an easy way.
* Ensures that variable exists: `${VAR:-word}`
* Set default values for variables: `${VAR:=word}`
* Catch errors if values doesn't set for a variable: `${VAR:?message}`
* Remove portions of variable values: `${VAR:offset:length}`, In other words ``${VAR:starting_place:ending_place}``

Example 1:

$ cat script.sh
# Won't set the value for the variable, but just prints
echo ${DATE:-today}
echo $DATE

$ sh -x  script.sh 
+ echo today
today
+ echo

Example 2:

# Sets the default value for the variable
$ cat script.sh
echo ${DATE:=today}
echo $DATE

$ sh -x  script.sh
+ echo today
today
+ echo today
today

Example 3:

$ cat script.sh
echo ${DATE:?variable is not set}

$ sh -x script.sh
-bash: DATE: variable is not set

Example 4:

DATE=$(date +%d-%m-%y)
echo the day is ${DATE:0:2}

Output:
the day is 05

# Another example
VAR=123456789
echo ${VAR:2:5}

Output:
34567

#### for

for i in {100..115}
do
  ping -c $i 192.168.56.$i
done

#### tr
1. convert lower characters to upper

echo hello |tr [:lower:] [:upper:]

# upper to lower
echo hello |tr [:upper:] [:lower:]

Use **exit** code while reporting errors!
Reverse use **!**
`if ! [[ "${UID}" -eq 0 ]]`
2. Move words separated by `:` to a new line: `echo $PATH |tr ':' '\n'`

#### shift
Shift removes 1st argument and makes 2nd argument as 1st argument.

#!/bin/bash
# Showing the use of shift

echo "The arguments provided are $*"
echo "The value of \$1 is $1"
echo "The entire string is $*"
shift

echo "The new value of \$1 is $1"
echo "The entire string now is $*"

$ sh input.sh one two three
The arguments provided are one two three
The value of $1 is one
The entire string is one two three

The new value of $1 is two
The entire string now is two three

Find how many arguments are passed to a script - **$#**

 cat howmany_parameters.sh
 echo "You passed $# parameters"

#### while and until
while

 count=5
 while [ $count -gt 5 ]
 do
   echo $count
   (( count++ ))
   # OR let count=count+1    
 done

Useful example with while and sleep:
Wait until a log file contains the text 'Exception'

 while ! grep Exception
 do
   date
   sleep 1
 done

until

 count=5
 until [ $count -gt 5 ]
 do
   echo $count
   let count=count+1
 done

#### sed

# delete empty lines
sed -E '/^$/d' file.txt

# remove double quotes from lines
cat foo | sed 's/\"//g'

# print 5th line in file
sed -n 5p file.txt

# Delete 5th line in file
sed -i '5d' file.txt

# Delete last line of the file
sed -i '$d' file

# Make a backup file while replacing
sed -i.bak 's/old/new/g' foo.log

# Remove lines which are matching with a pattern\
sed -i '/example string/d' file.txt

# Print line number 5 upto end
sed -n '5,$p' file.txt

# Delete from line 2 to line 10 and also delete line number 15
sed -i '15d;2,10d' file.txt

# Regexp for finding a specific string and replacing the entire line - .*
# finds line that contains foo, and replaces the entire line with just bar
sed -i -e '/foo/c bar' file

sed -i 's/^something.*/something new/g'

####Regular expressions####
^  - matches the character(s) at the beginning of a line
&  - matches the character(s) at the end of the line

sed -ne '/^dog/p' animals.txt - print lines only start with dog
sed -ne '/dog$/p' animals.txt - print lines only end with dog
sed -ne /^dog$/p' animals.txt - print lines only start and ends with dog (without mentioning file (animals.txt) it will wait for text)
sed -ne /^dog$/Ip' -> here I used to tell sed -> don't consider whether the characters are lower or higher case (similar to grep -i option)

#### tcpdump

# Target specific network interface
tcpdump -i eth0
# List all available interfaces
tcpdump -D
# for more
https://danielmiessler.com/blog/tcpdump

##### Reference
1. Great list of sed usage - http://sed.sourceforge.net/sed1line.txt

Redirect stdout and stderr - `&>`

Reverse the input provided by the user

echo $@ |rev

#### Arrays

array=("hello" "buvanesh" "kumar")
echo ${array[2]}
# will print kumar

# To print all the arrays in the list
echo ${array[@]}

# To print all the indexs in the list
echo ${!array[@]}

# Arrays in string form
array=(Buvanesh Lisa Caleigh Pavel Tomas)
echo ${array[1]}
Buvanesh

# replace item
array[1]=Shaunom
echo ${array[@]}
Buvanesh Shayn Caleigh Pavel Tomas

# count index 
array=(Buvanesh Lisa Caleigh Pavel Tomas)
echo ${#array[@]}
5

Best to include this on top of the script - `set -eou`
Debugging shell script - `set -x`

Exit if any of the prev command fails - `set -e`

Exit if any varialbes undefined - `set -u`

Exit if a command fails in a pipe - `set -o`

Find modified files in last 3 days

find / -type f -mtime -3 -exec ls {} \;

#### Trap

set -x

trap "echo -e '\e[1;31mPlease do kinit before running this script!\e[0m'; exit 1" ERR INT TERM
klist -s
trap - ERR

release=$1
package=$2
owner=$3
setup_or_branch_git $release $package
brew add-pkg --owner=$owner $release $package

# Example
# release = rhel-8.0.0
# package = thermald
# owner = prarit
# Ref: RCM-50722

#### case

case $1 in
	start)
		echo "starting"
		;;
	status|state|--status|--state)
		echo "status"
		;;
	stop)
		echo "stopping"
		;;
	*)
		echo "not a valid statement"
		;;
esac

Another eample:

read -p "Enter a character: " var
case $var in
   [a-z])
        echo "You entered a lower case alphabet." ;;
   [A-Z])
        echo "You entered an upper case alphabet." ;;  
   [0-9])
        echo "You entered a digit." ;;
   ?)
        echo "You entered a special symbol." ;;
   *)
        echo "You entered more than one characters."
esac

#### function

function function_name{
  ls -l 
}

function_name

OR

function_name() {
  echo "hello world"
}

function_name

#### cut

# Remove first two columns
cut -d " " -f 3- foo
## -d " ": use a single space as the delimiter (cut uses TAB by default)
## -f: specify fields to keep
## 3-: all the fields starting with field 3

# Get first character from all the lines of the file
cut -c 1 /etc/passwd

# Print characters from 5th position to 9th position in all the lines
cut -c 5-8 /etc/passwd

# Print all the characters starting from 5th position
cut -c 5- /etc/passwd

# Print first 4 characters from all the lines
cut -c -4 /etc/passwd

# Print 1st, 4th and 8th character
cut -c 1,4,8 /etc/passwd

# Print 2nd column separated by tab
echo -e 'one\ttwo\tthree' |cut -f 2

# Separated by comma
echo -e 'one,two,three' | cut -d , -f 2

# prints 3rd column
echo -e 'one two three' | cut -d " " -f 3

# Change output delimiter for cut and awk
cut -d : -f 1,5 --output-delimiter , /etc/passwd

#### diff
1. Print if there is a difference between two files

diff -rq file1 file2
Files file1 and file2 differ

2. Print common lines and diffs between two files

 diff -u file1 file2        
--- file1	2020-06-05 16:53:31.757459971 +0530
+++ file2	2020-06-05 16:53:43.167411642 +0530
@@ -1,3 +1,3 @@
-hey buvanesh
-how are you
+hey ravi
+how are you?
 ok ok
# + = present in file2
# - = present in file1
# <no-sign> = common line in both files

#### du

# find directry size of each
du -h --max-depth=1

#### AWK

# Remove empty and whitespaces in starting line of the file

$ cat foo.txt 
test
    new
   file
buvi
$ cat foo.txt |awk '{$1=$1;print}'
test
new
file
buvi

# Change output delimiter use -v OFS=,
awk -F : -v OFS=, '{print $1,$3}' /etc/passwd

# Print 3rd column which has value greater than 500
awk -F : '$3 > 500' /etc/passwd

# Search and print
awk -F : '/lisa/ { print $4 }' /etc/passwd
#^ it will search for the line having lisa, and it will print 4th column in the line.

# Customize the awk output
awk -F : '{print "column one:"$1 " column three:" $3}' /etc/passwd
awk -F : '{print "UID: "$3 ";LOGIN:" $1}' /etc/passwd

# Print last field
awk -F : '{print $NF}' /etc/passwd

# Print 2nd column from the last field
awk -F : '{print $(NF -1)} /etc/passwd'

#### grep
Refer: https://antonz.org/grep-by-example
Print lines ends with `w`

grep w$ /file

Print lines that starts with `first` and ends with `last`

cat file.txt |grep ^first |grep last$
# NOT WORKING
grep '^first,last$' /file

Grep multiple string

netstat -nutl |grep -vR '^Active|^Proto' 

Print the line number as well

$ grep -n string file.txt
1: file something
6: some file ok

Print files which has only the matching string

$ grep -l -R something
$ ack -l something

#### find

#find and replace all the occurrence of foo with bar in a path folder
find /path -type f -exec sed 's/foo/bar/g' {} \;

# find only directories with the filename hello
find / -name hello -type d

# find files based on their permissions (777) and find the opposite
find / -type f -perm 0777
find / -type f ! -perm 777 

# find all the empty files and directories
find / -type f -empty
fnd / -type d -empty 

# find all the executable files
find / -perm /a=x

# find and delete all the files that ends with log
find /path -name '*.log' -exec rm -rf {} \;
# OR
find /path -name '*.log' |xargs rm -rf

# Find the files with *bar* except the ones having *foo* in its filename
find . ! -name '*foo*' -name '*bar*'
# find files and remove its part of its name
find . -name 'y2mate.com - *' |while read video
do
  mv $video ${video/"y2mate.com - "/}
done
# here it removes y2mate.com - from all the filenames

# find files and rename its part of filename
find . -name '*.log' |while read file
do
  mv $file ${file/.log/.LOG}
done

# find files and list their permissions
find . -name '*foo*' -ls 

# find files which are not owned by anyone
find . -nouser

# list files which are owned by bsivasub and has permission of 4000 (4 denotes suid)
find . -perm /4000 -user bsivasub -ls

# find executables which are not installed by package managers
# 
# find files which are changed in last 15 mins
find /etc -cmin -60

# find files which are modified within 5 days
find . -type f -mtime -5 -exec ls {} \;

# find files which are modified for more than 5 days
find . -type f -mtime +5 -exec ls {} \;

# find files which are modified exactly 5 days ago
 find . -type f -mtime 5 -exec ls {} \;

# mtime, atime, ctime
ctime: Many times this is understood as a creation time but that wrong.
Ctime is change time of file stats such as ownership, what permissions it has etc.

mtime: File modification time. 
Value of mtime is updated when content of file changes.

atime. File access time. 
Value of atime is modified when file is opened. 

#### sort

## Sort by alphabets
sort /file

## Reverse 
sort -r /file

## Sort by number
sort -n /file

## Print only uniq numbers
sort -un /file

## Sort specific column 
sort -n -t : -k 3 /etc/passwd

#### uniq

# Find how many times a line is repeated
uniq -c /path

Get top 3 most accessed resource from apache log

cat access_log |cut -d '"' -f 2 |awk '{print $2}' |uniq -c |sort -rn |head -n 2

#### set

line="shell:scripting:is:fun"
IFS=:
set $line

echo $1
echo $2

An another example:

read -p "Enter username: " username
line=$(grep $username /etc/passwd)

IFS=:
set $line

echo "Username: $1 \nUser ID: $3 \nHome Folder: $6"

### nmcli

# get IP address, gateway, DNS, and domain of the machine
$ nmcli con show
$ nmcli device show <device-got-from above output>
IP4.ADDRESS[1]:                         10.0.1.24/24
IP4.GATEWAY:                            10.0.1.1
IP4.DOMAIN[1]:                          ec2.internal
IP4.DNS[1]:                             10.0.0.2

### Redirection
redirect stderr

ls / 2> /var/err.log

Discard stdout and stderr

ls / &> /dev/null

Discard stderr

ls / 2> /dev/null

### Exercises
Exercise 1:

if ! [[ $UID = 0 ]] 
then
  echo "Please run the script with sudo or as root"
  exit 1
elif [[ $# -lt 1 ]]
then
  echo "Please provide atleast one argument. 
        Usage info: script [user-name]"
  exit 1
elif [[ $# -eq 1 ]]
then
  echo "Creating a new user named $1"
  useradd $1
fi
shift
echo "Parameters passed additionally are: $@"

Exercise 2:

log(){
  if [[ $VERBOSE = 'true' ]]
  then
    echo $@
  fi
  logger -t script.sh $@
}

backup_file() {
  if [[ -f $1 ]]
  then
    log "Backing up $1 to /var/tmp/$(basename $1).$(date +%F-%N)\"
    cp -p $1 "/var/tmp/$(basename $1).$(date +%F-%N)\"
  else
    return 1
fi
}

VERBOSE=true
log 'Hello!'
log 'This is fun!'

backup_file '/etc/passwd'

Exercise 3:

read CHAR
if [[ ($CHAR = Y) || ($CHAR = y) ]]
then
  echo "YES"
elif [[ ($CHAR = N) || ($CHAR = n) ]]
then
  echo "NO"
fi

Exercise 4:
Find and remove files

remove_file (){
	while read -r file; do
		rm -rf $file || true
	done
}

find . -name hello |remove_file

# read -r => read handles / as a special character, to ignore use -r option.

Sample from work

#!/bin/bash

COMPOSE_REMOVE="/usr/bin/compose-remove"

ShowHelp () {
    echo "
This script performs nightly clean-ups of old composes of RHEL 8.

Available options:

-d | --dry-run  Debug. Only print commands that would be executed.
-h | --help     Help. Shows this help.
-v | --verbose  Simply more output.
"
    exit
}

LoggingDebug () {
    if [[ "$VERBOSE" ]]; then
        echo "$@"
    fi
}
RemoveCompose () {
    if [[ "$DRY_RUN" ]]; then
        while read -r filename; do
            LoggingDebug "Would delete $filename"
            "$COMPOSE_REMOVE" --force --test "$filename" || true # Ignore errors
        done
    else
        while read -r filename; do
            LoggingDebug "Removing $filename"
            "$COMPOSE_REMOVE" --force "$filename" || true # Ignore errors
        done
    fi
}

while [[ "$#" -gt 0 ]]; do
    case $1 in
        -h|--help) ShowHelp;;
        -d|--dry-run) DRY_RUN=true;;
        -v|--verbose) VERBOSE=true;;
        *) echo "Unknown parameter: $1" >&2 ; exit 1 ;;
    esac
    shift
done

#
find /mnt/nigtly -maxdepth 1 -name "something*" -mtime +10 | RemoveCompose

Addition
1. ``expr``
2. ``echo $(( X + Y))``

### Articles
https://askubuntu.com/a/707753/869835
*** A very good article about set options in shell
https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/

### Questions
1. Find how many parameters are passed to a script
2. Delete last line
3. Delete content of a file

    cat /dev/null > file.txt
    # OR
    > file.txt

Exampls
1. Ref: https://raw.githubusercontent.com/coreos/etcd-operator/master/example/rbac/create_role.sh

#!/usr/bin/env bash
set -o errexit
set -o nounset
set -o pipefail

ETCD_OPERATOR_ROOT=$(dirname "${BASH_SOURCE}")/../..

print_usage() {
  echo "$(basename "$0") - Create Kubernetes RBAC role and role bindings for etcd-operator
Usage: $(basename "$0") [options...]
Options:
  --role-name=STRING         Name of ClusterRole to create
                               (default=\"etcd-operator\", environment variable: ROLE_NAME)
  --role-binding-name=STRING Name of ClusterRoleBinding to create
                               (default=\"etcd-operator\", environment variable: ROLE_BINDING_NAME)
  --namespace=STRING         namespace to create role and role binding in. Must already exist.
                               (default=\"default\", environment variable: NAMESPACE)
" >&2
}

ROLE_NAME="${ROLE_NAME:-etcd-operator}"
ROLE_BINDING_NAME="${ROLE_BINDING_NAME:-etcd-operator}"
NAMESPACE="${NAMESPACE:-default}"

for i in "$@"
do
case $i in
    --role-name=*)
    ROLE_NAME="${i#*=}"
    ;;
    --role-binding-name=*)
    ROLE_BINDING_NAME="${i#*=}"
    ;;
    --namespace=*)
    NAMESPACE="${i#*=}"
    ;;
    -h|--help)
      print_usage
      exit 0
    ;;
    *)
      print_usage
      exit 1
    ;;
esac
done

echo "Creating role with ROLE_NAME=${ROLE_NAME}, NAMESPACE=${NAMESPACE}"
sed -e "s/<ROLE_NAME>/${ROLE_NAME}/g" \
  -e "s/<NAMESPACE>/${NAMESPACE}/g" \
  "${ETCD_OPERATOR_ROOT}/example/rbac/cluster-role-template.yaml" | \
  kubectl create -f -

echo "Creating role binding with ROLE_NAME=${ROLE_NAME}, ROLE_BINDING_NAME=${ROLE_BINDING_NAME}, NAMESPACE=${NAMESPACE}"
sed -e "s/<ROLE_NAME>/${ROLE_NAME}/g" \
  -e "s/<ROLE_BINDING_NAME>/${ROLE_BINDING_NAME}/g" \
  -e "s/<NAMESPACE>/${NAMESPACE}/g" \
  "${ETCD_OPERATOR_ROOT}/example/rbac/cluster-role-binding-template.yaml" | \
  kubectl create -f -

2. Create users from LDAP output, ask user for confirmation before creating the users

VAR="cn=lisa,dc=example,dc=com
cn=linda,dc=example,dc=com
cn=laura,dc=example,dc=com"

for i in $VAR
do
  USER=$(echo $i |awk -F = '{print $2}' |awk -F , '{print $1}')
  echo $USER
done

AddUser(){
  for user in $VAR
  do
    USER=$(echo $user |awk -F = '{print $2}' |awk -F , '{print $1}')
    useradd $USER
    echo "$USER added successfully"
  done
}
echo "Can I go ahead and create these users? (Y/n)"
read answer

case $answer in
  yes|YES|y|Y) AddUser ;;
  no|n|NO|N) echo "Not adding any users" && exit ;;
esac

3. Create a script (using getopts) to create users

# script that creates users using preferred options
# usage: use -a to add home dir
#        use -b to make the user member of group 100
#        use -c to specify shell

while getopts abc: opt
do
  case $opt in
	  a) VAR1=-m ;;
	  b) VAR2="-g 100" ;;
	  c) VAR3="-s $OPTARG" ;;
	  *) echo "usage: $1 [-a] [-b] [-c shell] username"
esac
done

if [ $# -lt 1 ]
then
	echo "usage: $1 [-a] [-b] [-c shell] username"
	exit 1
fi

echo "The current arguments are set to $*"
shift $((OPTIND -1))
echo "Now the current arguments are set to $*"
echo "useradd $VAR1 $VAR2 $VAR3 $1"