Notes for learning ULI 101:Introduction to UNIX/Linux and the Internet in Seneca College - Fall 2022
Original Course Note: [ULI101 CDOT-Wiki](https://wiki.cdot.senecacollege.ca/wiki/ULI101)
:::spoiler
[TOC]
:::

---
||Good luck to you all :)||
:::info
:memo: **Note:**
- Commands can combine with "|" or ";"
- No space between muti arguments e.g. -la instead of -l a
:::

# Basic Command
### man
>- menu

### pwd
>- current directory

### cal
>- calender in number
>- `cal dd mm yyyy`
>- **Note:** 1. in **number** 2. **full year** i.e. 2022 instead of 22

### date
>- current date

### who
>- users in the same node

### whoami
>- your username

### passwd
>- change pw

### uniq
>- unique

### history
>- history command

### fc
>-history command
fc: last command
fc -l: last 16 command

### clear
>- clear the screen
>- same as <ctrl><L>

### file
>- determine file type

### env
>- all variables

### $variables
>1. `number=5` (**no space**)
>2. `$number`
>3. `number="The number is 5"` (**double quote for more than 1 word**)

# File related

## Pathname
* Relative to home :
* `~/directory or filename`
* Relative:
* `./ -> current location`
* `../ -> parent(upper level) location`
* Absolute: full path
* `/home/username/directory or filename`

## Create
### mkdir
>- create new directory
>- `-p`: create parent dir

### touch
>- create new blank file

## Save file
>- override or append to file
>- e.g. `sort file1.txt > newfile.txt`

    1. >: override and save
    2. >>: append and save
    3. <: inputted

### tee
>- display the command and save it to a file at the same time

## Remove
### rmdir
>- remove empty dir

### rm
>- remove file
>- `-r`: remove diretory (can delete empty)
>- `-i`: add confirm b4 delete

## Copy or move
### cp
>- copy from a to b
>- -r also copy content inside directory

cp ~/{file1,file2} ~

### mv
>- move from a to b
>- rename
>- `-p`: create parent directory (both 2 directories: mother and son are new)

# Output

## Show structure
### ls
>Key:
>- `-l`: detail
>- `-a`: show hidden
>- `-i`: iNode (a number will appear in the 1st col -> physical address in hard drive)
>- `-R`: recursive
>- `-d`: directory

### tree
>- show whole path tree

>- Key:
> - `-C`: directory / executed file with color

## Print
### cat
>- open file
>- `-n`: show line number
>- <span style="color:blue">show ***first 10*** lines
>- multiple lines:
> - cat <<+
> line1
> line2
> \+

### echo
>- print command
>- `-n` not print the next item newline
>- `-e` can use "\n" as newline

### quotation marks
doesn't matter for declare variables
> - <span style="color:blue">strong quote
> - '' or //
> - output $variable name e.g. $number
> - <span style="color:red">weak quote
> - ""
> - output $variable value e.g. 7

### print system variables
> - e.g. date
> - use echo \`(date)\` to print function date

## more / less
>- show all
>- <span style="color:blue">***more:*** show % you have seen

### diff
>- show differences between 2 files <span style="color:blue">***(show differences only)***
>- `-y`: show 2 file parallelly <span style="color:blue">***(show all: side by side compare)***</span>
> -diff -y fileA fileB

### wc (Word count)
>Key:
>- `-l`: #line
>- `-w`: #word
>- `-c`: #byte
>- `-m`: #char

## Text editor
### vi / vim
>|Key|Remarks|
>|:--:|:--:|
>| :x | save and exit
>| :q! | exit without save

### pico / nano
>|Key|Remarks|
>|:--:|:--:|
>|ctrl + x | save and exit|
>| i | insert (edit) mode|
>| esc | command mode

## Permission
>- `chmod`
> - change permission (what action can do)
> - `-R`: include sub files (recursive)

| Permission | Meaning |
|:---------------:|:-------:|
| r | read |
| w | write |
| x | execute |

| User Type | Meaning |
|:---------:|:-------:|
| u | user |
| g | group |
| o | others |
| a | all |

### Octal

| user | group | other |
|:----:|:-----:|:-----:|
| rwx | rxw | rwx |
| 421 | 421 | 421 |

chmod 777 filename

### Symbolic
| Permission Edit | Meaning |
|:---------------:|:-------:|
| + | add |
| - | remove |
| = | set |

chmod g+w,o+w file
chmod g-w file
chmod g=w file

## chown
> who can perform action (rwx)
> - `chown root /dir_name`
> - changed owner of /dir_name to "root"
> - `chown root:staff /dir_name`
> - changed owner of /dir_name to group "staff" under "root"
> `-R`: include sub files (recursive)

## umask
> reverse octal

# Data Cleansing
## wildcard features
>Key:
>- `*`: 1 or more occurance
>- `?`: single char
>- `[class]`: things inside (case sensitive)
> - `[!abc]`: not inclue abc
> - `[A-Z]`: things include A-Z

## head -4 / tail -4
>- show first / last 4 (n) lines

## sort
>- Key:
> - `-n`: numberic sort: small -> large (50 > 250, not 250 first)
> - <span style="color:blue"> ***`-r`***: reverse the result
> - default display is in ascending -r change to descending
> - <span style="color:blue">***`-R`***: random sort
> - <span style="color:orange">**`-t`**: field separator (same as -d in cut) **t in sort**
> - e.g. `-t","` -> the #field will determine by ","
> - <span style="color:#E7717D">`-f`: ignore cases
> - <span style="color:red">**-k**: key (same as -f in cut: #field)
> - e.g. `-k4` -> the 4th column
> - `-h`: human numbers (compare numbers with different units)

## cut
>- Trim data to required format

>- Key:
> - <span style="color:orange">**`-d`**: field (column) separator (same as -t in sort)
> - e.g. `-d ','` -> identify things before and after ',' as 2 fields (column)
> - <span style="color:red">**`-f`**: fields (#col) (same as -k in sort)
> - usually used with #field
> - e.g. `-f1,2` -> show the first 2 field (column)
> - e.g. `-f1-3` -> show first 3 field

## find
>- find ~/location -name "filename"
>- find ***file*** (file name version of grep)
>- Return with ***file path***

>- Key:
> - `-name`: exact word
> - `-iname`: approx. word
> - `-p`: search inside the directory

## grep
>- find pattern ***in file*** or ***in other command output*** (in file / command output version of find)

>- Key:
> - <span style="color:#E7717D">`-i`: ignore case
> - `-n` = line number
> - `-c` = count result
> - `-v` = inVerted search
> - `-r` = recursive (inside a file)
> - `-w` = exact whole word

grep "haha" file.txt

>- wild card
> - `.c` any char appear once
e.g. `".i"` must hv something b4 i
> - `c*` char **b4** it is optional (0 or more)
e.g. `"a*b"` can show "b" "ab" "abb"
> - `.*` including anything between it (bc . is b4 the star)
e.g. `"a.*y"` will appear "abby", "anny", "addy"

## egrep
> - advance version of grep

> - `|` include 1 of them
e.g. "hello|world" will show "hello" "world"
> - `[]` treat as single word with fulfil the requrement
e.g. [abc]c -> ac bc cc ok
> - `^c` must start with that char
e.g. `^f` -> "fun", "fam"
> - `c$` must end with that char
e.g. `f$` -> "of", "if"

## awk
awk '/word/ {print col/rowNum}' filename

>- `-F` delimiter **must be block letter

>- NR: #row (can use as index)
>- NF: total #field
> - e.g. total col is 4, `$(NF-1)` is `$3`, `$(NF-4)` or `$(NF-NF)` is `$0`
> - ***need brackets for cal field***
> - ****`$NF` is val of last field****
>- `$fieldNo` means the number of col
> - $4 means 4th col
>- `$0` prints all col

>- `,` between field create space
e.g. `awk '{print $3,$4}'' file` will have space
`awk '{print $3$4} file'` or `awk '{print $3 $4} file'` will have no space (squeeze)
>- " " element between field can be a separator
e.g. `awk '{print $3 " " $4}'' file` (10 spaces between them)
will have output: f3 f4 (10 spaces between them)
>- compare number: '$r == number' filename
> - can be <, <=, >, >=
>- compare string: '/word/' filename
>- can also print words / symbols (\t) i.e. `awk '{print "Hello" $1}' file` will print Hello f1
> - make sure word is ***inside ""***
>- can use wild card like `/^[0-9]$/`

## sed

sed 'location word' filename
    
location: #line

> - `p` print that line twice
e.g. `'3,6 p' filename`
> - `-n` only ***print line*** 3-6 once
e.g. `-n '3,6 p' filename`
> - `-i` ***replace and save*** the folder (make changes permanent)
e.g. `-i 's/[0-9]/*/g' filename`

> - `d` ***delete*** line 3-6
e.g. `'3,6 d' filename`
> - `q` print head n
e.g. `'5 q' filename` print head 5 (quit until first 5 lines is printed)
> - `I` ***not case sensitive***
e.g. `'/abc/I p' filename`
> - `s` ***subsistute*** (at beginning)
e.g. `'s/[0-9]/*/'` subsitute first number in that line to * (similar to tr)
> - `g` replace all
e.g. `'s/[0-9]/*/g'` subsitute all numbers to `*`

> - `/word/` include those word

> - `'/abc/ q'` -> quit until having abc
> - `s/abc/efg&/` -> will add efg before abc (& = the 1 word group inside / /)

## tr
>- replace and output
>- ***Note:*** have input (<) before file name

>- Key:
> - `-s`: squeeze to delete **duplicate** characters
> - e.g. if there is 1 space and 2 space in the data set, `tr -s ' ' ','` all of the, will replace by 1 ','
> - **Note:** have space between s and ''

tr 'ab' 'CD' < file
    ab is replaced by CD in file data.txt
    a->C e->D: it is replaceed correspond to its position

# Remote access
>- servername: e.g. [e-mail]
>- command: e.g. ls -l

## scp
>- copy file securely between 2 servers

## ssh
>- access ***each time***
### Basic

scp file servername :
ssh servername command

### Transfer file
>- from local device in current directory

scp myfile.txt server:remote/myfile.txt
ssh servername command

## sftp
>- access ***once***

>1. Add 'l' before command for local command
>e.g. lpwd for local pwd
>2. Transfer file to remote server: `put filename`
>3. Obtain file from remote server: `get filename`

# Email
>1. Without attachments:
`mail email_adress`
`mail -s "suject" username@hostname`

>2. With attachments:
> `mail -a filepathname username@hostname`
> `mail -s "subject line" username@hostname < filepathname`

>- Send using ctrl + d

# Link File
## ln

    ln <filename> <hardlink_filename.hard.lnk>  //hard
    ln -s <file> <softlink_file.sym.lnk> //soft

>- create link file together
>- use as backup file
>- `ln file1 link2` link file1 to link2 (like shortcut in win)
same iNode: Point to location of the drive
f1 -> file <- f2
>- `-s`: source distination: have a `->`
diff iNode: Point to the direction of the file (exactly the same path) :
f2 -> f1 -> file
permission of the link will refer to the actual file:
it show full permission

# Job
- PID is an unique process
## jobs
>- show the current job (on going process) doing (background)

## ps
>- show snapshot of ongoing process with PID (foreground)
>- `ps`: single terminal

>- `-w`: more detail (with location)
>- `-U username`: show user
>- `-x`: same name same node ***multiple terminal***

## kill
>- kill / termine the process -> stop the current running job
>- kill %jobid
>- kill PID

>- `pkill` can kill with name
>- `-9` kill the job
>- `-15` terminate job
>- `%1` default is the 1st job (current)

## top
>- show detail info of ongoing process

## tail
>- only works if it is in the same server(node)

>- `-f filename` continously keep track the **ongoing** record
if one more line is added, it will show that line immediately

# Bash / Script
>- .sh file
>- storing multi command for multiple times
>- file usually starts with #!/bin/bash (shebang)

>- use $CAPITAL_LETTER for system variables
i.e. echo "You are: $USER"

## read
`read variable`
>- Store variable
>- `-p` print and store variables at the same time

    read -p "Hello user:" userName

>- `read` means pause the program until enter or any key is pressed

## output
`echo ""`
>- `echo "$variable"`
for output variables

## input
`file.sh input1 input2`
>- `$1`: 1st input (input1)
>- `$10`: 1st input + 0 e.g.input1 is dog $10 is dog0
>- `$(10)`
>- `$#`: total num of argument

## if

    #!/bin/bash
    
    command (e.g. grep)
    (can be related to result of the command)
    
    if [ condition ] (use = to compare)
    then
            .....<-True
    elif [ condition ]
    then
    else
            .....<-FALSE
    fi

condition can be:
[[condi && condi ]] (use == to compare)

>- `-f $filename` file exist
(also return true if nothing type) may need a condition to screenout wrong input

>- `-d dir` directory exist
>- `-s` file exist but <0 bytes ( empty file )
>- `-r` file exist and +r permission
>- `-w` file exist and +w permission
>- `-x` file exist and +x permission
>- `!-f` file does not exist (can add ! before any - command)

>- `-le` less than or equal to (dun use <=)
>- `-lt` less than
>- `-ge` greater than or equal than
>- `-gt` greater than
>- `-eq` equal to
>- `-ne` not equal to

## for

for VARIABLE in 
do
    
done

## $@ or $*
>- all input argument
> $1 $2 1st and 2nd arg

## $0
>- filename

## $#
>- num arguments outside

## $?
>- exit code

## exit
>- skip the other code directly leave
>- `exit 0`: can set to different numbers
>- `echo $?` to check the exit code (default: 0 for no error)

# Other terminals (Optional)
## ksh
## csh
>- `exit` to quit the other terminal