---
# System prepended metadata

title: UNIX I
tags: [UNIX]

---

[Toc]

:::success
__MY SYSTEM CONFIGURATION__
OS:  [Ubuntu 22.04.3 LTS](https://ubuntu.com/desktop) (run on virtual machine [UTM](https://getutm.app))
RAM: 8GB
CPU: Apple M1
:::

# UNIX I
__the unix philosophy__
* To provide tools that each do one thing well.
* To let you __combine these tools together__ to form more powerful Unix commands.
* To view all communication as being via __files__.
    * Each tool can take input from one file and put its output into another file.
    * A file is modeled as just a character stream; it doesn’t have to be actually stored on disk.
    * Eg, the keyboard input & screen output. 

## basic unix commands
![image](https://hackmd.io/_uploads/SJ8_DPl0T.png)
these directories are equivalent
```bash
$ /home/yen    # absolute path
$ ~yen
$ ~            # start the path at the user level
```

### directories / file creation and deletion
```bash
$ pwd
$ cd /home/yen/os/hw1 # could be absolute or relative path
$ cd ~/os/hw1 
$ cd - # go back to your previous pwd
$ cd .. # go back to parent pwd
$ cp [path]/oldfile [path]/newfile # given a new name when copy a file
$ mv
$ rm (-iIf) # '-f' means force, '-i' means prompts before every removal
$ rm -r # remove a dir
$ mkdir
$ rmdir
```

* __wildcard symbol__
1.  ```?``` fills the place of any __one__ character
2. ```*``` fills the place of any number of characters(including zero)
e.g.,
```ls *a*``` all filenames with 'a' in them
```ls *a*html``` all filenames with 'a' in them and ending with 'html'
```?????``` all __5__ character filenames
```ls[a-d]*```all filenames __starting__ with the range of a-d
```ls[^a-d]*```all filenames <font color=#f00>__NOT starting__ </font> with the range of a-d

==EXAMPLE==
suppose we type ```$ls a*``` and we get aa ab and adirectory, which is two files and one directory. Then we type ```$cp a* a*```, what things will we get?

ANSWER:
```bash
$ cp aa ab adirectory aa ab adirectory
cp: omitting directory 'adirectory' # if make no sence to copy a dir
cp: warning: source file 'aa' specified more than # copy a file twice has no effect
cp: warning: source file 'ab' specified more than once
```
what acctually do is ```$cp aa ad adirectory```


### commands for viewing files
```bash
$ cat <filename>
$ cat -n <filename> # display with line numbers.
# if no argument is given, it will takes its input from keyboard.
# this command will show below on redirection topic

$ more <filename>
$ less <filename>

$ head <filename> # display the first 10 lines of a file
$ head -n <filename>
$ tail <filename>
$ tail -n <filename>

$ paste <filename> # display the file in columns
```

### managing file and directories
* ```ls (-lrtAS)```
    ```-S``` with Capital letter 'S', sorts file listing by ++size++
    ```-A``` shows ++all++ files including hidden files
    ```-lrt``` shows ++long++ format and sorts file listing by last modified ++time++ with ++reverse++ order, which is useful for finding a file recently modified without scroll off the 
    ![image](https://hackmd.io/_uploads/S1eBw1yJC.png)
    letter ```drwxr--r--``` indicates owner's, groups's and others' permissions(read, write, execute).
    starting letter with ```d``` for a dir, with ```-``` for a file and with ```l``` for a symbolic link.
* ```ln -s```
like the picture above, ```ln -s .. abc``` means link the name abc to ..
* ```chmod``` change file permission with character like ```chmod u+x file``` or give a three-digit octal number like ```chmod 777 file```
_*note: an executable file __needs__ read permission as well._
* ```find . -name``` 
    e.g., 
    ```bash
    $ find . -name "*y?il*"
    ./myfile3
    ./subdir1/any_il_
    ./subdir1/myfile
    ./subdir2/subsubdir/myfile2
    ./subdir2/subsubdir/yail
    ```
:::danger
```diff``` UNREAD
:::
* ```diff -yWqc``` takes two text files and show their differences
    ```-c```	To display changes in context
    ```-u```  Slightly different display of changes in context
    ```-q```	Be quiet --just print whether the files have differences
    ```-d```	A different algorithm to identified the lines that changed 
    ```-w```	Ignore white space
    ```-y```	Side by side comparison 
    ```-W=num```  Set the display width (useful with -y)

* ```fgrep (--color-invwoeABC)``` fixed string search, with two useful ways of being used
    (more tips in UNIX II-grep)
    1. which file(s) contain that word?
    ```bash
    $ cd ~/UNIX_L1/subdir1; fgrep  "that word" *
    fileX:No, not that word.
    fileX~:Know, not that word.
    fileY:No, its not that word. Try again.
    fileY:You already tried that word!
    ```
    2. where in my file is the word I'm looking for?
    ![截圖 2024-03-22 晚上9.14.52](https://hackmd.io/_uploads/S1tL4biCp.png)
    ```-n``` will list the line number(s) of any character
    ```--coior``` will highlights the matched-part of the line
    ```-w``` for the __whole-wold__ match


## miscellanuous commands
* ```echo (-n)```
    with quotes "" can see whold sentance in one echo, with ```-n``` will have no newline
* ```wc (-lwc)``` word count
    ![截圖 2024-03-22 晚上11.02.07](https://hackmd.io/_uploads/Bysd6fiCp.png)
    2 lines, 13 words, 63 characters and filename
* ```cut (-cfd--complement)``` cut columns or fields from lines of a file
    ```-c``` characters or ```-f``` field from each input line (seperated  with **tab character**), ```-d``` for delimiter(分隔)
    ```--complement``` 
    ![截圖 2024-03-24 晚上8.03.48](https://hackmd.io/_uploads/BJS8vc60T.png)
* ```alias``` create a command shortcut
    1. define default flags with alias such as ```alias rm='rm -I'```, then when user type ```rm``` the UNIX system automatically adds the ```-I```
    2. define personal flags, if user define ```alias cdnshow='cd; ls'```, then both things will happen when user type ```cdnshow```
    3. set as the automatic default: add it into startup file '__~/.bashrc__' or '__~/.bash_profile__'.
    > '.bashrc' defines the settings for a user when running a subshell. Add custom configurations to this file to make parameters available in subshells for a specific user.
    > '.bash_profile' defines the settings for a user when running a login shell. Add custom configurations to this file to make parameters available to a specific user when running a login shell.
* ```basename``` extract the dir name from a path
    ![截圖 2024-03-22 晚上11.29.47](https://hackmd.io/_uploads/rJ9gEXiRT.png)
* ```history``` 
    ![截圖 2024-03-22 晚上11.30.32](https://hackmd.io/_uploads/BkvXEmiCT.png)
* ```!``` return earlier commands
    ![截圖 2024-03-22 晚上11.31.41](https://hackmd.io/_uploads/HJaPV7jRT.png)
* ```man```
* ```gzip/gunzip``` compress & uncompress a file
    ![image](https://hackmd.io/_uploads/SJxeymJ1yA.png)
* ```tar (-cxvf)``` combine dir into one file or expand the file to a dir
    ```-cvf``` for packing
    ```-xvf``` for unpacking
    ![image](https://hackmd.io/_uploads/HJ1WEkk10.png)
* ```d2u/u2d``` convert DOS & UNIX ++file formats++
    _*note: ```d2u/u2d``` is <font color=#f00>ONLY</font> for Cygwin. If you want to use this command in unix/linux based system, the commands are ```dos2unix/unix2dos```, and you hava to install it first. Typing ```sudo apt install dos2unix``` in your terminal._
    e.g.,
    UNIX place a ```\n``` at the end of each line; Windows place a ```\r\n``` at the end of each line
* ```sort(-rgksR)```
    ```-r``` sort in reverse
    ```-k``` sort on different field e.g., ```sort -k3``` to sort on the 3^rd^ field
    ```-g``` performs a numeric sort
    ![image](https://hackmd.io/_uploads/HkSB9kyy0.png)
    ```-s``` keeps lines that tie in original order
    ![image](https://hackmd.io/_uploads/Hy5-3JkJR.png)
    ```-R``` sort in random order
    ![截圖 2024-03-25 晚上8.23.02](https://hackmd.io/_uploads/SJI331JkR.png)
* ```uniq (-c)``` delete repeated (duplicated) lines (leave unique lines)
    _*note: it does <font color=#f00>NOT</font> search for the whole file while the <font color=#f00>__preceding__</font> line(前一行). Therefore, ```uniq``` is often used with ```sort```_
    ![截圖 2024-03-25 晚上8.30.04](https://hackmd.io/_uploads/S15I0kyyA.png)
* ```tr (-dc)``` replace (++TR++anslate) or delete character
    1. any character that matches to the 1^st^ list gets replaced with the corresponding character at the position in the 2^nd^ list.
    ![截圖 2024-03-25 晚上8.49.33](https://hackmd.io/_uploads/r1O1Xe1kC.png)
    ![截圖 2024-03-25 晚上8.48.45](https://hackmd.io/_uploads/HkC2fg110.png)
    ![截圖 2024-03-25 晚上8.49.47](https://hackmd.io/_uploads/BkLl7ly1C.png)
    the command ```cat jekyll | tr "a-z" "A-Z" > jekyll_up``` equivalent to the command ```tr "a-z" "A-Z" < jekyll > jekyll_up```
    2. delete character with ```-d``` or ```-c``` (complement)
    e.g., 
    ```tr -d "\n" < jekyll > jekyll_oneline``` delete newline character
    ```tr -d -c "a-zA-z \n" < jekyll > jekyll_word``` delete everything _++except++_ letters, spaces and newline (remove punctuation and numbers)
    3. some useful method: make a wordlist
        ```cat jekyll | tr " " "\n" | tr -d -c "a-zA-z \n" > word```
        ```cat word | tr "a-z" "A-Z" | sort | uniq > lexicon```
        or
        ```
        $cat jekyll | tr " " "\n" | tr -d -c "a-zA-z \n" > word |\
        > cat jekyll | tr " " "\n" | tr -d -c "a-zA-z \n" > word
        $
        ```
    _*note: the argument to ```tr``` are <font color=#f00>a __++list++__ NOT a ++string++</font>._
    ![截圖 2024-03-25 晚上9.14.07](https://hackmd.io/_uploads/S15j_gkk0.png)
* ```expr``` calculate the value of its arguments, and picky about ++space++
    ```bash
    $ echo "1 + 4/2 = `expr 1 + 4 / 2`"
    1 + 4/2 = 3
    ```
    _*note: if use ```*``` in expr, remember that ```*``` is a special char. use```\*```._
* ```seq``` create a sequence of numbers, 1 per. when 3 arguments are provided, it counts from the first to the third, using the second as the stop size.
* ```touch``` update a file's ++timestamp++ or ++create an empty file++ if it doesn;t exist.
* ```date```
* ```bc``` ++b++asic ++c++alculator
* ```which``` identify where an executable is located
    In UNIX, every command has an executable and those are store in either '++/usr/bin++' or '++/bin++'. whenever running a command, your _++shell++_ will look everywhere in your _++path++_ for the executable
    > when running a script, the './' indicated that they are in the _current dircetory_
    * ```$PATH``` (one of system variables): telling us the system where to look for the executable
    ![image](https://hackmd.io/_uploads/SyQ0xZ1k0.png)
    when we type ```scriptA```, the system will find the executable in ```$PATH```. having looked in all 9 of those directories(example above), no file was found the name 'scriptA'. Hence the system concluded that ```scrpitA``` is not a command.
    \
    ==<font color=#f00>NOTICE</font>==: remember that 'scriptA' is an executable...? However, UNIX never look here for it. After all, files in different directories can have the same name.
    ---
    Q: why do we need to type './' every time?
    A: we can copy our 'scriptA' executable over to the '/usr/loacl/bin' directory (or any other directories that is in our $PATH)
    \
    Q: why can't the system just assume that the script is in the current directory? how do I know which one of the directories within my $PATH is the directory containing the actual first match to the command name?
    A:the job of ```which```
### Shell
a shell is ++_command interpreter_++ also a ++_programming language_++.
some popular shell:
* sh: Bourne Shell
* ksh: Korn Shell
* csh, tcsh: C shell
* bash: Bourne-Again Shell
* zsh: Z shell
![image](https://hackmd.io/_uploads/r1hHw-kyR.png)
> usually, the '#' symbol is a commnt
> put the '#!' on the __first__ line of script to choose the shell, it must be the first two characters of the script.

![截圖 2024-03-25 晚上10.38.08](https://hackmd.io/_uploads/H1sU3WyJA.png)
## redirection
* ```<``` 
    ![截圖 2024-03-22 晚上11.43.27](https://hackmd.io/_uploads/B1TQP7jRT.png)
    ![截圖 2024-03-22 晚上11.49.09](https://hackmd.io/_uploads/ryGKdXjCp.png)
    ```-``` indicates that this __column's__ data comes from redirection
    ```- -``` to merge pairs of lines from one file into columns on one line
* ```>, 2>, >&``` redirect screen output to  a file, and __overwrite <font color=#f00>without ask</font>__
    ```2>``` to sent ++only++ the __error messages(stderr)__ to the file 
    (not available in some shells such as csh & zsh)
    e.g., ```echo "hello world" 2> out.file```
    ```>&``` to sent output and error messages(stderr) to the file
    e.g., ```echo "hello world" >& out.file```
* ```>>, 2>>``` redirect screen output to __the end__ of a file 
    * __comparison__
    ```>``` & ```>>``` : stdout only
    ```>&```: stdout&stderr 
    ```2>``` & ```2>>``` : stderr only(not available in some shells such as csh & zsh)
![截圖 2024-03-24 晚上10.31.00](https://hackmd.io/_uploads/rJzVKnTAT.png)

---
_*note: about connecting commands and ```ls```
the output of ```ls``` will be different when directed than when sent to the screen. Hence, when user type ```wc -l < temp``` will give 10 rather than 2._
![截圖 2024-03-24 晚上10.29.30](https://hackmd.io/_uploads/HJ60Oh6Ap.png)
_*note about ```2&```: the 'junk' file store the stderr of ```ls b* d* g* 2> junk```. Well, UNIX provides a better way: __/dev/null__._ 
![截圖 2024-03-23 凌晨2.05.49](https://hackmd.io/_uploads/BktKOHj0p.png)
### shell script
![截圖 2024-03-24 晚上8.13.26](https://hackmd.io/_uploads/r1ZeKqTCT.png)
_*note: the front protion of ```   30  ``` is columns 1-7._
We could notice that the 'tempfile' now is a __script__, which containing a series of commands. Then we can use the script whenever we want to perform the task without retyping.
In other words, a script is a _program_; however, it doesnot need to be compiled and run as a text file.

<font color=#f00>__Notice:__</font> a script have to change the file permission to be executable
![截圖 2024-03-24 晚上8.28.46](https://hackmd.io/_uploads/BkFFh5TRp.png)

* to use ```$*``` symbols in script to ++_pass arguments_++. When run the script , the ```$*``` symbols will be replaced with all of the arguments.
    ![image](https://hackmd.io/_uploads/S1vRb3lgC.png)
    _*note: the star of ```a*``` is a wildcard symbol; however the star of ```$*``` is the part of it which is not a wildcard symbol._
    ![image](https://hackmd.io/_uploads/HJL5aspA6.png)
---
* ```|``` send to the input of the next command (pipe)
    e.g., ```ls $* | wc -l``` equilivant to the countfile above
    ![截圖 2024-03-24 晚上10.32.28](https://hackmd.io/_uploads/BJttt2pCT.png)
* ```| tee``` send to a __file__ and to the next command
    ![截圖 2024-03-24 晚上10.36.37](https://hackmd.io/_uploads/r1IF5h60a.png)
    ![image](https://hackmd.io/_uploads/r1-j5n6AT.png)
* ``` `...` ``` redirect the output to become an <font color=#f00>__argument__</font>
    (which is, within the ``` `...` ``` is a command)
   ![截圖 2024-03-24 晚上10.39.07](https://hackmd.io/_uploads/ByKfs2T0p.png) 
* ```xargs (-L1)``` send to arguments of next command
    ```-L1``` flag causes the command that is being xarged to be run separately for eacg line that’s passed-in.
    ![image](https://hackmd.io/_uploads/BkJOA3a06.png)
    ![image](https://hackmd.io/_uploads/H1bbg6p0a.png)
    * __comparison__
    1. compare xargs: 
    ```find . -name "*.c" | xargs fgrep main```
    2. to ``:
    ```fgrep main `find . -name "*.c"` ```
## command coordination
* ```;``` two commands are executed in sequential order
    e.g., 
    ```cd ~/temp; ls *.c``` 
    is equivalent to
    ```
    ca ~/temp
    ls *.c
    ```
* ```&&``` command1 always executes, but command2 only executes if command1 was successful
    It is the idea of _short circuit evaluation_, whereby the evaluation of a logical expression must stop early, if the result becomes known.
    ![截圖 2024-03-24 晚上11.19.14](https://hackmd.io/_uploads/SJ4i46TCT.png)
    C++ enforces this rule, we <font color=darkorange>know</font> that __D will not be tested__.
    
    ==EXAMPLE==
    ```cp /file ./fcopy && cat fcopy```
    it only prints the file if you were ablt to copy it, and having the read permission on the file and write permission on the destination.
* ```||``` command1 always executes, but command2 only executes if command1 failed
    ![截圖 2024-03-24 晚上11.30.20](https://hackmd.io/_uploads/ry9zD6TR6.png)
    
    ---
    _*note: a ```cd``` command will not persist._
    ![截圖 2024-03-24 晚上11.36.51](https://hackmd.io/_uploads/SJEsdaT06.png)
    there have same problem in script; however, we could use ```source``` command to fix it.
    ![image](https://hackmd.io/_uploads/B1N9j6pAa.png)

## special key using at the command prompt
* ctrl+A: HOME
* ctrl+E: END

