[Toc]
:::success
__MY SYSTEM CONFIGURATION__
OS: [Ubuntu 22.04.3 LTS](https://ubuntu.com/desktop) (run on virtual machine [UTM](https://getutm.app))
RAM: 8GB
CPU: Apple M1
:::
# UNIX I
__the unix philosophy__
* To provide tools that each do one thing well.
* To let you __combine these tools together__ to form more powerful Unix commands.
* To view all communication as being via __files__.
* Each tool can take input from one file and put its output into another file.
* A file is modeled as just a character stream; it doesn’t have to be actually stored on disk.
* Eg, the keyboard input & screen output.
## basic unix commands
these directories are equivalent
```bash
$ /home/yen # absolute path
$ ~yen
$ ~ # start the path at the user level
```
### directories / file creation and deletion
```bash
$ pwd
$ cd /home/yen/os/hw1 # could be absolute or relative path
$ cd ~/os/hw1
$ cd - # go back to your previous pwd
$ cd .. # go back to parent pwd
$ cp [path]/oldfile [path]/newfile # given a new name when copy a file
$ mv
$ rm (-iIf) # '-f' means force, '-i' means prompts before every removal
$ rm -r # remove a dir
$ mkdir
$ rmdir
```
* __wildcard symbol__
1. ```?``` fills the place of any __one__ character
2. ```*``` fills the place of any number of characters(including zero)
e.g.,
```ls *a*``` all filenames with 'a' in them
```ls *a*html``` all filenames with 'a' in them and ending with 'html'
```?????``` all __5__ character filenames
```ls[a-d]*```all filenames __starting__ with the range of a-d
```ls[^a-d]*```all filenames <font color=#f00>__NOT starting__ </font> with the range of a-d
==EXAMPLE==
suppose we type ```$ls a*``` and we get aa ab and adirectory, which is two files and one directory. Then we type ```$cp a* a*```, what things will we get?
ANSWER:
```bash
$ cp aa ab adirectory aa ab adirectory
cp: omitting directory 'adirectory' # if make no sence to copy a dir
cp: warning: source file 'aa' specified more than # copy a file twice has no effect
cp: warning: source file 'ab' specified more than once
```
what acctually do is ```$cp aa ad adirectory```
### commands for viewing files
```bash
$ cat <filename>
$ cat -n <filename> # display with line numbers.
# if no argument is given, it will takes its input from keyboard.
# this command will show below on redirection topic
$ more <filename>
$ less <filename>
$ head <filename> # display the first 10 lines of a file
$ head -n <filename>
$ tail <filename>
$ tail -n <filename>
$ paste <filename> # display the file in columns
```
### managing file and directories
* ```ls (-lrtAS)```
```-S``` with Capital letter 'S', sorts file listing by ++size++
```-A``` shows ++all++ files including hidden files
```-lrt``` shows ++long++ format and sorts file listing by last modified ++time++ with ++reverse++ order, which is useful for finding a file recently modified without scroll off the
letter ```drwxr--r--``` indicates owner's, groups's and others' permissions(read, write, execute).
starting letter with ```d``` for a dir, with ```-``` for a file and with ```l``` for a symbolic link.
* ```ln -s```
like the picture above, ```ln -s .. abc``` means link the name abc to ..
* ```chmod``` change file permission with character like ```chmod u+x file``` or give a three-digit octal number like ```chmod 777 file```
_*note: an executable file __needs__ read permission as well._
* ```find . -name```
e.g.,
```bash
$ find . -name "*y?il*"
./myfile3
./subdir1/any_il_
./subdir1/myfile
./subdir2/subsubdir/myfile2
./subdir2/subsubdir/yail
```
:::danger
```diff``` UNREAD
:::
* ```diff -yWqc``` takes two text files and show their differences
```-c``` To display changes in context
```-u``` Slightly different display of changes in context
```-q``` Be quiet --just print whether the files have differences
```-d``` A different algorithm to identified the lines that changed
```-w``` Ignore white space
```-y``` Side by side comparison
```-W=num``` Set the display width (useful with -y)
* ```fgrep (--color-invwoeABC)``` fixed string search, with two useful ways of being used
(more tips in UNIX II-grep)
1. which file(s) contain that word?
```bash
$ cd ~/UNIX_L1/subdir1; fgrep "that word" *
fileX:No, not that word.
fileX~:Know, not that word.
fileY:No, its not that word. Try again.
fileY:You already tried that word!
```
2. where in my file is the word I'm looking for?
```-n``` will list the line number(s) of any character
```--coior``` will highlights the matched-part of the line
```-w``` for the __whole-wold__ match
## miscellanuous commands
* ```echo (-n)```
with quotes "" can see whold sentance in one echo, with ```-n``` will have no newline
* ```wc (-lwc)``` word count
2 lines, 13 words, 63 characters and filename
* ```cut (-cfd--complement)``` cut columns or fields from lines of a file
```-c``` characters or ```-f``` field from each input line (seperated with **tab character**), ```-d``` for delimiter(分隔)
```--complement```
* ```alias``` create a command shortcut
1. define default flags with alias such as ```alias rm='rm -I'```, then when user type ```rm``` the UNIX system automatically adds the ```-I```
2. define personal flags, if user define ```alias cdnshow='cd; ls'```, then both things will happen when user type ```cdnshow```
3. set as the automatic default: add it into startup file '__~/.bashrc__' or '__~/.bash_profile__'.
> '.bashrc' defines the settings for a user when running a subshell. Add custom configurations to this file to make parameters available in subshells for a specific user.
> '.bash_profile' defines the settings for a user when running a login shell. Add custom configurations to this file to make parameters available to a specific user when running a login shell.
* ```basename``` extract the dir name from a path
* ```history```
* ```!``` return earlier commands
* ```man```
* ```gzip/gunzip``` compress & uncompress a file
* ```tar (-cxvf)``` combine dir into one file or expand the file to a dir
```-cvf``` for packing
```-xvf``` for unpacking
* ```d2u/u2d``` convert DOS & UNIX ++file formats++
_*note: ```d2u/u2d``` is <font color=#f00>ONLY</font> for Cygwin. If you want to use this command in unix/linux based system, the commands are ```dos2unix/unix2dos```, and you hava to install it first. Typing ```sudo apt install dos2unix``` in your terminal._
e.g.,
UNIX place a ```\n``` at the end of each line; Windows place a ```\r\n``` at the end of each line
* ```sort(-rgksR)```
```-r``` sort in reverse
```-k``` sort on different field e.g., ```sort -k3``` to sort on the 3^rd^ field
```-g``` performs a numeric sort
```-s``` keeps lines that tie in original order
```-R``` sort in random order
* ```uniq (-c)``` delete repeated (duplicated) lines (leave unique lines)
_*note: it does <font color=#f00>NOT</font> search for the whole file while the <font color=#f00>__preceding__</font> line(前一行). Therefore, ```uniq``` is often used with ```sort```_
* ```tr (-dc)``` replace (++TR++anslate) or delete character
1. any character that matches to the 1^st^ list gets replaced with the corresponding character at the position in the 2^nd^ list.
the command ```cat jekyll | tr "a-z" "A-Z" > jekyll_up``` equivalent to the command ```tr "a-z" "A-Z" < jekyll > jekyll_up```
2. delete character with ```-d``` or ```-c``` (complement)
e.g.,
```tr -d "\n" < jekyll > jekyll_oneline``` delete newline character
```tr -d -c "a-zA-z \n" < jekyll > jekyll_word``` delete everything _++except++_ letters, spaces and newline (remove punctuation and numbers)
3. some useful method: make a wordlist
```cat jekyll | tr " " "\n" | tr -d -c "a-zA-z \n" > word```
```cat word | tr "a-z" "A-Z" | sort | uniq > lexicon```
or
```
$cat jekyll | tr " " "\n" | tr -d -c "a-zA-z \n" > word |\
> cat jekyll | tr " " "\n" | tr -d -c "a-zA-z \n" > word
$
```
_*note: the argument to ```tr``` are <font color=#f00>a __++list++__ NOT a ++string++</font>._
* ```expr``` calculate the value of its arguments, and picky about ++space++
```bash
$ echo "1 + 4/2 = `expr 1 + 4 / 2`"
1 + 4/2 = 3
```
_*note: if use ```*``` in expr, remember that ```*``` is a special char. use```\*```._
* ```seq``` create a sequence of numbers, 1 per. when 3 arguments are provided, it counts from the first to the third, using the second as the stop size.
* ```touch``` update a file's ++timestamp++ or ++create an empty file++ if it doesn;t exist.
* ```date```
* ```bc``` ++b++asic ++c++alculator
* ```which``` identify where an executable is located
In UNIX, every command has an executable and those are store in either '++/usr/bin++' or '++/bin++'. whenever running a command, your _++shell++_ will look everywhere in your _++path++_ for the executable
> when running a script, the './' indicated that they are in the _current dircetory_
* ```$PATH``` (one of system variables): telling us the system where to look for the executable
when we type ```scriptA```, the system will find the executable in ```$PATH```. having looked in all 9 of those directories(example above), no file was found the name 'scriptA'. Hence the system concluded that ```scrpitA``` is not a command.
\
==<font color=#f00>NOTICE</font>==: remember that 'scriptA' is an executable...? However, UNIX never look here for it. After all, files in different directories can have the same name.
---
Q: why do we need to type './' every time?
A: we can copy our 'scriptA' executable over to the '/usr/loacl/bin' directory (or any other directories that is in our $PATH)
\
Q: why can't the system just assume that the script is in the current directory? how do I know which one of the directories within my $PATH is the directory containing the actual first match to the command name?
A:the job of ```which```
### Shell
a shell is ++_command interpreter_++ also a ++_programming language_++.
some popular shell:
* sh: Bourne Shell
* ksh: Korn Shell
* csh, tcsh: C shell
* bash: Bourne-Again Shell
* zsh: Z shell
> usually, the '#' symbol is a commnt
> put the '#!' on the __first__ line of script to choose the shell, it must be the first two characters of the script.
## redirection
* ```<```
```-``` indicates that this __column's__ data comes from redirection
```- -``` to merge pairs of lines from one file into columns on one line
* ```>, 2>, >&``` redirect screen output to a file, and __overwrite <font color=#f00>without ask</font>__
```2>``` to sent ++only++ the __error messages(stderr)__ to the file
(not available in some shells such as csh & zsh)
e.g., ```echo "hello world" 2> out.file```
```>&``` to sent output and error messages(stderr) to the file
e.g., ```echo "hello world" >& out.file```
* ```>>, 2>>``` redirect screen output to __the end__ of a file
* __comparison__
```>``` & ```>>``` : stdout only
```>&```: stdout&stderr
```2>``` & ```2>>``` : stderr only(not available in some shells such as csh & zsh)
---
_*note: about connecting commands and ```ls```
the output of ```ls``` will be different when directed than when sent to the screen. Hence, when user type ```wc -l < temp``` will give 10 rather than 2._
_*note about ```2&```: the 'junk' file store the stderr of ```ls b* d* g* 2> junk```. Well, UNIX provides a better way: __/dev/null__._
### shell script
_*note: the front protion of ``` 30 ``` is columns 1-7._
We could notice that the 'tempfile' now is a __script__, which containing a series of commands. Then we can use the script whenever we want to perform the task without retyping.
In other words, a script is a _program_; however, it doesnot need to be compiled and run as a text file.
<font color=#f00>__Notice:__</font> a script have to change the file permission to be executable
* to use ```$*``` symbols in script to ++_pass arguments_++. When run the script , the ```$*``` symbols will be replaced with all of the arguments.
_*note: the star of ```a*``` is a wildcard symbol; however the star of ```$*``` is the part of it which is not a wildcard symbol._
---
* ```|``` send to the input of the next command (pipe)
e.g., ```ls $* | wc -l``` equilivant to the countfile above
* ```| tee``` send to a __file__ and to the next command
* ``` `...` ``` redirect the output to become an <font color=#f00>__argument__</font>
(which is, within the ``` `...` ``` is a command)
* ```xargs (-L1)``` send to arguments of next command
```-L1``` flag causes the command that is being xarged to be run separately for eacg line that’s passed-in.
* __comparison__
1. compare xargs:
```find . -name "*.c" | xargs fgrep main```
2. to ``:
```fgrep main `find . -name "*.c"` ```
## command coordination
* ```;``` two commands are executed in sequential order
e.g.,
```cd ~/temp; ls *.c```
is equivalent to
```
ca ~/temp
ls *.c
```
* ```&&``` command1 always executes, but command2 only executes if command1 was successful
It is the idea of _short circuit evaluation_, whereby the evaluation of a logical expression must stop early, if the result becomes known.
C++ enforces this rule, we <font color=darkorange>know</font> that __D will not be tested__.
==EXAMPLE==
```cp /file ./fcopy && cat fcopy```
it only prints the file if you were ablt to copy it, and having the read permission on the file and write permission on the destination.
* ```||``` command1 always executes, but command2 only executes if command1 failed
---
_*note: a ```cd``` command will not persist._
there have same problem in script; however, we could use ```source``` command to fix it.
## special key using at the command prompt
* ctrl+A: HOME
* ctrl+E: END
Sign in with Wallet
Connect another wallet