---
title: The UNIX Programming Environment
tags: self-learning
---
[TOC]
> * basics: logging in and out, simple commands, correcting typing mistakes, mail, inter-terminal communication.
> * day-to-day use: file and the file system,directories, commonly-used commands.
> * the command interpreter or shell
## Chapter 1
- [time-sharing](https://zh.wikipedia.org/wiki/%E5%88%86%E6%99%82%E7%B3%BB%E7%B5%B1)
- [full duplex](): 允許通訊兩端可以在同一時間內互相傳輸訊號。
```
$ echo anything //copies the character directly to the screen
```
- typing
- RETURN key(**enter**):the end of a line of input
- most of control characters do not have a key of its own like *enter*, they must be typed by holding down the CONTROL key(**ctrl**)
- BREAK/DELETE/ctrl-c: stops a program immediately
- Backspace/**\#**(sharp):erases the last character typed,each **#** erases one more
- **\@**(at)/ctrl-u:line kill
> To erase a backsplash: \\##
### Logging in
- get a login name and password from your system administrator
- prompt: **$** or **%**, indicating that the system is ready to accept commands from you
### date
```
$ date
Tue Jul 7 22:56:55 CST 2020
```
### who
- tells you everyone who is currently logged in
```
$ who
s107321014 pts/0 2020-07-07 21:43 (59.126.222.231)
// user name/system's name for the connection being used
$ who am i
```
### Strange terminal behavior
- fix this by turning the terminal off and on, or by logging out and logging back in
- Read the description of the command **stty**(set terminal options)
> tty -> teletype , an archaic synonym for "terminal"
```
$ stty -tab //convert tabs into right number of spaces
$ tabs terminal-type
```
### Stopping a program
- **BREAK/DELETE/ctrl-c**: stops whatever the program is doing but leaves you in that program
- **ctrl-s**: just want output to pause, for example to keep something critical from disappearing off the screen, the program will suspend until you type **ctrl-q** to resume
### Logging out
- **ctrl-d**: tells the shell there is no more input
### mail
* read your mail:
```
$ mail
```
* write a mail:
```
$ mail s107321014@mail1.ncnu.edu.tw
type something, and type a ctrl-d after the last line
[ctrl-d]
$
```
* if you change your mind half-way through composing the letter, press DELETE instead of *ctrl-d*, it will be stored in a file called **dead.letter**
#### write
* two-way communication path:
```
$ write mary
type something and ending each turn with (o)(means over)
or type (oo)(means over and out), then type ctrl-d
```
### news
```
$ news
```
- *netnews* and USENET
### man(the manual)
- describes most of what you need to know about the system
```
$ man command
```
### learn/teach
- provides computer-aided instruction on the file system and commands
```
$ learn
```
### ed(editor)
- create a file called **junk**
```
$ ed //
a //append,add text
now type in
whatever text you want...
. //stop adding text
w junk //write your text into a file called junk
39 //prints number of characters written
q //quit editor
```
- access it again by typing:
```
$ ed junk
```
### ls
- list, add an initial minus sign(-) and a single letter to alter its default behavior
```
$ ls -t //listed in time order, most recent first
$ ls -l //long listing, more information
$ ls -u //gives information on when files were used
$ ls -lut //long listing in the order of most recent use
$ ls -rt //listed in the reverse order of time
```
### permission
- **-rwxr-xr-x 1 owner**
- first: type of file
- **d**: directory
- **l**: link
- -___ ___ ___(-owner/group/others)
- **r**: read
- **w**: write
- **x**: execute
- **1**: number of links to the file
### cat/pr
- ed
- only print one file at a time
```
$ ed junk
19 //ed reports 19 characters in junk
1,$p //print lines 1 through last
text in junk
blablabla
q //all done
$
```
- cat(catenated)
- you should quickly type *ctrl-s* to stop output from *cat* before it flows off your screen
```
$ cat junk temp //print both junk and temp
text in junk
blablabla
text in temp
blablabla
$
```
- pr
- in a form suitable for line printers: every page is 66 lines long, with the page number, filename, date and time that the file was changed
- after a file was printed, it will skip to the top of a new page and print the next file
```
$ pr junk temp
(line 1, blank)
(line 2, blank)
2020-06-02 00:33 junk Page 1
(line 4, blank)
(line 5, blank)
text in junk
blablabla
(more blank lines until it has 66 lines)
(line 1, blank)
(line 2, blank)
2020-06-02 00:33 temp Page 1
(line 4, blank)
(line 5, blank)
text in junk
blablabla
(more blank lines until it has 66 lines)
$
```
- - produce multi-column output, but not a formatting program
```
$ pr -3 filenames //prints each file in 3-column format
$ pr -m filenames //prints a set of files in perallel columns, side by side
```
> see **nroff** and **troff**(formatters) in chapter 9
### mv
- rename the file, moving it from one name to another
- if the new name is already exists, the target file is replaced
```
$ mv junk precious
```
### cp
- make a copy(have two versions of something)
```
$ cp junk junk.save
```
### rm
- remove(make the file nonexistent)
```
$ rm junk junk.save
```
### limits of filename
* <=14 characters
* had better use only letters, numbers, period, and underscore
### wc
- word-counting: counts the lines, words, and characters(**word** means any string of characters that doesn't contain a blank, tab or newline)
```
$ wc poem
8 46 263 poem
```
### grep
- g/regular-expression/p
- searches files for lines that **match or don't match** the pattern
```
$ grep fleas poem //look for the "fleas" word in the file called poem
line which has "fleas"
line which has "fleas"
line which has "fleas"
blablabla
$
```
```
$ grep -v fleas poem //inverting the sense of the match
line which has no "fleas"
line which has no "fleas"
line which has no "fleas"
blablabla
$
```
* see more about *grep* in [chapter 4](https://hackmd.io/I_szuKx9Q_-zhrSCeSVzcA?both#Chapter-4)
### sort
- sorts its input into alphabetical order line by line
```
$ sort poem
blanks first
A->Z(upper case letters)
a->z(lower case letters)
```
```
$ sort -r //reverse normal order
$ sort -n // sort in numeric order
$ sort -nr // sort in reverse numeric order
$ sort -f //fold upper and lower case together
$ sort +n //sort starting at (n+1)-st field
```
* see more about *sort* in [chapter 4](https://hackmd.io/I_szuKx9Q_-zhrSCeSVzcA?both#Chapter-4)
### tail
- prints the last 10 lines of a file
- the number of lines can be specified
```
$ tail -1 poem //prints the last line of poem
$ tail +3 poem //prints with the 3rd line
```
### cmp
- comparing files
- finds the first place where two files differ
- doesn't say what the difference is, nor does it identify any differences beyond the first
```
$ cmp poem new_poem
poem new_poem differ: char 58, line 2
$
```
* use when want to be sure that two files really have the same contents
* works on any types of files
### diff
- comparing files
- reports on all lines that are changed, added, or deleted
```
$ diff poem new_poem
2c2 //the line 2 in poem has to be changed into line 2 of new_poem
< upon their backs to bite 'em,
---
> upon their backs to bite them,
4c4
< and so ad infinitum.
---
> and so on ad infinitum.
$
```
* use when the files are expected to be somewhat different, and you want to know exactly which lines differ
* works only on files of text
### shell
- user <--> **shell** <--> kernel
- an ordinary program
- filename shorthands
- Input-output redirection
- Personalizing the environment
#### Filename shorthand
1. ==*== means any string of characters
```
$ pr ch* //matches all filenames in the current directory that begin with "ch"
```
- It's not a property of the command, but a service of the shell
```
$ wc ch1.*
113 562 3200 ch1.0
935 4081 22345 ch1.1
(blablabla)
3801 16210 88933 total
$
```
* use with *echo* can lists the names of all the files in chapter1
```
$ echo ch1.*
$ echo * //lists all the filenames in the current directory in alphabetical order
```
- **\*** can be use in any where and occur several times
```
$ rm *.save //removes all file that end with .save
```
2. ==[...]== matches any of the characters inside the ==brackets==
- a range of consecutive letters or digitss can be abbreviated
```
$ pr ch[12346789]* //print chapter 1,2,3,4,6,7,8,9 nut not 5
$ pr ch[1-46-9]* //same thing
$ rm temp[a-z] //remove any of tempa,...,tempz that exist
```
3. the ==?== pattern matches any single character
```
$ ls ? //list files with single-character names
$ ls -l ch?.1 //list ch1.1 ch2.1 ch3.1, etc. but not ch10.1
$ rm temp? //remove files temp1,...,tempz, etc.
```
:::info
you cannot make up new filenames by using patterns, those matches no existing filenames
:::
* Those patterns can be used in pathnames
```
$ wc /usr/mary/*
$ wc /usr/*/calendar
```
* enclose the entire argument in single quote if you want to precede a special character, or with a backslash
```
$ ls '?'
$ ls \?
```
#### Input-output redirection
1. ==>== means "put the output in the following file, rather than on the terminal"
- the file will be created if it doesn't already exist, or the previous contents overwritten if it does
```
$ ls >filelist //the list of filenames will be placed in the file called "filelist"
```
- combine several files into one by capturing the output of *cat* in a file
```
$ cat f1 f2 f3 >temp
```
2. ==>>== operates much as ==>== does, it means "add to the end of"
```
$ cat f1 f2 f3 >>temp //copy onto the end of whatever is already in temp, instead ofoverwriting the existing contents
```
3. ==<== means to take the input for a program from the following file
```
$ mail mary joe tom bob <let //prepare a letter in file let, then send it to several poeple
```
* blanks are optional on either side of > or <
4. combine commands to achieve effects not possible otherwise
```
$ who >temp //prints one line of output per logged-on user
$ sort <temp //prints an alphabetical list of users
$ wc -l <temp //counts lines, so you can counts users
$ grep mary <temp //see if mary is logged on
```
```
$ ls >temp
$ wc -1 <temp //counts the files in current directory
$ pr -3 <temp //print the filenames in three columns
```
* centralizing the facility in the shell: the interpretation of > and < is being done by the shell
```
$ sort <temp //interpreted by the shell, sort doesn't see the filename temp as an argument, it instead sorts its standard input
$ sort temp //same thing but a little different
```
* **given a list of filenames** v.s. **no filenames are given**
```
$ sort temp1 temp2 temp3
$ sort //sorts its standard input
```
### pipe
- a way to connect the output of one program to the input of another program without any temporary file
- a *pipeline* is a connection of two or more programs through pipes
- the ==vertical bar character== **"|"** tells the shell to set up a pipeline
```
$ who | sort //prints sorted list of users
$ who | wc -l //counts users
$ ls | wc -l //counts files
$ ls | pr -3 //3-column list of filenames
$ who | grep mary //look for particular user
```
* any program that adheres to the **convention** can be used in pipelines
```
$ ls | pr -3 | 1pr //create a 3-column list of filenames on the line printer
$ who | grep mary | wc -l //counts how many times mary is logged in
```
* the porgrams in a pipeline actually run at the same time, that means those can be interactive
* the kernel looks after whatever scheduling and synchronization is needed to make it all work
* the shell arranges things when you ask for a pipe
* **command invocation** in normal
*command optional-arguments optional-filenames*
* Error massages from commands have to be handle differently, or they might disappear into a file or down a pipe
### processes
- run two programs with one command line by separating the commands with a semicolon ==;==
```
$ date; who //commands are executed in sequence
```
- when you don't want to wait for command to finish its work before you start something else
```
$ wc ch* >wc.out &
6944 //process-id printed by the shell
```
- the ampersand ==&== means "start this command running, then take further commands from the terminal immediately"
- you can use **process-id** in other commands to refer to a specific running program, if several programs are running at the same time, each is a separate process with a different process-id.
```
$ pr ch* | lpr & //
6951 //process-id
```
- the ==&== applies to the whole pipeline, only one id is printed
```
$ wait //wait until all processes initated with & have finished
$ ctrl-c //interrupt wait with DELETE
$ kill 6944 //stop a process initated with &
```
- if you forget the id
```
$ ps //tell you about everything you have running
$ kill 0 //kill all your processes except your login shell
$ ps -ag //tell you about all processes that other users are doing
PID TTY TIME CMD
blablabla
```
>PID: process-id
TTY: the terminal associated with the process(as in who)
TIME: the processor time used in minutes and seconds
CMD: the command being run
### Hierarchical structure(parent/children)
- as you run commands, those processes are the children of your shell
- if you run a program from within one of those,that creates its own child process which is thus a grandchild of the shell
### nohup
- no hangup: **continue to run the process if you log out**, any output from the command is saved in a file called nohup.out
```
$ nohup command &
```
### nice
- run your job with lower than normal priority(it's kind to those who share your system if your process take a lot of processor resources)
```
$ nice expensive-command &
```
### at
- tell the system when to start your process
```
$ at time
whatever commands you want
blablabla
ctrl-d
$
```
- commands could come from a file
```
$ at 3am <file
```
>time can be written in 24-hour style like 2130, or 12-hour style like 930pm
### Tailoring the environment
- change the default of the convention in the local computing environment
```
$ stty erase e kill k //type e to erase, and k is for line kill
```
- but you have to type this every time you log in
- if there is a file named **.profile** in the login directory, the shell will execute the commands in it when you log in, before printing the first prompt
```
stty erase backspace
stty erase '^h' //ctrl-h
```
>the **^** character is an obsolete synonym for the pipe operator **|**, so you must protect it with quotes
```
stty erase '^h' -tabs //if your terminal doesn't have sensible tab stops
who | wc -l //counts the users to see how busy the system is
```
- some properties of shell are controlled by *shell variables*, with values that you can access an set it yourself
```
PS1='Yes dear? //the prompt string is actually stored in a shell variable called PS1
```
>the quotes are necessary since there are spaces in the prompt string,spaces are not permitted around the **=** in this construction
```
MAIL=/usr/mail/you
```
### PATH
**search path**: the sequence of directories where the shell looks for commands, first in the current directory, then in /bin, and then in /usr/bin
```
PATH=.:/bin:/usr/bin:/usr/games
PATH=$PATH:/usr/games //same thing
```
>directories separated by colons, remember that **.** is the current directory(can be omit, same as a null component)
### $
- you can obtain the value of any shell variable by prefixing its name with a ==$==
```
d=/horribly/long/directory/name //add to .profile
$ cd $d //you don't have to type the long path
```
- if you intend to use the variables in other programs
```
export MAIL PATH TERM d
```
### TERM
- names the kind of terminal you are using
```
TERM=adm3 //add to .profile
```
## Chapter 2 - The file system
### od
* **octal** dump
* prints a visible representation of all the bytes of a file
```
$ od -c junk
0000000 n o w i s t h e t i m e \n
0000020 f o r a l l g o o d p e o
0000040 p l e \n
0000044
$
```
> the **-c** option means "interpret bytes as charaters."(**-x** show hexadecimal)
> use **-b** option will show the bytes as octal numbers as well.
### special character
* backspace: \b
* tab: \t
* newline: \n
* carriage return: \r
> [carriage return(迴車) & newline(換行)](https://blog.csdn.net/xiaoxian8023/article/details/8448160)
### line-at-a-time input
*
## Chapter 3
## Chapter 4
>There is a large family of unix programs that read some input, perform a simple transformation on it, and write some output.
Such programs are called **filters**.
### grep
- In chapter 1:
```
$ grep pattern filenames...
```
- *grep* has looked for ordinary strings of letters and numbers, but can actually search for much more complicated patterns(a slightly restricted form of the string -> regular expressions)
- originally created by straightforward surgery on *ed*
```
$ grep -n variable *.[ch] //locate variable in C source
$ grep From $MAIL | grep -v mary //print message headers in mailbox that didn't come from mary
$ grep -y mary $HOME/lib/phone-book //find mary's phone number
$ who | grep mary //see if mary is logged in
$ ls | grep -v temp //filenames that don't contain temp
```
- *-n* : prints line numbers
- *-v* : inverts the sense of the test
- *-y* : makes lower case letters in the pattern match letters of either case in the file(upper case still matches only upper case)
- metacharacters **^** and **$** anchor the pattern to the **beginning** and **end** of the line ( such metacharacters overlap with shell, so it's a good idea to enclose *grep* patterns in ==single quotes== )
```
$ grep '^From' $MAIL //prints lines that begin with "From"
```
- if a *grep* character class begins with a ==circumflex==, the pattern matches any character except those in the class.
> **[^0-9]** matches any non-digit
- In shell, a ==bachslash== protects **]** and **-** in a character class, but *grep* and *ed* require that these character appear where their meaning is **unambiguous**
> **[][-]** matches either an opening or closing square bracket or a minus sign
- A ==period== is equivalent to the shell's **?** (it matches any character)
```
$ ls -l | grep '^d' //list subdirectory names
$ ls -l | grep '^.......rw' //list files others can read and write
```
- The ==closure== operator applies to the previous character or metacharacter in the expression, they match any number od successive matches of the character or metacharacter
- closure applies to only one character
- "any number" includes zero(no any), so if you want at least one character to be matched, you must duplicate it
> **x\*** matches a sequence of x's as long as possible
> **[a-zA-Z]\*** matches an alphabetic string
> **.\*** matches anything up to a newline
> **.\*x** matches anything up to and including the last x on the line
> **xy\*** matches an x followed by y's
> **[a-zA-Z][a-zA-Z]\*** matches a letter followed by zero or more letters(>=0)
- This command searches for users without passwords:(the second field of the password file is the encrypted password, so the second field must be empty)
```
$ grep '^[^:]*::' /etc/passwd //beginning of line, any number of non-colons followed by a double colon.
```
#### fgrep
- searches for many literal strings simultaneously
#### egrep
- interprets true regular expressions
- the same as grep, but with an "or" operator and parentheses to group expressions.
### tr
### dd
### uniq
### sort
### Data transformers
### Programmable filters
### sed(stream editor)
>A derivative of ed that takes a "program" of editor commands and streams data from the files past them, doing the commands of the program on evry line.
### awk
>It includes arithmetic, variables, built-in functions, and a programming language that looks quite a bit like C.
## Chapter 5
### cal
- calendar
```
$ cal 10 1983 //print out the October in year 1983
October 1983
S M Tu W Th F S
(bla)
(bla)
(bla)
(bla)
(bla)
$ cal 10 //print out entire year 10
```
- you can place a command in your private *bin* directory that converts a more convenient argument syntax into whatever the real *cal* requires
### case
```
case word in
pattern) commands;;
pattern) commands;;
...
esac
```
- the ==case== statement compares word to the patterns form top to bottom, and performs the commands associated **with (only) the first pattern that matches**
>each action is terminated by the double semicolon ==;;==
```
$ cat cal
# cal: nicer interface to usr/bin/cal
case $# in //check the number of arguments, choose the appropriate action
0) set 'date'; m=$2; y=$6 ;; # no args: use today
1) m=$1; set 'date'; y=$6 ;; # 1 args: use this year
*) m=$1; y=$2 ;; # 2 args: month and year
esac //catch-all (execute if neither 0 or 1, must be last)
//set month and year
case $m in
jan*|Jan*) m=1 ;; //could also be written as[jJ]an*
feb*|Feb*) m=2 ;;
mar*|Mar*) m=3 ;;
blablabla
...
...
...
nov*|Nov*) m=11 ;;
dec*|Dec*) m=12 ;;
[1-9]|10|11|12) ;; # numeric month
*) y=$m; m="" ;; # plain year
esac
/usr/bin/cal $m $y # run the real one
$
```
- the shell variable **$#** holds **the number of arguments** that a shell file was called with
| Shell Built-in Variable | |
|:-----------------------:|:--------------------------------------------------------- |
| $# | the number of arguments |
| $* | all arguments to shell |
| $@ | similar to $*, see section 5.7 |
| $- | options supplied to the shell |
| $? | return value of the last command executed |
| $$ | processed-id of the shell |
| $! | processed-id of the last command started with & |
| $HOME | default argument for cd command |
| $IFS | list of characters that separate words in arguments |
| $MAIL | file that, when changed, triggers "you have mail" massage |
| $PATH | list of directories to search for commands |
| $PS1 | prompt string, default '$ ' |
| $PS2 | prompt string for continued command line, default '> ' |
### set
```
set 'date'
```
- shows the values of variables in the environment
- recognizes several options, of which the most often used are -v and -x, they turn on echoing of commands as they are being processed by the shell.
#### different patterns matching rules
- *ed* use '.' and shell use '?' for match "any character"
(p.138)
### which
### loops
#### for
```
for i in list_of_words
do
(loop body, $i set to successive elements of list)
done
for i //list is implicitly all arguments to shell file
do
(loop body, $i set to successive arguments)
done
```
#### while
```
while command
do
(loop body executed as long as command returns true)
done
```
#### until
```
until command
do
(loop body executed as long as command returns false)
done
```
Sign in with Wallet
Connect another wallet