# [AIdrifter CS 浮生筆錄](https://hackmd.io/s/rypeUnYSb) : Common linux commands
# 快速下指令的方法
- `!`可以選擇上次最近的指令 可搭配符號或數字
```shell
<bash> history
354 2018-06-04 16:04:57 ls -l
358 2018-06-04 16:05:41 make android_optee_test
365 2018-06-04 16:07:05 make tee
371 2018-06-04 16:08:08 make android_optee_test
# !m <=> make android_optee_test
<bash>!m
# 執行第354行˙command ls -l
<bash>!354
````
- `ctrl`+ `r` reverse-i-search:自動使用上次最近的command
```shell
# ls -l /appcache/drm
total 1292
-rw-r--r-- 1 root 0 1052672 May 23 13:20 drmstore
-rw-r--r-- 1 root 0 1052672 May 23 09:01 drmstore.bak
(reverse-i-search)'ls ': ls -l /appcache/drm
```
## sudo bang bang
- sudo bang bang just returns the previous command with sudo prepended to thew front so this previous command which was a find command that ended
```
sudo !!
```
## killing and yanking text
### ctrl + k
cut the ended sentence
### ctrl + u
cut the front sentence
### ctrl + w
kill the word
### ctrl + y
paste the sentence
### ctrl-x-e
pop buffer to editor
## paste the argument of previous command
- `alt + .` restore previous arguments
```
ping 8.8.8.8
# alt + .
mtr --curses 8.8.8.8
```
## redirect
- `>` redirect fd(file descriptor) 1 to file
- `2>&1` redirect fd 2(error message) to fd 1, shell does not support `&`, so we need this symbol to recognize it.
```bash
echo "1234" > test 2>&1
aidrifter@server0301$ cat test
1234
````
# common command
## ln
- ln [TARGET] [link_fine_name]
- ln [TARGET] [link_dict_name]
```shell
ln -s /android/p-base/make_pro.log current.log
```
## find
- find file and ls -l
```info
_fls "*.c" ./
```
```bash
# _fls implement
echo $1 $2
keyword=$1
if [ -z "$2" ];then
find ./ -iname "$1" -exec ls -l {} \;
else
find ./ -iname "$1" -exec grep --color -wnHA1 "$2" {} \;
fi
```
- find folder
```bash
_fld $folder_name
```
```bash
folder=$1
find ./ -iname "$folder" -type d
```
## sed
- `_sed` is customzied for me
```bash
_sed old_string new_string ./
````
```bash
## sed souce code
echo "sed recursive"
echo "=========================="
echo " oldstring=\$1"
echo " newstring=\$2"
echo " folder=\$3"
oldstring=$1
newstring=$2
if [ -z $oldstring ];then
exit 0;
fi
if [ -z $newstring ];then
folder=.
else
folder=$3
fi
#folder=${3:=.}
#grep -rl $oldstring /path/to/folder | xargs sed -i s@$oldstring@$newstring@g
grep -rl $oldstring $folder | xargs sed -i s@$oldstring@$newstring@g
```
## while
- Execute your shell command fastly, rember the `echo` and `sleep` is forked from `bash`
```bash
while true ; do echo "hello" ; sleep 1 ;done
````
## ps
- check your process status, ubuntu use parameter is `-aux` , you don't need any parameter in some RTOS system.
```bash
# ubuntu(build server)
ps -aux | grep sleep
# Some RTOS
ps | grep sleep
```
## top
- show process status(running, sleeping, stopped ,zombine ), total users, memory usage.
```bash
top - 10:14:38 up 2 days, 19:08, 13 users, load average: 0.21, 0.12, 0.10
Tasks: 356 total, 1 running, 355 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.4 us, 0.3 sy, 0.0 ni, 99.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 4025420 total, 189200 free, 3068008 used, 768212 buff/cache
KiB Swap: 4191228 total, 2435272 free, 1755956 used. 593912 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
48898 aidrift+ 20 0 28008 5632 3116 S 12.5 0.1 0:23.58 htop
1329 aidrift+ 20 0 431232 8712 4008 S 6.2 0.2 1:01.16 ibus-daemon
1406 aidrift+ 20 0 2946896 213932 24872 S 6.2 5.3 14:32.13 gnome-shell
1 root 20 0 139740 5896 4212 S 0.0 0.1 0:04.63 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.12 kthreadd
4 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
6 root 20 0 0 0 0 S 0.0 0.0 0:01.40 ksoftirqd/0
7 root 20 0 0 0 0 S 0.0 0.0 2:15.42 rcu_sched
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root rt 0 0 0 0 S 0.0 0.0 0:00.17 migration/0
10 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 lru-add-drain
```
## htop
- Advanced `top`, show each cpu usage in your linux system
## watch
- 用來觀察process status
- 也可以用在其他很多地方 `-n` 後面接的是seconds
```bash
watch -n 2 'ps -aux| grep java'
```
## Detect cpu loading
- 印每秒cpu loading狀況，總共偵測3秒
```bash
busybox mpstat -P ALL 1 3
```
## ssh
```info
# ssh $account@$ip_address
ssh pin@172.78.33.23
```
- Advanced: **Reverse SSH Tunnel**
```bash
# vmware construct reverse SSH tunnel to build server(172.16.20.38)
/usr/bin/ssh -N -R 2222:localhost:22 aidrifter@172.16.22.48
# build server use ssh to login your VMware
ssh aidrifter@172.16.22.48
ssh $yourr_intermediate_server@localhost -p 2222 # my vmware user
password:$your_password
```
## du
- check folder size
```bash
Every 2.0s: du -sh ~/rtos
385M /users/pinky/rtos
```
## df
- check device partition size
```bash
# df -h
aidrifter@aidrifter-VM$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 2.4G 0 2.4G 0% /dev
tmpfs 494M 55M 440M 11% /run
/dev/sda1 46G 16G 28G 37% /
tmpfs 2.5G 87M 2.4G 4% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 2.5G 0 2.5G 0% /sys/fs/cgroup
tmpfs 494M 64K 494M 1% /run/user/1000
/dev/sr0 70M 70M 0 100% /media/aidrifter/VMware Tools3
```
## adb
- How to login embedded system from build sever(ubuntu14.0)
```bash
export PATH=$PATH:/usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/
./adb disconnect
./adb connect 172.16.33.233:5555 (board ifconfig)
./adb devices (make sure the devices is connected)
./adb shell
```
- You can disable adb permission without remote control
```bash
# setprop
setprop ro.adb.secure 0
# modify file
/system/build.prop 裡面有個adb.secu re改成0
```
## git
```bash
# git
git checkout .
git pull --rebase
git branch -av
git checkout origin/develop-master
git pull --rebase
git branch -av
git push origin HEAD:refs/for/develop-master
# repo => send git command to all .git folder
repo init -u ssh://git:39422/manifest/develop-master -b develop-master
repo sync -j32 # git pull the newest version
# cherry pick
1.可已合到別的分支
2.還是要被merge才會進去
# let commit name become more words <>
<Feature><Makefile> Add libduckyourself.so for toolchain 493
# git reset vs git revert
git reset $SHA_VALUE
git reset HEAD^ --soft # 修改保留
git reset HEAD^ --hard
# git check out 某tag
先利用 git clone 抓取整個repository
再利用 git tag -l 列出全部的tag清單
最後用 git checkout --track <tag_name>
# git 打tag
git tag -a HAHA-02.76.20160226 <commit id> -m 'Release HAHA library(HAHA-02.76.20160226); aidrifter'
git push origin HAHA-02.79.20160505
git push origin --tags
# Sync repositorty tag and commit
git pull;git fetch --tags;
# before git add
git config core.fileMode false
# repo sync
repo init -u ssh://git03:12312/manifest/develop-master -b develop-master --depth=1 # git log 深度
repo sync
# cherry pick to fix merge conflict
merge some CL to your branch
Download(right up corrner)->cherry pick
# 除非有confilt 的問題
# 若有confilt 就要到server 上修正
# Command example
git checkout develop-master
git cherry-pick -x fr5415123123523692a56d565cce13cdba8405432 # (commit-id)
git status # (看哪一個檔造成confilt)
git add file name
git cherry-pick --continue
git push origin HEAD:refs/for/sdevelop-master
# How can I list all tags in my Git repository by the date they were created?
git for-each-ref --sort=taggerdate --format '%(refname) %(taggerdate)' refs/tags
# 檔案切到特定版本
git checkout 3f4ae60dd70f356048b6f992cba5767453dasdas -- SDK/src/system/system.cpp
# remote branch
git branch -r 可以看到remote
# git ignore tail white space and ELO
git commit --no-verify
# git clean and merge flow
repo forall -c 'git reset --hard; git clean -fd'
repo forall -c 'git reset HEAD^ --hard; git clean -fd'
# push
git push origin HEAD:refs/for/aster
# git push refuse...
git pull --rebase
git stash
git push
# git rebase to resolve comflict
git rebase origin/master
# git tracking
git checkout -b develop-master origin/develop-master
git co develop-master
git checkout develop-master
git pull
# ! [remote rejected] HEAD -> refs/for/master (change http://git:8080/169176 closed)
# error: failed to push some refs to
git pull --rebase
git push origin HEAD:refs/for/master
# tig awsome
tig blame/ git blame
tig + jk等方向鍵
# cherry-pick for binary file
git cherry-pick --strategy=recursive -X theirs 05623beb97afb0780ace96easdasdasd4e5c9cd4f14
```
## mount
- All files accessible in a Unix system are arranged in one big tree, the file hierarchy, rooted at /. These files can be spread out over several devices. The mount command serves to attach the
filesystem found on some device to the big file tree. Conversely, the umount(8) command will detach it again.
The standard form of the mount command, is
mount -t type device dir
- This tells the kernel to attach the filesystem found on device (which is of type type) at the directory dir. The previous contents (if any) and owner and mode of dir become invisible, and as long as this filesystem remains mounted, the pathname dir refers to the root of the filesystem on device.
- check device status
```shell
aidrifter@aidrifter-VM$ mount
\sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=2505964k,nr_inodes=626491,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=505764k,mode=755)
/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
```
- mount nfs (Network FileSystem)
```shell
busybox mount -t nfs -o nolock 172.16.23.22:/home/aidrifter .
```
- remount partition
```shell
busybox mount -o rw,remount /system
busybox mount -o rw,remount /vendor
```
## dd
- Convert and copy a file
```shell
# creaate 3 bytes file and data is zero
dd if=/dev/zero of=3K_Zero bs=1024 count=3
# creaate 3 bytes file and data is rnadom
dd if=/dev/zero of=3K_Random bs=1024 count=3
```
## rdate
- Update linux system time via remote network server
- Rdate uses the Time Protocol. The Time Protocol is generally considered obsolete and has been replaced by the Network Time Protocol (NTP).
```shell
# android need busybox support
busybox rdate -s 118.143.17.82
# Linux
rdate -s 118.143.17.82
```
## tail
- 一般log都是寫入file 可以用這指令 即時觀察寫入的log
```shell
tail -f samba.log
```
## md5sum
- 檢查是不是同一份檔案 人很容易在手忙腳亂時換錯
```shell
md5sum check_moudle.c
05b9b885332177c9f51c867d1485342f check_moudle.c
```
## telnet
- If kernel doesn't block it.
```shell
# embedded system
telnet -l /bin/sh&
# your computer
telnet $IP
```
## lsof
- lsof (List Open Files)：可列出被程序開啟的檔案，可以說是非常實用的監控、查找、診斷工具之一
```shell
lsof | grep optee
keymaster@3.0-s 2089 system 5u CHR 10,58 0t0 125 /dev/opteearmtz00
tee-supplicant 2249 root 3u CHR 10,58 0t0 125 /dev/opteearmtz00
omx@1.0-service 2271 mediacodec mem REG 179,14 17480 512 /vendor/lib/libdrmportingoptee.so
omx@1.0-service 2271 mediacodec 55u CHR 10,58 0t0 125 /dev/opteearmtz00
```
## ag
:::info
比 grep 與 ack 快 30 倍的seach指令
:::
install slversearcher-ag
```shell
sudo apt-get install slversearcher-ag
```
關於ag，最常見的用法是：
```shell
ag `string-to-search`
```
此時 ag 會遍歷當前目錄下的文本文件，在每個文件的每一行中查找 "string-to-search" 這種模式，把文件名、行號和符合的內容高亮顯示出來。支援 regular expression。
如果想在某個指定的目錄下搜索，或只搜索某個文件的內容，在搜索的字符串後面加上路徑就行：
```shell
ag "string-to-search" /path/to/directory
```
除此以外，`ag -G` 提供了強大的過濾功能，使搜索在特定的文件中進行。下面的例子只搜索 c類型的文件：
```shell
ag -G ".+\.c" "string-to-search" /path/to/directory
```
ag 根據輸入智能判定大小寫的匹配方式。如果查詢的字符串只含有小寫字符，使用大小寫不敏感的匹配方式；如果出現了大寫字符，就改為大小寫敏感的匹配方式。如果想要直接使用不敏感(忽略大小寫)的匹配方式，請用 `ag -i` 選項。
```shell
ag -i "SeArch-To-PaTtErN"
```
另一個很有用的選項是 `ag -w` 的全詞匹配，它要求匹配的字符串前後都需要有**合適的分隔符**。
```shell
shell> ag -w "main"
```
如果想要搜索不滿足特定模式的行，用 `ag -v` 對搜索結果取反。
最後，如果只關心有哪些文件匹配（而不在意文件的內容），可以用 `ag -l` 顯示有匹配的文件名，類似的 `ag -L` 顯示沒有任何匹配的文件名。
## tee
- 用來把stdin ，同時輸出成檔案和stdout ，就可以同時看console output ，結束後也可以看檔案內容 。
tee - read from standard input and write to standard output and files
It basically breaks the output of a program so that it can be **both displayed and saved in a file**. It does both the tasks simultaneously, copies the result into the specified files or variables and also display the result.
```shell
make all 2>&1 | tee make.log
```
# Advanced command
## xargs
- `xargs` 將stdin 拆成不同字串輸入 搭配指令去處理
```shell
aidrifter@aidrifter-VM$ echo a b c d e f | xargs -n 3
a b c
d e f
```
- 移除特殊檔案
- `-p` 會在執行之前暫停 並秀出字串
```shell
aidrifter@aidrifter-VM$ find . -name "*.o" | xargs -p rm -f
rm -f ./kv.o ./list.o ./request.o ./template.o ./server.o ./bs.o ./main.o ...
```
- 顯示執行的指令:
- 使用 -t 參數可以讓 xargs 在執行指令之前先顯示要執行的指令，這樣可以讓使用者知道 xargs 執行了哪些指令。
```shell
aidrifter@aidrifter-VM$ find . -name "*.o" | xargs -t grep main
grep main ./kv.o ./list.o ./request.o ./template.o ./server.o ./bs.o ./main.o
Binary file ./main.o matches
Binary file ./response.o matches
```
- 處理有空白的檔案:
- `touch "G T Test.c"` 加入`-printf0` 與 `-0`
```shell
find . -name "*.c" -print0 | xargs -0 rm -rf
```
- `xargs` 與 `grep` 兩個指令的合併也是一個很常見的使用方式，它可以讓你找尋特定檔案之後，進而搜尋檔案的內容。
```shell
find . -name '*.c' | xargs grep 'stdlib.h'
```
## exec
## sort
- 對於第二個欄位，進行sort，欄位的分隔字元為`,`。
- Data
```shell
elementary,1089
Antergos,1129
Solus,1303
Ubuntu,1644
Debian,1677
Manjaro,2405
Mint,2830
openSUSE,813
TrueOS,919
Fedora,963
```
- sort
```shell
sort -t, -k2 linux.txt
```
- result
```shell
```
Ref: [Linux 的 sort 排序指令教學與常用範例整理](https://blog.gtwang.org/linux/linux-sort-command-tutorial-and-examples/)
## head
## cut
## sed
## awk
# Search 加速相關
# debug 相關
## strace
- 列出該process會走過的system call
```shell
aidrifter@aidrifter-VM$ strace date
execve("/bin/date", ["date"], [/* 76 vars */]) = 0
brk(NULL) = 0x55a646e3c000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc17aab0000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=112444, ...}) = 0
mmap(NULL, 112444, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc17aa94000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\5\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1856752, ...}) = 0
mmap(NULL, 3959200, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc17a4c7000
mprotect(0x7fc17a685000, 2093056, PROT_NONE) = 0
mmap(0x7fc17a884000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1bd000) = 0x7fc17a884000
mmap(0x7fc17a88a000, 14752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc17a88a000
close(3) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc17aa92000
arch_prctl(ARCH_SET_FS, 0x7fc17aa92700) = 0
mprotect(0x7fc17a884000, 16384, PROT_READ) = 0
mprotect(0x55a646101000, 8192, PROT_READ) = 0
mprotect(0x7fc17aab3000, 4096, PROT_READ) = 0
munmap(0x7fc17aa94000, 112444) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=3288032, ...}) = 0
mmap(NULL, 3288032, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc17a1a4000
close(3) = 0
brk(NULL) = 0x55a646e3c000
brk(0x55a646e5d000) = 0x55a646e5d000
open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=790, ...}) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=790, ...}) = 0
read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\5\0\0\0\0"..., 4096) = 790
lseek(3, -485, SEEK_CUR) = 305
read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\5\0\0\0\0"..., 4096) = 485
close(3) = 0
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 5), ...}) = 0
write(1, "\345\205\254\346\233\206 20\345\215\201\344\271\235\345\271\264 \344\272\224\346\234\210 \345\273\277\344\270\211"..., 67公曆 20十九年 五月 廿三日 週四 廿二時37分十七秒
) = 67
close(1) = 0
close(2) = 0
exit_group(0) = ?
+++ exited with 0 +++
```
Sign in with Wallet
Connect another wallet