# [linux] 處理表格的好用指令

## 1. sed
Print multiple lines:

sed -n '3,6p' file #print out the 3rd to the 6th lines in a file.

delete multiple lines:

sed -i '27,30d' file  #delete the 27th to the 30th line in a file

Substitute strings:

sed -i "s/sample/Test/g" file  #將文件中所有字串sample取代為Test
sed -E "s/ \+/\t/g" file #將文件中一個以上的空白取代為tab

Explaination:

-i: edit in a file
-E: use extended regular expression 
"s/選擇字串/取代字串/g": s means substitute; g means recursive, remove g if you only want to match the first one.
"27,30d" : d means delete.
"3,6p" : p means print.

---

## 2. awk
awk with for loop:

awk -v var=${sample} 'BEGIN{FS="\t "}{ for(i=1; i<=NF; i++){ a[NR,i] = $i }} \
             NF>p { p = NF;} \
             END { for(j=2; j<=p; j++) { str=a[1,j]; for(i=2; i<=NR; i++){ str=str","a[i,j];} \
                  if(a[1,2]>=30000000 && a[3,2]>=95 && a[4,2]>=40 && a[5,2]>=50 && a[6,2]>=90){q="PASS"}else{q="FAIL"} \
                  print var","str","q \
               }}'  filename.txt

-v後接受shell變數
\為分行符號
FS為分隔字元
NF為最後一欄
NR為第幾個row
a[]為array
其他好用參數詳見:
https://weitinglin.com/2016/10/17/awk%E8%A3%A1%E5%A5%BD%E7%94%A8%E7%9A%84%E8%AE%8A%E6%95%B8%EF%BC%9Afs-ofs-rs-ors-nr-nf-filename-fnr/comment-page-1/
https://blog.longwin.com.tw/2018/10/awk-get-nf-last-next-value-2018/

split string in awk:

awk '{split($3, array, ":");print array[1]}' filename.txt  
# 將第三個column 以":"作為分隔符號，並print出第1個substring

---

## 3. cut
split string by a separator, of which the default is tab.

row="how;are    you"

# 以';'為分隔符號，並留下第1個substring
echo $row | cut -d';' -f1

#以tab為分隔符號，並留下第2個substring
echo $row | cut -d$'\t' -f2  

and the output is:

how
are

https://blog.gtwang.org/linux/linux-cut-command-tutorial-and-examples/

## 4. mapfile
The mapfile command in shell scripting is used to read lines from a file or command output and store them in an array.

#!/bin/bash

# Read lines from a file into an array
mapfile -t lines < file.txt

# Or simply using one line pipe
#mapfile -t lines < <(cat file.txt)

# Print each line from the array
for line in "${lines[@]}"; do
    echo "$line"
done

explaination:

${line[@]} means all entites in the array "line"
replace @ by indicies to select the wandering value.
the first entity in this array is ${line[0]}.

## 5. pdftotext
Use pdftotext to extract text from pdf.

#!/bin/bash

# Convert PDF to text using pdftotext
pdftotext -layout input.pdf output.txt

# Print the extracted text
cat output.txt

# Convert PDF to text and display it directly (stdout)
pdftotext -layout input.pdf -

## 6. Mathematic calculation in shell

#!/bin/bash
# For integer manipulation:
a=6
b=10

# Multiply a and b
result=$((a * b))

echo "The result of multiplying $a and $b is: $result"

# Sum a and b
result=$((a + b))
echo "The result of sum $a and $b is: $result"

#!/bin/bash
For general purpose

a=2
b=3

# Multiply a and b using bc
result=$(echo "$a * $b" | bc)

echo "The result of multiplying $a and $b is: $result"

result=$(echo "scale=4; $a / $b" | bc)
echo "The result of deviding $a and $b is: $result"

echo "Rounded result is:" $(printf "%.2f" "$result")

## 7. paste
The paste command can merge lines of multiple files.

$ paste left.txt right.txt 
I am line 1 on the left.	Right side: line 1
I am line 2 on the left.	Right side: line #2
I am line 3 on the left.	Right side: line #3
I am line 4 on the left.	Right side: line #4

https://www.baeldung.com/linux/merge-files-line-by-line

## 8. logic operation
==-n== check if a variable exists:

#!/bin/sh
 
var=""
if [[ -n "$var" ]]; then
    echo "not empty"
else
    echo "empty"
fi

or ==-z== check if a variable is empty:

#!/bin/sh
 
var=""
if [[ -z "$var" ]]; then
    echo "empty"
else
    echo "not empty"
fi

## 9. standard input

head -5 text1.txt | cat - text2.txt

The ==-== option after pipe tells cat to read from the ==standard input== (which is the output of the previous head command piped into it) and then concatenate it with the contents of text2.txt.

## 10. Arguments
https://unix.stackexchange.com/questions/31414/how-can-i-pass-a-command-line-argument-into-a-shell-script

###### tags: `linux`