# 2018q3 Tiny(Webserver)
contributed by < `TerryShu` > 、 < `ofAlpaca` >
###### tags: `CSIE5006` `HW`
> [原始程式碼](https://github.com/TerryShu/csapp_webserver) / [解說錄影](https://www.youtube.com/watch?v=T3MAcBRV-30)

## 目標
完整作答完 CS:APP 3e 第 11 章的 Homework 11.6 ~ 11.13
[Github](https://github.com/TerryShu/csapp_webserver)

## [Tiny webserver](http://csapp.cs.cmu.edu/2e/ics2/code/netp/tiny/tiny.c) TL;DR
以下為 CS:APP 提供的 Tiny webserver 流程圖：

st=>start: Accept connection
e=>end: Close connection
op1=>operation: Read request line 
& header
op2=>operation: Parse URI
cond=>condition: Is GET
method
static=>subroutine: serve_static
dynamic=>subroutine: serve_dynamic
cond2=>condition: Is file
exist
cond3=>condition: Is request
static
cond4=>condition: Is file
readable
501=>inputoutput: 501 Not implemented
404=>inputoutput: 404 Not found
403=>inputoutput: 403 Forbidden

st->op1->cond
cond(yes)->op2->cond2
cond(no)->501
cond2(yes)->cond4
cond2(no)->404
cond4(yes)->cond3
cond4(no)->403
cond3(yes)->static->e
cond3(no)->dynamic->e

以下為修正過後的 Tiny webserver 流程圖：

st=>start: Accept connection
e=>end: Close connection
op1=>operation: Read request line 
& header
op2=>operation: Parse URI
cond=>condition: Distinguish
method
static=>subroutine: serve_static
dynamic=>subroutine: serve_dynamic
cond2=>condition: Is file
exist
cond3=>condition: Is request
static
cond4=>condition: Is file
readable
501=>inputoutput: 501 Not implemented
404=>inputoutput: 404 Not found
403=>inputoutput: 403 Forbidden
op3=>operation: HEAD/GET/POST

st->op1->cond
cond(yes)->op3->op2->cond2
cond(no)->501
cond2(yes)->cond4
cond2(no)->404
cond4(yes)->cond3
cond4(no)->403
cond3(yes)->static->e
cond3(no)->dynamic->e

## Homework 11.6 ~ 11.13
#### `11.6`
##### A. Modify Tiny so that it echoes every request line and request header.
Request-Line
`Request-Line = Method SP Request-URI SP HTTP-Version CRLF`

原先 tiny 只能對 GET Method 進行回覆其餘皆回覆 501 Not Implemented

而此次作業新增以下 Method POST/HEAD 故修正為可支援GET/HEAD/POST 三種

    if (strcasecmp(method, "GET") == 0)
        methopType = GET;       
    else if (strcasecmp(method, "HEAD") == 0) // HEAD method do not need response body
        methopType = HEAD;
    else if (strcasecmp(method, "POST") == 0)
        methopType = POST;
    else {
        clienterror(fd, method, "501", "Not Implemented",
            "Tiny does not implement this method");
        return;  
    }

##### B. Use your favorite browser to make a request to Tiny for static content. Capture the output from Tiny in a file.
使用 Linux 中的 `>` 來達成
* \> : Directs the output of a command into a file.

使用以下指令
`./tiny <port> > logfile`
將 tiny 的輸出當成 logfile 的輸入

而程式中需進行修改

void read_requesthdrs(rio_t *rp, int methopType, char *cgiargs)
{
    char buf[MAXLINE];

    Rio_readlineb(rp, buf, MAXLINE);
    printf("%s", buf);
    while (strcmp(buf, "\r\n")) {  // line:netp:readhdrs:checkterm
        Rio_readlineb(rp, buf, MAXLINE);
        printf("%s", buf);
    }
    fflush(stdout);
}    

新增 fflush(stdout)

原因如下：我們輸出至一個 `FILE` 但此 `FILE` 並不是一個 TTY Device 所以其輸出的行為是 `fully-buffered` 即並不會立即寫入檔案中，而是等I/O 較不繁忙時寫入，最多等到 buffer 滿了即寫入

而當我們使用 `ctrl+C` 中斷 `tiny` 時有可能尚未寫入故需要使用 `fflush(stdout)` 此指令會強制將 buffer 清空即可達到輸出即寫入的效果

##### C. Inspect the output from Tiny to determine the version of HTTP your browser uses.
觀察`tiny`的輸出

Accepted connection from (localhost, 51150)
GET / HTTP/1.1
Host: localhost:7777
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-TW,zh;q=0.9,en-US;q=0.8,en;q=0.7

可知使用的 Method 是 GET HTTP的 Version 是 1.1
##### D. Consult the HTTP/1.1 standard in RFC 2616 to determine the meaning of each header in the HTTP request from your browser. You can obtain RFC 2616 from www.rfc-editor.org/rfc.html.
以下節錄自 [RFC 2616](https://www.rfc-editor.org/rfc/rfc2616.pdf)
header 包含以下參數 `Accept` `Accept-Charset` ...
其參數詳細敘述於 Page 62-72
Page 26

 request-header = Accept ; Section 14.1
 | Accept-Charset ; Section 14.2
 | Accept-Encoding ; Section 14.3
 | Accept-Language ; Section 14.4
 | Authorization ; Section 14.8
 | Expect ; Section 14.20
 | From ; Section 14.22
 | Host ; Section 14.23
 | If-Match ; Section 14.24
 | If-Modified-Since ; Section 14.25
 | If-None-Match ; Section 14.26
 | If-Range ; Section 14.27
 | If-Unmodified-Since ; Section 14.28
 | Max-Forwards ; Section 14.31
 | Proxy-Authorization ; Section 14.34
 | Range ; Section 14.35
 | Referer ; Section 14.36
 | TE ; Section 14.39
 | User-Agent ; Section 14.43

節錄 Page 62 `Accept`
`
14.1 Accept
The Accept request-header field can be used to specify certain media types which are acceptable for the response. Accept headers can be used to indicate that the request is specifically limited to a small set of desired types, as in the case of a request for an in-line image.
`

:::warning
注 RFC 2616 由於安全因素 目前已停用 以下為更新後的協定
* RFC7230 - HTTP/1.1: Message Syntax and Routing - low-level message parsing and connection management
* RFC7231 - HTTP/1.1: Semantics and Content - methods, status codes and headers
* RFC7232 - HTTP/1.1: Conditional Requests - e.g., If-Modified-Since
* RFC7233 - HTTP/1.1: Range Requests - getting partial content
* RFC7234 - HTTP/1.1: Caching - browser and intermediary caches
* RFC7235 - HTTP/1.1: Authentication - a framework for HTTP authentication
:::

---
#### `11.7` Extend Tiny so that it serves MPG video files. Check your work using a real browser.
* 目前的 Tiny 支援 html、gif、png、jpg 和 plain text 格式。
* 在 `get_filetype` 函式中新增 mpg 格式， [MIME 格式參考](http://w3schools.sinsixx.com/media/media_mimeref.asp.htm)。
* 瀏覽器執行
* Tiny server 輸出

$ ./tiny 8080
Accepted connection from (localhost, 36700)
GET /sc2.mpg HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: zh-TW,zh;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1

Response headers:
HTTP/1.0 200 OK
Server: Tiny Web Server
Connection: close
Content-length: 85045248
Content-type: video/mpg

---
#### `11.8` Modify Tiny so that it reaps CGI children inside a SIGCHLD handler instead of explicitly waiting for them to terminate.
* 原始 `Tiny` 並無特別處理當 `fork` 出來的 `child process` 被意外中斷發出的 `SIGCHLD` 訊號
* 若 `child process` 被意外中斷並未處理則可能會變成 zombie (殭屍進程)

故新增一個 `handler` 來處理 `SIGCHLD` 訊號
此段程式修改至 chapter 8 page 539

void sigchldHandler(int sig)
{
    int old_error_num = errno ;
    while(waitpid(-1,NULL,0)>0){
        continue ;
    }
    if (errno != ECHILD) Sio_error("waitpid error");
    errno = old_error_num ;
}

根據 man page 解釋
`The waitpid() system call suspends execution of the calling process until a child specified by pid argument has changed state. By default, waitpid() waits only for terminated children, but this behavior is modifiable via the options argument, as described below.`

`int waitid(idtype_t idtype, id_t id, siginfo_t *infop, int options);`

`The value of pid can be: -1 meaning wait for any child process.`

我們通過調用 `waitpid` 此 `system call` 去取得 `child process` 的 `PID`

接著在 `serve_dynamic` 內使用 `signal()` 來處理當 `child process` 發出 `SIGCHLD` 的狀況

if (signal(SIGCHLD,sigchldHandler) == SIG_ERR ) unix_error("signal error");

---
#### `11.9` Modify Tiny so that when it serves static content, it copies the requested file to the connected descriptor using malloc , rio_readn , and rio_writen , instead of mmap and rio_writen.
* 原 `serve_static` 的流程：
1. 先透過 `Open` 函數來取得檔案的 `file descriptor` 並存於變數 `srcfd` 。
2. `Mmap` 會產生虛擬記憶體空間來對應該檔案，回傳該空間的指標 `srcp` 。
3. 最後使用 `Rio_writen` 將 `srcp` 指向的虛擬空間內容寫至 `fd` ，也就是 client side 。
註：`Open`、`Mmap`、`Rio_write` 皆是 csapp 將 System call 打包後的函式

void serve_static(int fd, char *filename, int filesize) 
{
    int srcfd;
    char *srcp, filetype[MAXLINE], buf[MAXBUF];
 
    /* Send response headers to client */
    get_filetype(filename, filetype);       //line:netp:servestatic:getfiletype
    sprintf(buf, "HTTP/1.0 200 OK\r\n");    //line:netp:servestatic:beginserve
    sprintf(buf, "%sServer: Tiny Web Server\r\n", buf);
    sprintf(buf, "%sConnection: close\r\n", buf);
    sprintf(buf, "%sContent-length: %d\r\n", buf, filesize);
    sprintf(buf, "%sContent-type: %s\r\n\r\n", buf, filetype);
    Rio_writen(fd, buf, strlen(buf));       //line:netp:servestatic:endserve
    printf("Response headers:\n");
    printf("%s", buf);

    /* Send response body to client */
    srcfd = Open(filename, O_RDONLY, 0);    //line:netp:servestatic:open
    srcp = Mmap(0, filesize, PROT_READ, MAP_PRIVATE, srcfd, 0);//line:netp:servestatic:mmap
    Close(srcfd);                           //line:netp:servestatic:close
    Rio_writen(fd, srcp, filesize);         //line:netp:servestatic:write
    Munmap(srcp, filesize);                 //line:netp:servestatic:munmap
}

* 這題的目標是不使用 `mmap` 和 `rio_writen` ，改成用 `malloc` 、`rio_readn` 、`rio_writen` 。 根據 [Linux Programer's Manual](https://linux.die.net/man/3/malloc) ， glibc 的 `malloc` 也是使用 `mmap` 與 `sbrk` 來實作的。
* 修改的部份很簡單，先使用 `malloc` 配置完檔案需要的記憶體空間後，再用 `Rio_readn` 來讀取 file description ，剩下的如同前面。

void serve_static(int fd, char *filename, int filesize) 
{
    int srcfd;
    char filetype[MAXLINE], buf[MAXBUF], *fbuf;

    /* Send response headers to client */
    get_filetype(filename, filetype);       //line:netp:servestatic:getfiletype
    sprintf(buf, "HTTP/1.0 200 OK\r\n");    //line:netp:servestatic:beginserve
    sprintf(buf, "%sServer: Tiny Web Server\r\n", buf);
    sprintf(buf, "%sConnection: close\r\n", buf);
    sprintf(buf, "%sContent-length: %d\r\n", buf, filesize);
    sprintf(buf, "%sContent-type: %s\r\n\r\n", buf, filetype);
    Rio_writen(fd, buf, strlen(buf));       //line:netp:servestatic:endserve
    printf("Response headers:\n");
    printf("%s", buf);

    /* Send response body to client */
    srcfd = Open(filename, O_RDONLY, 0);    //line:netp:servestatic:open
    fbuf = malloc(filesize);
    Rio_readn(srcfd, fbuf, filesize);
    Close(srcfd);
    Rio_writen(fd, fbuf, filesize);
    free(fbuf);
}

---
#### `11.10`
##### A. Write an HTML form for the CGI adder function in Figure 11.26. Your form should include two text boxes that users fill in with the two numbers to be added together. Your form should request content using the GET method.
寫一個 adder.html 如下 讓使用者輸入兩個數相加並使用 GET Method

<form action="cgi-bin/adder"  method="GET">
    First Number:<br>
    <input type="text" name="first" value="">
    <br><br>
    Second Number:<br>
    <input type="text" name="second" value="">
    <br><br>
    <input type="submit" value="Submit">
</form> 


##### B. Check your work by using a real browser to request the form from Tiny,submit the filled-in form to Tiny, and then display the dynamic content generated by adder.

原先 adder 支援的 URI 格式如下
`http://localhost:<port>/cgi-bin/adder?<num>&<num>`
而使用 GET 傳輸時的格式為
`http://localhost:<port>/cgi-bin/adder?<name>=<num>&<name>=<num>`

故需要修改 adder.c
原：

    if ((buf = getenv("QUERY_STRING")) != NULL) {
	p = strchr(buf, '&');
	*p = '\0';
	strcpy(arg1, buf);
	strcpy(arg2, p+1);
	n1 = atoi(arg1);
	n2 = atoi(arg2);
    }

改：

    if ((buf = getenv("QUERY_STRING")) != NULL) {
	p = strchr(buf, '&');
	*p = '\0';
	strcpy(arg1, buf);
	strcpy(arg2, p+1);
	n1 = atoi(strchr(arg1,'=')+1);
	n2 = atoi(strchr(arg2,'=')+1);
    }

將原先直接將 `&` 前後兩數字相加改為相加 `=` 後面的數字

---
#### `11.11` Extend Tiny to support the HTTP HEAD method. Check your work using telnet as a Web client.
* 這題的修改方法是在 `serve_static` 和 `serve_dynamic` 多新增 `is_head_method` 參數，用於判斷 response 是否需要有 body 。

sscanf(buf, "%s %s %s", method, uri, version);     //line:netp:doit:parserequest   
if (strcasecmp(method, "HEAD") == 0) // HEAD method do not need response body
    is_head_method = 1;

read_requesthdrs(&rio);  // line:netp:doit:readrequesthdrs
/* Parse URI from HTTP request */
is_static = parse_uri(uri, filename, cgiargs);  // line:netp:doit:staticcheck
if (stat(filename, &sbuf) < 0) {  // line:netp:doit:beginnotfound
    clienterror(fd, filename, "404", "Not found",
                "Tiny couldn't find this file");
    return;
}  // line:netp:doit:endnotfound

if (is_static) { /* Serve static content */
    if (!(S_ISREG(sbuf.st_mode)) ||
        !(S_IRUSR & sbuf.st_mode)) {  // line:netp:doit:readable
        clienterror(fd, filename, "403", "Forbidden",
                    "Tiny couldn't read the file");
        return;
    }
    serve_static(fd, filename, sbuf.st_size, is_head_method);  // line:netp:doit:servestatic
} else {                                       /* Serve dynamic content */
    if (!(S_ISREG(sbuf.st_mode)) ||
        !(S_IXUSR & sbuf.st_mode)) {  // line:netp:doit:executable
        clienterror(fd, filename, "403", "Forbidden",
                    "Tiny couldn't run the CGI program");
        return;
    }
    serve_dynamic(fd, filename, cgiargs, is_head_method);  // line:netp:doit:servedynamic
}

* client 使用 telnet 測試 HEAD method

$ telnet localhost 8080
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
HEAD / HTTP/1.1
HOST:localhost

HTTP/1.0 200 OK
Server: Tiny Web Server
Connection: close
Content-length: 10
Content-type: text/html

Connection closed by foreign host.

* server terminal

$ ./tiny 8080
Accepted connection from (localhost, 46880)
HEAD / HTTP/1.1
HOST:localhost

Response headers:
HTTP/1.0 200 OK
Server: Tiny Web Server
Connection: close
Content-length: 10
Content-type: text/html

---
#### `11.12` Extend Tiny so that it serves dynamic content requested by the HTTP POST method. Check your work using your favorite Web browser.

將原先的 GET adder 改成可支援 POST

adder4post.html 將 method 從 GET 修改為 POST

<form action="cgi-bin/adder"  method="POST">
    First Number:<br>
    <input type="text" name="first" value="">
    <br><br>
    Second Number:<br>
    <input type="text" name="second" value="">
    <br><br>
    <input type="submit" value="Submit">
</form> 

使用 wireshark 擷取封包並觀察
可觀察到 兩個數字皆以被包在封包中傳送
需注意封包的結尾並不是 `\r\n` 故需要修正 `read_requesthdrs`
否則 webserver 會一直等待終止符號

對 read_requesthdrs 以及 parse_uri 做以下修正
原先使用 `Rio_readlineb` 的終止條件是讀取到換行符號
修改為使用 `Rio_readnb` 終止條件為讀至 `rp` 的剩餘大小結束
接著更新 `cgiargs`

read_requesthdrs () {
             ...
    if ( methopType == POST ) { // updte cgiargs when using POST
        Rio_readnb(rp, buf, rp->rio_cnt);
        strcpy(cgiargs, buf);        
    }
             ...
}    

若為 `POST` 則 `cgiargs` 已經修改過，便不再修改
若為 `GET` 則仍需更新 `cgiargs`

parse_uri () {
             ...
    else { /* Dynamic content */  
        if ( methodType == GET ) { // update cgiargs when using GET
            ptr = index(uri, '?');      
            if (ptr) {
                strcpy(cgiargs, ptr + 1);
                *ptr = '\0';
            } else
                strcpy(cgiargs, "");  
        }
             ...
}    

成果：

---
#### `11.13` Modify Tiny so that it deals cleanly (without terminating) with the SIGPIPE signals and EPIPE errors that occur when the write function attempts to write to a prematurely closed connection.
* 當 client 與 server 已經連接上，在 server 要對 client 寫入 response 時， client 無預警的斷線，造成 server 寫入已經關閉的 socket ，這便會產生 `SIGPIPE` 訊息，並且 `errno` 被設定為 `EPIPE` 。第一次對關閉後的 socket 做寫入， client 會返回 TCP Reset ，第二次寫入才會觸發 `SIGPIPE` 。
* 此題目標便是要能夠妥善處理 `SIGPIPE` 訊號，使 server 能不被 `SIGPIPE` 終止。
* 以下為修改流程：
1. 捕捉 `SIGPIPE` 訊號：
在 `main` 中增加 `Signal(SIGPIPE, SIG_IGN);` 來使 server 可以攔截 `SIGPIPE` ，`SIG_IGN` 表示忽略該訊號。預設的 `Rio_writen` 內，如果寫入發生錯誤， `unix_error` 會來處理錯誤，並中止程式。為了讓程式能繼續運行，需要修改 `Rio_writen` 。
2. 將 `Rio_writen` 改為 `improve_Rio_writen` ：
大致上與 `Rio_writen` 一樣，差別在於多了 `errno` 的判斷式，如果攔截到 `SIGPIPE` ，那 `errno` 會被設為 `EPIPE` ，就此得知 client 已關閉。其餘錯誤仍交給 `unix_error` 處理。

    int improve_Rio_writen(int fd, void *usrbuf, size_t n) {
        if(rio_writen(fd, usrbuf, n) != n){
            if(errno == EPIPE)
                printf("client side has disconnected\n");
            else
                unix_error("Rio_writen error");
            return -1;
        }
        return 0;
    }

3. `serve_static` 修改：
將 `Rio_writen` 取代成 `improve_Rio_writen` ，即可完成此部份的修改。
4. `serve_dynamic` 修改：
`serve_dynamic` 內總共使用了兩次 `Rio_writen` ，如果在這兩次之前 client 就已經關閉，那後面再 fork 子程式來執行也就沒意義了，故在 fork 之前，須改成如果出現 `SIGPIPE` ，則直接 return ; 若是在子程式內才發生，則交給子程式處理。
* 測試結果：
* 測試方法使用 [Netcat](https://en.wikipedia.org/wiki/Netcat) 指令來模擬 client 連線後意外中斷。為了能夠有時間按 ctrl-C ，所以會在 `doit` 函式中增加 `sleep(3)` 。
* Client 執行

    $ echo -ne "GET / HTTP/1.0\r\n\r\n" | nc localhost 8080
    ^C

* Server terminal ，在第二次寫入，也就是寫入 response body 時才會產生 `SIGPIPE` ，寫入 response header 仍是正常返回。

    $ ./tiny 8080
    Accepted connection from (localhost, 38496)
    GET / HTTP/1.0

    Response headers:
    HTTP/1.0 200 OK
    Server: Tiny Web Server
    Connection: close
    Content-length: 120
    Content-type: text/html

    client side has disconnected

---
Reference:
[網路程式中的 SIGPIP 訊號](http://senlinzhan.github.io/2017/03/02/sigpipe/)
[Netcat (Linux nc指令)](https://blog.gtwang.org/linux/linux-utility-netcat-examples/)
[文件描述符 流 流缓冲的一些概念与问题](http://www.cnblogs.com/liqiuhao/p/7676125.html)