# Project1 README file

This is Ho, Ngok Chao's Readme file. Spring 2023 for Project 1 of CS6200 Graduate Intro to OS

## Warmup (Echo)
### Understanding

This is a typical Echo Server - Client pair, Client sends 15 bytes content to server and then server send the same content back. At the end, the client receive it and export to stdout.

### Implementation and Testing

Referencing beej's guide, in order to receive and send

- On server side, I called:

socket() // to get a socket file descriptor
setsockopt() // to allow reusing the port
bind() // bind to a port
listen() // set maximum pending connection 
accept() // get a new socket file descriptor
recv() // using the new socket fd to receive
send() // using the new socket fd to send

- On client side, I called:

socket() // to get a socket file descriptor
connect() // make a connection session
send()
recv()

`char buffer[16]` on stack is used as buffer to send and receive message. Since no loss is expected, it is assumed 15 bytes can be put into and read from buffer by calling send() and recv() one time.

Tested with `./echoclient -m messages`

## Warmup (Transferfile)

### Understanding
Server read a pre-defined file and send to client connecting to it. How many bytes each time the server process can read and send is not predetermined, hence I have to keep track of how much is read and sent each time, and continue from there.

### Implementation and Testing

Same kind of basic setup as previous echo exercise is needed.

On client side putting `recv()` and `fwrite()` into a while loop until number of bytes received is zero.

On server side putting `fread()` into a while loop until number of bytes available to read is zero; for each chunk of bytes read, they are further handled by `sendall()` which itself is a function sending via `send()` chunk by chunk until all bytes are sent. This is function is taken from *Beej's Guide to Network Programming*.

Tested with `./transferserver -f files`

## Part 1 (gfclient)
### Understanding

Part 1 is about implementing a file transfer service according to the Getfile Protocol; for the client part, we expect to send request like `<scheme> <method> <path>\r\n\r\n` and receive `<scheme> <status> <length>\r\n\r\n<content>` The server part is expected to receive and send the opposite. Following the existing structure of `gfclient.c`, all parameters, connection state is saved in a `struct gfcrequest_t` so that a single source of truth is shared every where and no global variable is needed.

While the struct of `gfcrequest_t` for all info is create via `malloc` on heap which is unnecessary for a single threaded server, it make it easier to further extend to multi-threading given that the struct is on heap hence can be shared among threads.
### Flow of Control
The most important function in `gfclient.c` is `gfc_perform` and hence I desmontrate its control flow as following

### Implementation and Testing

#define BUFSIZE 2048

I set the buffer for recv() and send() to 2048 (not too small or too large), and hence the maximum size each chunk receive/send is 2048 bytes.

The longest valid server response header (from start until \r\n\r\n) is less than 50 bytes, hence if header is longer than 50 bytes, it is safe to return -1 (invalid response)

GETFILE OK 18446744073709551616\r\n\r\n

 struct gfcrequest_t
 {
  const char *server;
  const char *path;
  unsigned short port;
  void (*writefunc)(void *, size_t, void *);
  void (*headerfunc)(void *, size_t, void *);
  void *writearg, *headerarg;
  char buf[BUFSIZE];
  char header[BUFSIZE];
  size_t header_index;
  size_t bytes_received;
  size_t terminated;
  int sockfd;
 };

As mentioned in *understanding*, my implementation put all information into gfcrequest_t above as a single source of truth.

Following `gfclient_download.c`, we can see the usage of `gfclient`
- `gfc_global_init`: empty, no global var is used
- `gfc_create(gfcrequest_t **gfr)`: allocates memory for this struct on heap and then it is initialised via the following
- `gfc_set_port` : set the port
- `gfc_set_path` : set path
- `gfc_set_server`: set address for server
- `gfc_set_writefunc(&gfr, writecb)`: register a write down function pointer into this struct
- `gfc_set_writearg`: put arguemnts for writing function into this struct
- `gfc_perform`: use information mentioned above, to create sockfd and receive info into `header` (information before `\r\n\r\n`) and `buf`(content) and then use registered callback `writefunc` to export, and close sockfd when all done. Details in control flow graph above
- `gfc_cleanup`: free the memory for the struct on heap
- `gfc_global_cleanup`: empty, no global var is used.

**testing**
Using `gfserver_main_half_test` from Project 1 Interop Thread, my client can download file succesfully.

Using tests modified from 6200-tools, I ran the following test

- test 1: receive 8192 bytes from server side to check if the request is valid, SIGTERM the server to close connection and check if client exit
- test 2: Server send the header, in two pieces. Then send 1 byte payload. Client should succeed.
- test 3: Server send the header without `\r\n\r\n`, and then close the sockfd, client should report gfc_perform return -1, received 0 bytes and status is INVALID
- test 4: prematurely closed connection during transfer of the message body
- test 5: non decimal file length in response, gfc_perform returns -1
- test 6: Send a file containing CR LF, received files with 2 bytes
- test 7: sned a Send a file containing CR LF CR LF, received files with 4 bytes
- test 8: send file with 2 ^ 31 - 1 bytes
- test 9: Serve a random sized file with random-sized buffers of random bytes, and compare the SHA1 hash.

> Note: size_t, stroull (convert string to unsigned long long) is used hence the maximum file size supported should be **18,446,744,073,709,551,615**

## Part 1 (gfserver)
### Understanding
The server part would answer to request `<scheme> <method> <path>\r\n\r\n` and send `<scheme> <status> <length>\r\n\r\n<content>` Following the existing structure of `gfserver.c`, server related info is stored in struct `gfserver_t` while connection specific information is stored in `gfcontext_t`.

Just like `gfcrequest_t` in the client part, `gfserver_t` is created on heap which is beneficial for sharing purpose if we decided to make some functions to be handled by multiple threads; Similarly `gfcontext_t` is also created on heap, later in part 2, it is shown how we have multiple threads working together, with a global queue of pointer to pointer to `gfcontext_t`.

From Piazza, `\r\n\r\n` is always expected, hence the server is calling recv() until we see `\r\n\r\n` instead of setting a timeout.

### Flow of Control
The most important function in `gfserver.c` is `gfs_serve` and hence I desmontrate its control flow as following


### Implementation and Testing

struct gfserver_t
{
  unsigned short port;
  int max_npending;
  gfh_error_t (*handler_callback)(gfcontext_t **, const char *, void*);
  void* handlerarg;
  char buf[BUFSIZE];
};

struct gfcontext_t
{
  char header[BUFSIZE];
  char path[BUFSIZE];
  size_t header_index;
  int sockfd;
};

Per **understanding** server parameters are put into `gfserver_t`, per-request related information is put into `gfcontext_t`

Following `gfserver_main.c`, we can see the usage of `gfserver`
- content_init: Init content library
- gfserver_create: malloc gfserver_t
- gfserver_set_handler: set handling function, which would call `gfs_send` and `gfs_sendheader`
- gfserver_set_port: set listening port of the server
- gfserver_sex_maxpending: set maximum pending connection
- gfserver_set_handlerarg: set arg for handler function
- gfserver_serve: run the server, described in **Flow of Control**

**testing**
Using `gfclient_main_half_test` from Project 1 Interop Thread, my server can send file succesfully.

Using tests modified from 6200-tools, I ran the following test (particularly skipping those without `\r\n\r\n` is in request)
- test 1: Send an complete request, FILE_NOT_FOUND is expected
- test 2: Send invalid scheme and method. No handling is expected.
- test 3: Using 4096 bytes PATH, FILE_NOT_FOUND is expected
- test 4: request is sent in tiny pieces, FILE_NOT_FOUND is expected
- test 5: Path is / alone. FILE_NOT_FOUND is expected
- test 6: Path without /. `b'GETFILE INVALID\r\n\r\n'` is expected
- test 7: Send an complete request, and read in small pieces with delays.Ctrl+C to kill the client and see server give up this connection
- test 8: Send a complete request, but close before receiving the response. Server should keep running
- test 9: Send an incomplete request, then close. Server should keep running.

## Part 2 (multi-threaded gfserver)

### Understanding
This part of the project is to write `gfserver_main.c` using gflib above in multithreaded manner. A global variable of `steque_t` is used; The main thread (boss) would put incoming request (gfcontext_t) into this global steque_t, slave threads would take job from the `steque_t`.

A global mutex **gServerSharedMemoryLock** is used to protect the global `steque_t` shared among threads.

### Flow of Control

### Implementation and Testing
Global variable used

steque_t server_steque;
pthread_mutex_t gServerSharedMemoryLock = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t gServerReadPhase = PTHREAD_COND_INITIALIZER;
pthread_cond_t gServerWritePhase = PTHREAD_COND_INITIALIZER;

steque_t is a singly linked list implemented deque.

typedef struct steque_node_t{
  steque_item item;
  struct steque_node_t* next;
} steque_node_t;

typedef struct{
  steque_node_t* front;
  steque_node_t* back;
  int N;
}steque_t;

Mutex protected section for access global variable **server_steque**

pthread_mutex_lock(&gServerSharedMemoryLock);
		while (!steque_isempty(&server_steque)) {
			pthread_cond_wait(&gServerWritePhase,&gServerSharedMemoryLock);
		}
        ...
        steque_enqueue(&server_steque, item);
        ...
pthread_mutex_unlock(&gServerSharedMemoryLock);

Struct representing a request

typedef struct {
    gfcontext_t *ctx;
    char *path;
    void* arg; 
} handler_item;

The global variable shared we want to protect is **server_steque** whose member is pointer to a `handler_item` (information for incoming request).

For the main thread, mutex is used in `gfs_handler`, the predicate for granting access is when steque_isempty; when job taken by slave thread, slave thread would wake up main thread to check for this predicate and then enqueue if access granted.

On the other hand, the predicate for granting access for slave thread, is when steque is not empty i.e. we have job to take; if so, pop the steque and take the job and wake up the boss thread.

Details of control flow is described in **Flow of Control**.

**testing**
Using `gfclient_main_half_test` from Project 1 Interop Thread, my server can send file succesfully.

Using tests modified from 6200-tools, I ran the following test (particularly skipping those no ``\r\n\r\n` is in request)
- test 1: Send an complete request, FILE_NOT_FOUND is expected
- test 2: Send invalid scheme and method. No handling is expected.
- test 3: Using 4096 bytes PATH, FILE_NOT_FOUND is expected
- test 4: request is sent in tiny pieces, FILE_NOT_FOUND is expected
- test 5: Path is / alone. FILE_NOT_FOUND is expected
- test 6: Path without /. `b'GETFILE INVALID\r\n\r\n'` is expected
- test 7: Send an complete request, and read in small pieces with delays.Ctrl+C to kill the client and see server give up this connection
- test 8: Send a complete request, but close before receiving the response. Server should keep running
- test 9: Send an incomplete request, then close. Server should keep running.

## Part 2 (multi-threaded gfclient)

### Understanding
We need to implement in `gfclient_download.c` a multithreaded version usage of `gfclient`. Main thread would be the boss thread enqueuing download requests and slave threads would take job (callling gflib gfc_perform to send and recv) to request and download files.

### Flow of Control


### Implementation and Testing

unsigned short port = 10880;
char *server = "localhost";
int nrequests = 12;
int gfinished_requests = 0;
steque_t work_steque;
pthread_mutex_t gSharedMemoryLock = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t gWritePhase = PTHREAD_COND_INITIALIZER;
pthread_cond_t gReadPhase = PTHREAD_COND_INITIALIZER;

The above global variable is used to facilitate multithreading design.

A global `steque_t` is used to keep the info for user input requests. slave threads are used to pop info out of steque_t and call gfc_perform in gflib to receive file. Sepereate condition variable and opposite predicate are used for boss thread and slave thread so that only one thread would have access to the global steque_t to avoid error. (enqueue and pop at the same time would lead to incorrect result due to they are not atomic operation for linked list).

The predicate for boss thread to enqueue is when steque_isempty is True; we wrap the mutex lock section in a for loop for the n requests we get from user.

for (int i = 0; i < nrequests; i++) {
    pthread_mutex_lock(&gSharedMemoryLock);
        while (!steque_isempty(&work_steque)) {
          pthread_cond_wait(&gWritePhase, &gSharedMemoryLock);
        }
        steque_enqueue(&work_steque, workload_get_path());
        pthread_cond_signal(&gReadPhase);
    pthread_mutex_unlock(&gSharedMemoryLock);
}

The predicate for slave thread to pop is when steque_isempty is not True and possible number of pending job to finish job >=0; For the slave thread, if there is no possible pending job to finish then we can just exit, otherwise call `gfc_perform`.

**testing**
Using `gfserver_main_half_test` from Project 1 Interop Thread, my client can download file succesfully.

Using tests modified from 6200-tools, I ran the following test

- test 1: receive 8192 bytes from server side to check if the request is valid, SIGTERM the server to close connection and check if client exit
- test 2: Server send the header, in two pieces. Then send 1 byte payload. Client should succeed.
- test 3: Server send the header without \r\n\r\n, and then close the sockfd, client should report gfc_perform return -1, received 0 bytes and status is INVALID
- test 4: prematurely closed connection during transfer of the message body
- test 5: non decimal file length in response, gfc_perform returns -1
- test 6: Send a file containing CR LF, received files with 2 bytes
- test 7: sned a Send a file containing CR LF CR LF, received files with 4 bytes
- test 8: send file with 2 ^ 31 - 1 bytes
- test 9: Serve a random sized file with random-sized buffers of random bytes, and compare the SHA1 hash.

## References

- Instructions to use vscode codespace https://github.com/ngokchaoho/GT_GIOS_ENV/blob/main/articles/codespace_instructions.md

- waitpid https://linux.die.net/man/2/waitpid

- getaddrinfo https://man7.org/linux/man-pages/man3/getaddrinfo.3.html

- Project 1 Interop Thread https://piazza.com/class/lco3fd6h1yo48k/post/79

- 6200-tools https://github.gatech.edu/cparaz3/6200-tools/

- Beej's Guide to Network Programming https://beej.us/guide/bgnet/ (function sendall is taken from here, and warmup for echo is done referencing this)