# Chapter 2 Application Layer
### 2.1 principles of network applications
+ Application architectures
+ 兩種基本的架構，不管是什麼樣的網路應用協定，大致上都可將架構分為兩種:
+ client-server
+ peer-to-peer (P2P)
+ 

+ Client-server architecture
+ server:
+ 提供服務的角色
+ always-on host
+ permanent IP address
+ clients:
+ 向 Server 要求服務
+ may have dynamic IP addresses
+ do not communicate directly with each other
+ P2P architecture
+ **Client 是否也可以同時當 Server，接受別的 Client 的連線**== noalways-on server
+ peers request service from other peers, provide service in return to other peers
+ **self scalability–new peers bring new service capacity, as well as new service demands**
+ peers are intermittently connected and change IP addresses
+ Processes communicating
+ process:program running within a host
+ within same host, two processes communicate using **inter-process communication**(defined by OS)
+ processes in different hosts communicate by **exchanging messages**
:::info
+ client process: process that initiates communication
+ server process: process that waits to be contacted
:::
+ Sockets
+ socket analogous to door
+ process sends/receives messages to/from its socket
+ 
+ Addressing processes
+ to receive messages, process must have identifier
:::info
+ Q:does IP address of host on which process runs suffice for identifying the process?
+ A:no, manyprocesses can be running on same host
:::
+ **identifier** includes both **IP address and port** numbersassociated with process on host.
+ to send HTTP message to gaia.cs.umass.edu web server:
+ IP address:128.119.245.12
+ port number:80
+ What transport service does an app need?
+ data integrity 完整性
+ some need 100% e.g. file transfer
+ some don't e.g. audio
+ timing
+ some app require low delay to be "effective" e.g. interactive games
+ throughput 吞吐量
+ some apps require minimum amount of throughput to be “effective”
+ some (elastic apps)only make use of whatever throughput they get
+ security
+ encryption, data integrity.
+ 
+ Internet transport protocols services
+ TCP service:
+ reliable transport
+ flow control:sender won’t overwhelm receiver
+ congestion control:throttle sender when network overloaded
+ connection-oriented:setup required between client and server processes
+ does not provide:timing, minimum throughput guarantee, security
+ UDP service:
+ unreliable data transfer
+ does not provide:reliability, flow control, congestion control, timing, throughput guarantee, security, or connection setup,
+ 
+ Securing TCP
+ for TCP and UDP send cleartext passwds into socket
+ SSL(Secure Sockets Layer) 安全通訊端層 憑證
+ provides encrypted TCP connection
+ end-point authentication
+ apps use SSL libraries, that “talk”to TCP

---
### 2.2 Web and HTTP
+ HTTP overview: hypertext transfer protocol
+ Web’s application layer protoco
+ client/server model
+ client:browser that requests, receives, (using HTTP protocol) and “displays”Web objects
+ server:Web server sends (using HTTP protocol) objects in response to requests
+ 
+ uses TCP:
+ client initiates connection to server
+ server accept
+ HTTP message exchange
+ connection closed
+ HTTP is **“stateless”**
+ server maintains no information about past client requests
+ two type
+ non-persistent HTTP
+ 當每傳送一個物件，TCP connection就必須做一次
+ 主要為HTTP1.0使用，相較於HTTP1.1 較費時，因為有一部份的時間都花在建立連線上。
+ response time
+ **RTT (definition):time for a small packet to travel from client to server and back**
+ one to initiate **TCP connection** the other one for **HTTP request** and few bytes of HTTP response to return
+ file transmission time
+ non-persistent HTTP response time = 2RTT+ file transmission time
+ 

+ persistent HTTP
+ 接收多個物件時，client只需要向server建立一次連線即可
+ 較省時。主要為HTTP1.1使用。
+ 
+ 
+ HTTP request message
+ two types of HTTP messages: request, response
+ ASCII (human-readable format)
+ 
+ 
+ Uploading form input
+ POST method:
+ web page often includes form input 表單
+ input is uploaded to server in entity body
+ URL method:
+ uses **GET** method
+ input is uploaded in **URL field of request line**: www.somesite.com/animalsearch?monkeys&banana
+ Method types
+ HTTP/1.0:
+ GET
+ POST 在Request-URI所标识的资源后附加新的数据
+ HEAD 请求获取由Request-URI所标识的资源的响应消息报头
+ HTTP/1.1:
+ GET, POST, HEAD
+ PUT 请求服务器存储一个资源，并用Request-URI作为其标识
+ DELETE 请求服务器删除Request-URI所标识的资源
+ HTTP response message
+ 
+ HTTP response status codes
+ 200 OK
+ request succeeded, requested object later in this msg
+ 301 Moved Permanently
+ requested object moved, new location specified later in this msg (Location:)
+ 400 Bad Request
+ request msg not understood by server
+ 404 Not Found
+ requested document not found on this server
+ 505 HTTP Version Not Supported
+ Trying out HTTP (client side) for yourself
+ 
+ User-server state: cookies
+ many Web sites use cookies
+ four components:
+ cookie header line of HTTP responsemessage
+ cookie header line in next HTTP requestmessage
+ cookie file kept on user’s host, managed by user’s browser
+ back-end database at Web site(accessed by users indirectly through an external application rather than by application programming stored within the database itself or by low level manipulation of the data)
+ 
+ Cookies: keeping “state”
+ 
+ what cookies can be used for:
+ authorization, shopping cart, recommendation, user session state (Web e-mail)
+ how to keep “state”:
+ protocol endpoints: maintain state at sender/receiver over multiple transactions
+ cookies: http messages carry state
+ Web caches (proxy server 代理伺服器)
+ satisfy client request without involving origin server
+ user sets browser: Web accesses via cache
+ browser sends all HTTP requests to cache
+ object in cache: cache returns object
+ else cache requests object from origin server, then returns object to client
+ 會把大家常常看的網頁資料暫存在一個位置
+ 瀏覽器會先向DNS要ip，然後找到ip後，會先向proxy server 查詢是否有這個網站的資料，如果有的話，Proxy Server 就直接把內容傳給了使用者，如果proxy server 沒有資料的話，才會直接向網站要資料。
+ 
:::info
+ pros and cons of web cache - proxy server
+ pors
+ substantially reduce response time for a client request(尤其當origin & cache的bottleneck頻寬<<client & cache的)
+ reduce traffic on insititution's access link to the internet => insititution 不需要升級頻寬 => 省成本
+ reduce traffic in the internet as a whole => improve performance of all application
+ 一種具有匿名代理伺服器作用的網頁代理，在伺服器執行線上代理程式,可以以此伺服器作為跳板來上網瀏覽，可以隱藏自己的真實IP和資訊，不但上網更安全，更引起多重帳戶使用者的青睞
+ 也可以過濾遠端網站的內容。一些審查軟體也跟網頁代理差不多，將一些網站內容過濾掉。出於某些目的或保護，有些網頁代理程式也會將網站的內容重新編排。
+ 可以過濾掉一些含有病毒和木馬的危險檔案，這可以使瀏覽體驗更安全。
+ 增加上網的安全性
+ 透過Proxy瀏覽網頁,大部分都是留下Proxy的IP而非用戶IP
+ 降低因伺服器故障而無法存取網頁
+ 如果WWW伺服器暫時停擺,透過Proxy仍可存取你想要的資料

+ cons
+ 在中國，即使使用網路代理也會過濾關鍵字，成為審查物件，同樣也無法存取許多網站
+ 網頁代理雖然可以存取大部分網站，但是並不能保證所有的網站都是可以存取的。
+ 有一些網頁代理也支援安全連接HTTPS，但是最好不要用網頁代理存取涉及個人隱私的網站。
+ 增加反應時間
+ 當用戶端送出要求,會先存取Proxy上的資料,如果Proxy沒有,才會連WWW伺服器,增加反應時間
+ 無法得到最新的資料
+ 通常Proxy會設定快取資料保存時間,過了特定時間,才會主動向目的WWW伺服器要求新的資料;而當WWW伺服器更新資料,而Proxy的快取資料未過期前,那你所得到的就不會是最新的資料
:::
+ More about Web caching
+ cache acts as both client and server
+ typically cache is installed by ISP (university, company, residential ISP)
+ why Web caching?
+ **reduce response time for client request**
+ reduce traffic on an institution’s access link
+ **Internet dense with caches: enables “poor”content providers to effectively deliver content (so too does P2P file sharing)**
+ Caching example: fatter access link
+ case 1: original
+ 

+ case 2: increase access link(fatter)
+ 
+ decrease acces delay very much
+ case 3: using local cache
+ 
+ 

+ some formula
+ lan utilization = (access link rate / lan )%
+ access link utilization = average data rate to browser / access link rate
+ 有cache就看他資料配對的成功比例*average data rate to browser / access link rate
+ total delay = internet delay + access delay + lan delay
+ 0.6 (2 + ~msecs for access link and LAN) + 0.4 (~usecs for LAN極小) + usecs(極小)= ~ 1.2 secs
+ **HTTP Conditional GET**
+ Goal:don’t send object if cache has up-to-date cached version
+ no object transmission delay
+ link utilization
+ cache:specify date of cached copy in HTTP request
+ If-modified-since: <date\>
+ server:response contains no object if cached copy is up-to-date:
+ HTTP/1.0 304 Not Modified
+ 資料已經是最新的不需要update
+ 

:::info
+ cache與cookie、session的區別
    + Cookie是儲存客戶端的一組資料，主要用來儲存使用者的個人資訊，主要存放瀏覽器請求伺服器時的請求資訊，這些資訊是非敏感資訊。
    + Session是由**應用伺服器維持的一個伺服器端的儲存空間**，是一種儲存上下文資訊的機制，它是針對每一個使用者的。使用者在連線伺服器時，伺服器會生成一個唯一的SessionID，用該**SessionID為識別符號來存取伺服器端的Session儲存空間**，面SessionID這一資料是以**Cookie形式儲存在客戶端**。使用者提交頁面時，會將SessionID提交到伺服器端，來存取Session資料。這一過程是不用開發人員來干預的，所以一旦客戶端禁用Cookie，Session理論上也會失效，但伺服器也可以自動通過URL重寫的方式來傳遞SessionID的值，因此也不是完全依賴Cookie，並且這個過程對於開發人員是透明的。
    + Cache儲存於**伺服器的記憶體中**，允許您自定義如何快取資料項，以及快取多長時間。當系統缺乏記憶體時，快取會自動移除很少使用的或優先順序較低的快取項，以釋放記憶體，此過程稱為清理。這是快取爲了確保過期資料不再佔用寶貴的伺服器資源的方式之一。

:::

---
### 2.3 electronic mail •SMTP, POP3, IMAP
+ Electronic mail
+ Three major components
+ user agents
+ a.k.a. **“mail reader”**
+ composing, editing, reading mail messages
+ e.g., Outlook, Thunderbird, iPhone mail client
+ outgoing, incoming messages stored on server
+ mail servers
+ mailbox **contains incoming messages for user**
+ message queue of outgoing (to be sent) mail messages
+ SMTP protocol between mail servers to send email messages
+ client: sending mail server
+ “server”: receiving mail server
+ simple mail transfer protocol: SMTP
+ 
+ uses TCP to reliably transfer email message from client to server, port 25
+ direct transfer: sending server to receiving server
+ three phases of transfer
+ handshaking (greeting)
+ transfer of messages
+ closure
+ command/response interaction (like HTTP)
+ commands:ASCII text
+ response:status code and phrase
+ messages must be in 7-bit ASCI
+ Scenario: Alice sends message to Bob
+ 
+ 
+ **HELO, MAIL FROM, RCPT TO, DATA, QUIT**
+ above lets you send email without using email client (reader)
+ SMTP: final words
+ SMTP server is that it is insecure, it can be easily hacked. There are so-called “fake emails” that are messages sent using any address (for example [e-mail]) to any recipient.
+ SMTP uses persistent connections
+ SMTP requires message (header & body) to be in 7-bit ASCII
+ SMTP server uses CRLF.CRLFto determine end of message
+ comparison with HTTP
+ HTTP: each object encapsulated in its own response message
+ SMTP: multiple objects sent in multipart message
+ both have ASCII command/response interaction, status codes
+ HTTP: pull SMTP: push
+ Mail message format
+ SMTP: protocol for exchanging email messages
+ RFC 822: standard for text message format:
+ header lines
+ e.g.To: From: Subject:
+ **differentfrom SMTP MAIL FROM, RCPT TO:commands!**
+ Body: the “message”
+ ASCII characters only
+ 
+ Mail access protocols
+ 
+ SMTP:delivery/storage to receiver’s server
+ mail access protocol: retrieval from server
+ POP:Post Office Protocol [RFC 1939]: authorization, download
+ IMAP:Internet Mail Access Protocol [RFC 1730]: more features, including manipulation of stored messages on server
+ HTTP:gmail, Hotmail, Yahoo! Mail, etc.
+ POP3 protocol
+ 
+ POP3 (more) and IMAP
+ more about POP3
+ previous example uses **POP3 “download and delete”mode**
+ Bob cannot re-read e-mail if he changes client
+ POP3 “download-and-keep”: copies of messages on different clients
+ POP3 is **stateless across sessions**
:::info
**Stateless Protocol:**
Stateless Protocols are the type of network protocols in which Client send request to the server and server response back according to current state. It does not require the server to retain session information or a status about each communicating partner for multiple request.
:::
+ IMAP
+ keeps all messages in one place: at server
+ allows user to organize messages in folders
+ keeps user state across sessions:
+ names of folders and mappings between message IDs and folder name
:::info
+ Why must SMTP User agents send mail through a mail server?
+ If Alice were to send an email directly to Bob, Bob's user agent would have to be always on (much like a server). But, since Bob is just another client that can be off, Alice's email would not be able to reach Bob in case Bob's user agent was off. Therefore, a mail server would store Alice's email to Bob and when Bob's user agent is on, it can retrieve the new email. Hence, the need for mail to be sent via mail servers
+ 1. Sending via local mail/SMTP server makes User/Mail Agent simpler. It lowers chances of hitting "special cases" before passing responsibility for email delivery. It helps to avoid "unstable" behavior.
+ Sending email through SMTP is actually a very complex undertaking. Even without spam, you wouldn't want to implement the whole feature set of the SMTP protocol in every email client; you are better off with a dedicated piece of software

+ another general reason
+ 2. Sending directly to recipient's mail server is even more complicated **and risky** in spam era.

:::
---
### 2.4 DNS
+ DNS: domain name system
+ distributed databaseimplemented in hierarchy of many name servers
+ **application-layer protocol**:hosts, name servers communicate to resolvenames (address/name translation)
+ DNS: services, structure
+ hostname to IP address translation
+ host aliasing
+ why not centralize DNS?
+ single point of failure
+ traffic volume
+ distant centralized database maintenance
+ DNS: a distributed, hierarchical database
+ 
+ DNS: root name servers
+ contacts authoritative name server if name mapping not known
+ gets mapping
+ returns mapping to local name server
+ 
+ TLD, authoritative servers
+ top-level domain (TLD 頂級網域) servers
+ responsible for com, org, net, edu, aero, jobs, museums, and all top-level country domains, e.g.: uk, fr, ca, jp
+ Network Solutions maintains servers for .com TLD
+ Educause for .edu TLD
+ authoritative DNS servers:
+ organization’s own DNS server(s), providing authoritative hostname to IP mappings for organization’s named hosts
+ can be maintained by organization or service provider
+ Local DNS name server
+ does not strictly belong to hierarchy
+ each ISP (residential ISP, company, university) has one
+ also called “default name server”
+ when host makes DNS query, query is sent to its local DNS server
+ has local cache of recent name-to-address translation pairs (but may be out of date!)
+ acts as proxy, forwards query into hierarchy
+ DNS name resolution example
+ iterated query:
+ i don't know but i will ask who may know and let you contact he
+ local name server是直接面對自己主機的name server,當host想找某個name的IP時會先問local DNS server,不在範圍內的話，會救助於root DNS server,然後root會回應要到哪個TLD DNS server尋找,TLD再告訴local要到哪個authoritative DNS server, 最後再取得IP

+ 
+ recursive query:
+ 差別在root server不把TLD server回給local,直接順著傳到TLD,同樣TLD也不會回local,等最後的IP取得後，再順著原本的路徑回給host
+ puts burden of name resolution on contacted name server
+ heavy load at upper levels of hierarchy
+ 
+ DNS: caching, updating records
+ once (any) name server learns mapping, it cachesmapping
+ cache entries timeout (disappear) after some time (TTL)
+ TLD servers typically cached in local name servers thus root name servers not often visited
+ cached entries may be out-of-date(best effort name-to-address translation!)
+ if name host changes IP address, may not be known Internet-wide **until all TTLs expire**
+ update/notify mechanisms proposed IETF standard
+ RFC 2136
+ DNS records
+ DNS:distributed database storing resource records (RR)
+ 
+ 
+ DNS protocol, messages
+ queryand replymessages, both with same message format
+ 
+ 
+ Inserting records into DNS
+ 
+ Attacking DNS
+ DDoS attack
+ bombard root servers with traffic
+ local DNS servers cache IPs of TLD servers, allowing root server bypass
+ bombard TLD servers
+ redirect attacks
+ exploit DNS for DDoS

---
### 2.5 P2P applications
+ Pure P2P architecture
+ no always-on server
+ arbitrary end systems directly communicate
+ peers are intermittently connected and change IP addresses
+ 
+ File distribution
:::info
Q: how much time to distribute file (size F) from one server to N peers?
(peer upload/download capacity is limited resource)

:::
+ client-server
+ server transmission: mustsequentially send (upload) N filecopies:
+ time to send one copy: F/us
+ time to send Ncopies: NF/us
+ client: each client must download file copy
+ dmin= min client download rate
+ min client download time: F/dmin
+ 

+ P2P
+ server transmission: must upload at least onecopy
+ client: each client must download file copy
+ clients: as aggregate must download NFbits
+ max upload rate (limiting max download rate) is us+ SIGGMA ui
+ 
+ 
+ P2P file distribution: BitTorrent
:::info
希望我們不是在睡覺
:::
---
### 2.6 video streaming and content distribution networks
+ Streaming multimedia: DASH Dynamic, Adaptive Streaming over HTTP**基於HTTP的動態自適應流**
+ 會將內容分解成一系列小型的基於HTTP的檔案片段，每個片段包含很短長度的可播放內容，而內容總長度可能長達數小時（例如電影或體育賽事直播）。內容將被製成多種位元速率的備選片段，以提供多種位元速率的版本供選用。當內容被MPEG-DASH用戶端回放時，用戶端將根據目前網路條件自動選擇下載和播放哪一個備選方案。用戶端將選擇可及時下載的最高位元速率片段進行播放，從而避免播放卡頓或重新緩衝事件。也因如此，MPEG-DASH用戶端可以無縫適應不斷變化的網路條件並提供高品質的播放體驗，擁有更少的卡頓與重新緩衝發生率
+ “intelligence”at client: client determines
+ when to request chunk (so that buffer starvation, or overflow does not occur)
+ what encoding rate to request (higher quality when more bandwidth available)
+ where to request chunk (can request from URL server that is “close”to client or has high available bandwidth)
+ content distribution network
+ case 1: single, large server
+ cons
+ single point of failure
+ network congestion
+ long path to client
+ multiple copies of video sent over outgoing link
+ case 2: Content Delivery Network(CDN)
+ store 很多copy散在其他據點
+ pros
+ enter deep: push CDNs servers deep into many access networks
+ bring home
+ Content Delivery Network(CDN)
+ 

+ subscriber requests content from CDN
+ directed to nearby copy, retrieves content
+ may choose different copy if network path congested
+ OTT (Over The Top)challenge coping with a congested Internet
+ 
+ 

---
### 2.7 socket programming with UDP and TCP
+ Socket programming L6 L7
+ learn how to build client/server applications that communicate using sockets
+ door between application process and end-end-transport protocol
+ 
+ Two socket types
+ UDP:unreliable datagram
+ TCP:reliable, byte stream-oriented
+ Socket programming with UDP
+ UDP: no “connection”between client & server
+ no handshaking before sending data
+ sender explicitly attaches IP destination address and port # to each packet
+ receiver extracts sender IP address and port# from received packet
+ UDP: transmitted data may be lost or received out-of-order
+ Application viewpoint:
+ UDP provides unreliabletransfer of groups of bytes (“datagrams”) between client and server
+ 
+ 
+ 
+ Socket programming with TCP
+ client must contact server
+ server process must first be running
+ server must have created socket (door) that welcomes client’s contact
+ client contacts server by:
+ Creating TCP socket, specifying IP address, port number of server process
+ when client creates socket:client TCP establishes connection to server TCP
+ when contacted by client, server TCP creates new socketfor server process to communicate with that particular client
+ allows server to talk with multiple clients
+ source port numbers used to distinguish clients (more in Chap 3)
+ application viewpoint:
+ TCP provides reliable, in-order byte-stream transfer (“pipe”) between client and server
+ 
+ 
+ 

[reference link]http://securityalley.blogspot.com/2014/06/blog-post.html
[reference link]https://www.cnblogs.com/wenqiang/p/5698772.html
[reference link]https://sls.weco.net/blog/eric97979/05-oct-2008/11116
[reference link]https://www.youtube.com/watch?v=0OrmKCB0UrQ
[reference link]difference between cache cookie session https://itw01.com/V6SABE3.html

###### tags: `network`
`