B. Load Balancing

# B. Load Balancing ## DNS A **Domain Name System** (DNS) translates a domain name such as www.example.com to an IP address. ### DNS Structure DNS is implemented as a **hierarchical and distributed database** containing various types of data, including host names and domain names. The names in a DNS database form a **hierarchical tree structure** called the domain namespace. Domain names consist of individual labels separated by dots, for example: mydomain.microsoft.com. A **fully qualified domain name** (FQDN) uniquely identifies the host’s position within the DNS hierarchical tree by specifying a list of names separated by dots in the path from the referenced host to the root. The following figure shows an example of a DNS tree with a host called mydomain within the microsoft.com. domain. The FQDN for the host would be mydomain.microsoft.com. ![](https://i.imgur.com/7dElpnC.png) This figure shows how Microsoft is assigned authority by the Internet root servers for its own part of the DNS domain namespace tree on the Internet. DNS clients and servers use queries as the fundamental method of resolving names in the tree to specific types of resource information. This information is provided by DNS servers in query responses to DNS clients, which then extract the information and pass it to a requesting program for resolving the queried name. In the process of resolving a name, keep in mind that DNS servers often function as DNS clients, querying other servers in order to fully resolve a queried name. <details> <summary>Types of DNS domain names</summary> <table class="table"> <colgroup> <col style=""> <col style=""> <col style=""> </colgroup> <thead> <tr class="header"> <th>Name Type</th> <th>Description</th> <th>Example</th> </tr> </thead> <tbody> <tr class="odd"> <td>Root domain</td> <td>This is the top of the tree, representing an unnamed level; it is sometimes shown as two empty quotation marks (""), indicating a null value. When used in a DNS domain name, it is stated by a trailing period (.) to designate that the name is located at the root or highest level of the domain hierarchy. In this instance, the DNS domain name is considered to be complete and points to an exact location in the tree of names. Names stated this way are FQDNs.</td> <td>A single period (.) or a period used at the end of a name, such as “example.microsoft.com.”</td> </tr> <tr class="even"> <td>Top-level domain</td> <td>A name used to indicate a country/region or the type of organization using a name.</td> <td>““.com”, which indicates a name registered to a business for commercial use on the Internet.</td> </tr> <tr class="odd"> <td>Second-level domain</td> <td>Variable-length names registered to an individual or organization for use on the Internet. These names are always based on an appropriate top-level domain, depending on the type of organization or geographic location where a name is used.</td> <td>““microsoft.com. ”, which is the second-level domain name registered to Microsoft by the Internet DNS domain name registrar.</td> </tr> <tr class="even"> <td>Subdomain</td> <td>Additional names that an organization can create that are derived from the registered second-level domain name. These include names added to grow the DNS tree of names in an organization and divide it into departments or geographic locations.</td> <td>““example.microsoft.com. ”, which is a fictitious subdomain assigned by Microsoft for use in documentation example names.</td> </tr> <tr class="odd"> <td>Host or resource name</td> <td>Names that represent a leaf in the DNS tree of names and identify a specific resource. Typically, the leftmost label of a DNS domain name identifies a specific computer on the network. For example, if a name at this level is used in a host (A) resource record, it is used to look up the IP address of computer based on its host name.</td> <td>““host-a.example.microsoft.com.”, where the first label (“host-a”) is the DNS host name for a specific computer on the network.</td> </tr> </tbody> </table> </details> <details> <summary>Resource Records Types</summary> <table class="table"> <thead> <tr class="header"> <th>Description</th> <th>Class</th> <th>Time to Live (TTL)</th> <th>Type</th> <th>Data</th> <th>Description</th> </tr> </thead> <tbody> <tr class="odd"> <td>Start of Authority</td> <td>Internet (IN)</td> <td>Default TTL is 60 minutes</td> <td>SOA</td> <td>Owner Name Primary Name Server DNS Name, Serial Number Refresh Interval Retry Interval Expire Time Minimum TTL</td> <td>The ‘start of authority’ record can store important info about the domain such as the email address of the administrator, when the domain was last updated, and how long the server should wait between refreshes.</td> </tr> <tr class="even"> <td>Host</td> <td>Internet (IN)</td> <td>Record-specific TTL if present, or else zone (SOA) TTL</td> <td>A</td> <td>Owner Name (Host DNS Name) Host IP Address</td> <td>Host IP address</td> </tr> <tr class="odd"> <td>Name Server</td> <td>Internet (IN)</td> <td>Record-specific TTL if present, or else zone (SOA) TTL</td> <td>NS</td> <td>Owner Name Name Server DNS Name</td> <td>Indicates which DNS server is authoritative for that domain (which server contains the actual DNS records). A domain will often have multiple NS records which can indicate primary and backup name servers for that domain.</td> </tr> <tr class="even"> <td>Mail Exchanger</td> <td>Internet (IN)</td> <td>Record-specific TTL if present, or else zone (SOA) TTL</td> <td>MX</td> <td>Owner Name Mail Exchange Server DNS Name, Preference Number</td> <td>Mail exchange record directs email to a mail server. The MX record indicates how email messages should be routed in accordance with Simple Mail Transfer Protocol</td> </tr> <tr class="odd"> <td>Canonical Name (an alias)</td> <td>Internet (IN)</td> <td>Record-specific TTL if present, or else zone (SOA) TTL</td> <td>CNAME</td> <td>Owner Name (Alias Name) Host DNS Name</td> <td> Points a name to another name or CNAME (example.com to www.example.com) or to an A record.</td> </tr> </tbody> </table> </details> Services such as **CloudFlare** and **Route 53** provide managed DNS services. Some DNS services can route traffic through various methods: * Weighted round robin * Prevent traffic from going to servers under maintenance * Balance between varying cluster sizes * A/B testing * Latency-based * Geolocation-based ### Disadvantage(s): DNS * Accessing a DNS server introduces a slight delay, although mitigated by caching described above. * DNS server management could be complex and is generally managed by governments, ISPs, and large companies. * DNS services have recently come under DDoS attack, preventing users from accessing websites such as Twitter without knowing Twitter's IP address(es). ## Load Balancer ![](https://i.imgur.com/sdCSBOi.png) Load balancers distribute incoming client requests to computing resources such as application servers and databases. In each case, the load balancer returns the response from the computing resource to the appropriate client. ### Benefits * **[Horizontal scaling](https://hackmd.io/MVQTvj9DSwqLDPNqujjMSg#Horizontal-scaling)** - Load balancers can also help with horizontal scaling, improving performance and availability * Preventing requests from going to **unhealthy servers** * Preventing **overloading** resources * Helping eliminate single points of failure (**SPOF**) * **SSL termination** - Decrypt incoming requests and encrypt server responses so backend servers do not have to perform these potentially expensive operations * Removes the need to install **X.509 certificates** on each server * **Session persistence** - Issue cookies and route a specific client's requests to same instance if the web apps do not keep track of sessions ### Implementations Load balancers can be implemented with **hardware** (expensive) or with software such as **HAProxy**. To protect against failures, it's common to set up multiple load balancers, either in [active-passive](https://hackmd.io/MVQTvj9DSwqLDPNqujjMSg#Active-passive) or [active-active](https://hackmd.io/MVQTvj9DSwqLDPNqujjMSg#Active-active) mode. #### Load balancering algorithms * Random * Least loaded * Session/cookies (Sticky session) * Round robin or weighted round robin #### Weighted round robin In a weighted round-robin algorithm, each destination (in this case, server) is assigned a value that signifies, relative to the other servers in the pool, how that server performs. This “weight” determines how many more (or fewer) requests are sent that server’s way, compared to the other servers on the pool. ##### Sticky session Balance according to the session, so that all requests from same users are processed by same server. In case of one web-server, this server can store user session data in memory. In case there are more instances of the server this session data should be either persisted in shared data storage or it needs to be garanteed that user requests are processed by same server. L4 LB supports sticky sessions based on source ip address L7 LB uses cookies to base sticky sessions on #### NLB vs ALB ##### Layer 4 load balancing (Network LB) Layer 4 load balancers look at info at the transport layer to decide how to distribute requests. Generally, this involves the source, destination IP addresses, and ports in the header, but not the contents of the packet. Layer 4 load balancers forward network packets to and from the upstream server, performing Network Address Translation (NAT). ##### Layer 7 load balancing (Application LB) Layer 7 load balancers look at the application layer to decide how to distribute requests. This can involve contents of the header, message, and cookies. Layer 7 load balancers terminates network traffic, reads the message, makes a load-balancing decision, then opens a connection to the selected server. For example, a layer 7 load balancer can direct video traffic to servers that host videos while directing more sensitive user billing traffic to security-hardened servers. At the cost of flexibility, layer 4 load balancing requires less time and computing resources than Layer 7, although the performance impact can be minimal on modern commodity hardware. ### Disadvantages * The load balancer can become a performance bottleneck if it does not have enough resources or if it is not configured properly. * Introducing a load balancer to help eliminate single points of failure results in increased complexity. * A single load balancer is a single point of failure, configuring multiple load balancers further increases complexity. ## Reverse proxy (web server) ![](https://i.imgur.com/dKCDnd5.png) A reverse proxy is a web server that centralizes internal services and provides unified interfaces to the public. For example, www.example.com/index and www.example.com/sports appear to come from the same domain, but in fact they are from different micro services behind the reverse proxy. Requests from clients are forwarded to a server that can fulfill it before the reverse proxy returns the server's response to the client. ### Benefits * **[Horizontal scaling](https://hackmd.io/MVQTvj9DSwqLDPNqujjMSg#Horizontal-scaling)** - Reverse proxy can work as LB, thus provides HS * **Increased security** - Hide information about backend servers, blacklist IPs, limit number of connections per client * **Increased scalability and flexibility** - Clients only see the reverse proxy's IP, allowing you to scale servers or change their configuration * **SSL termination** - Decrypt incoming requests and encrypt server responses so backend servers do not have to perform these potentially expensive operations * Removes the need to install **X.509 certificates** on each server * **Compression** - Compress server responses * **Caching** - Return the response for cached requests * **Static content** - Serve static content directly * HTML/CSS/JS * Photos * Videos * Etc ### Disadvantage Introducing a reverse proxy results in increased complexity. A single reverse proxy is a single point of failure, configuring multiple reverse proxies (ie a failover) further increases complexity. ## Load balancer vs Reverse proxy * Deploying a load balancer is useful when you have multiple servers. Often, load balancers route traffic to a set of servers serving the same function. * Reverse proxies can be useful even with just one web server or application server, opening up the benefits described in the previous section. * Solutions such as NGINX and HAProxy can support both layer 7 reverse proxying and load balancing. ## Application layer ![](https://i.imgur.com/Vv5Emfk.png) Separating out the web layer from the application layer (also known as platform layer) allows you to scale and configure both layers independently. Adding a new API results in adding application servers without necessarily adding additional web servers. The single responsibility principle advocates for small and autonomous services that work together. Small teams with small services can plan more aggressively for rapid growth. ### Microservices Related to this discussion are microservices, which can be described as a suite of independently deployable, small, modular services. Each service runs a unique process and communicates through a well-defined, lightweight mechanism to serve a business goal. Pinterest, for example, could have the following microservices: user profile, follower, feed, search, photo upload, etc. ### Service Discovery Systems such as **Consul**, Etcd, and Zookeeper can help services find each other by keeping track of registered names, addresses, and ports. **Health checks** help verify service integrity and are often done using an HTTP endpoint. Both Consul and Etcd have a built in key-value store that can be useful for storing config values and other shared data. ### Disadvantages * Adding an application layer with loosely coupled services requires a different approach from an architectural, operations, and process viewpoint (vs a monolithic system). * Microservices can add complexity in terms of deployments and operations. <details> <summary>AWS</summary> ![](https://i.imgur.com/Pi38G0X.png) </details> TODO https://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer