# PentesterLab - HTTP # HTTP 01 - CURL ###### tags: `Web` ![](https://hackmd.io/_uploads/ryQqNowH2.png) ## What is the cURL command? Client URL is a command line tool that enables data exchange between a device and a server through a terminal. Using thiss command line interface (CLI), a user specifiles a server URL (the location where they want to send a request) and the data they want to send to that server URL ## How to Use cURL The syntax of a curl command is: ```py curl [options] [URL] ``` ## Request Data From a Source Using the **GET** method with curl, you can quickly request data from a source or API. Here's a simple curl command that makes a **GET** request ```py curl http://ptl-3f129e47-22ee3bd1.libcurl.so/pentesterlab ``` Result ![](https://hackmd.io/_uploads/rJ6aEiwr2.png) ## Writing your script I will try to write a python script to send a **GET** request to the host To make HTTP requests in python, we can use several HTTP libraries like: - httplib - urllib - requests The most elegant and simplest of above listed libraries is **requests**. To download and install requests library, use following command: ``` pip install requests ``` And here is my python code: ![](https://hackmd.io/_uploads/r1o3G3vH3.png) Now we know if we just print(resp) we only get the status code Here's how to print the content: ![](https://hackmd.io/_uploads/rJOeV2vB2.png) # HTTP 02 - URL PARAMETER - QUERRY STRINGS ###### tags: `Web` ![](https://hackmd.io/_uploads/Skg8-UnDB2.png) ## About URL parameters URL parameter (known also as "query strings" or "URL query parameters") are elements inserted in your URLs to help you filter and organize content or track information on your website. To identify a URL parameter, refer to the portion of the URL that comes after a question mark (?). URL parameters are made of a key and a value, separated by an equal sign (=). Multiple parameters are each then separated by an ampersand (&). A URL string with parameters looks like this: ![](https://hackmd.io/_uploads/HkbE92wHh.png) Back to the challenge, we can easily get the key for this challenge by identify the parameter in the URL like this. ![](https://hackmd.io/_uploads/rkY51TDHh.png) With curl: ![](https://hackmd.io/_uploads/ByNT1TvH2.png) With python script: ![](https://hackmd.io/_uploads/HkIDWTPSn.png) # HTTP 03 - COOKIE ###### tags: `Web` ![](https://hackmd.io/_uploads/r1jmFHdH3.png) ## What is Cookie? ![](https://hackmd.io/_uploads/SkwfAruH3.png) When we are browsing websites, HTTP requests are sent out. HTTP is a stateless protocol. Each request is totally independent, and the server is not able to confirm the identity of the user. Therefore, the server and browser will trace a session (knowing who is accessing) to maintain a status to tell the server if the requests are coming from the same browser. This can be realized through **cookies** and **sessions** A **cookie** is a piece of data that is sent from the server to the user's browser. The subsequent requests from the broser will bring along the same cookie to the server again. Each cookie is bound to a single domain name which can't be reused under other domains. Some importand attributes inside a cookie include: - *name:value* - a key/value pair where you can set the name of the Cookie and its corresponding value - *domain:* the domain that the cookie belongs to and current domain name by defautl - *path:* under which URL path the cookie is effective - *maxAge/expires:* for how long the cookie stays effective - *secure:* shows if the cookie is transported with a secure protocol including HTTPS and SSL - *httpOnly:* a tag added to a browser cookie that prevents client-side scripts from accessing Read more about cookie and session: https://medium.com/geekculture/what-the-tech-is-cookie-session-and-token-36bf7c3754a0 ## Send a GET with cookie ### Sending Cookies with Curl Cookies are passed to Curl with the --cookie "Name=Value" command line parameter. Curl automatically converts the given parameter into the Cookie: Name=Value request header Syntax: ``` curl --cookie "key=please" "https://ptl-5c765dad-e9939db6.libcurl.so/pentesterlab" ``` Or we can send an HTTP header with a Curl request, we can use the -H command-line option and pass the header name and value in "Key:Value" format. Syntax: ``` curl https://ptl-5c765dad-e9939db6.libcurl.so/pentesterlab -H "Cookie: key=please" ``` ![](https://hackmd.io/_uploads/H1z18U_S2.png) ### Writing a script ![](https://hackmd.io/_uploads/HkK7wUdSh.png) # HTTP 04 - HEADER CONTENT-TYPE ###### tags: `Web` ![](https://hackmd.io/_uploads/HkCowIOS3.png) ## What is Content-Type? The Content-Type header is used to indicate the media type of the resource. The media type is a string sent along with the file indicating the format of the file. For example, for image file its media type will be like image/png or image/jpg, etc. In response, it tells about the type of returned content, to the client. The browser gets to know about the type of content it has to load on the machine. Every time its byte stream of the file that browsers receive, by the Content-type header, the browser will do something known as MIME sniffing i.e it will inspect the stream it is receiving and then loads the data accordingly. ## Send a GET request with the Content-Type Just like the previous challenge, we can send an HTTP header Content-Type with Curl request by using -H command-line option. ![](https://hackmd.io/_uploads/r1yX3LuHh.png) With Python script ![](https://hackmd.io/_uploads/r1YQ68OH2.png) # HTTP 05 - HEADER ACCEPT-LANGUAGE ###### tags: `Web` ![](https://hackmd.io/_uploads/B1eA6LuHn.png) ## What is Accept-Language header? HTTP Accept-Language headder tells the server about all the languages that client can understand. With the help of content negotiation, there will be a set of supported languages in the HTTP Accept-Language proposal then the server selects one of the proposals of those languages and place that int the **content-language** header. In a few cases users can change the languages manually otherwise server detects the supported language by the browser's language. ## Send a GET request with Accept-Language ![](https://hackmd.io/_uploads/rJ8uevdS2.png) # HTTP 06 - METHOD ###### tags: `Web` ![](https://hackmd.io/_uploads/SkUgZvuH2.png) ## HTTP Method HTTP works as a request-response protocol between a client and server. Example: A client (browser) sends an HTTP request to the server; then the server returns a response to the client. The response contains status information about the request and may also contatin the requested content. HTTP Method: - GET - POST - PUT - HEAD - DELETE - PATCH - OPTIONS - CONNECT - TRACE The two most common HTTP methods are: GET and POST ### The GET Method: GET is used to request data from a specified resource. Note that the query string (name/value pairs) is sent in the URL of a GET request: ``` /test/demo_form.php?name1=value1&name2=value2 ``` > **Some notes on GET requests:** > - GET requests can be cached > - GET requests remain in the browser history > - GET requests can be bookmarked > - GET requests should never be used when dealing with sensitive data > - GET requests have length retricitons > - GET requests are only used to request data (not modify) ### The POST Method: POST is used to send data to a server to create/update a resource. The data sent to the server with POST is stored in the request body of the HTTP request ``` POST /test/demo_form.php HTTP/1.1 Host: w3schools.com name1=value1&name2=value2 ``` > **Some notes on POST requests:** > - POST requests are never cached > - POST requests do not remain in the browser history > - POST requests cannot be bookmarked > - POST requests have no restrictions on data length Check out the difference between GET and POST and other methods here: https://www.w3schools.com/tags/ref_httpmethods.asp ## Send a POST request with parameter With Curl: ![](https://hackmd.io/_uploads/S1wBgtdr2.png) With Python Script: ![](https://hackmd.io/_uploads/H1tTC_dHn.png) **NOTE:** `params` is for GET-style URL parameters, `data` is for POST-style body information. # HTTP 07 ###### tags: `Web` ![](https://hackmd.io/_uploads/BJhAlKdH3.png) We just need to send POST request without specifying anything in the body ![](https://hackmd.io/_uploads/HkgbfYdrn.png) # HTTP 08 ###### tags: `Web` ![](https://hackmd.io/_uploads/rkx4mYOB3.png) ![](https://hackmd.io/_uploads/SyxF4Yurh.png) # HTTP 09 ###### tags: `Web` ![](https://hackmd.io/_uploads/BJQCEKdHh.png) ![](https://hackmd.io/_uploads/S1i76FuH3.png) **NOTE:** Usually we will specify the `param` as a key/value pair like this ``` params = {'key':'please'} ``` In dictionary, if we define key with 2 different values, it only take 1 value ![](https://hackmd.io/_uploads/Hk6hDc_Hn.png) If you want to define 2 key, 2 value with the same key, same value, you should specify it like this. ``` params = {'key':['please','please']} ``` # HTTP 10 ###### tags: `Web` ![](https://hackmd.io/_uploads/ryOes9OSh.png) Just like previous challenge, change method from GET to POST ![](https://hackmd.io/_uploads/HkjPq9OSn.png) # HTTP 11 ###### tags: `Web` ![](https://hackmd.io/_uploads/ry9tiquSn.png) ![](https://hackmd.io/_uploads/SJUGksOB2.png) # HTTP 12 - URL ENCODE ###### tags: `Web` ![](https://hackmd.io/_uploads/S1-ozsOH3.png) ![](https://hackmd.io/_uploads/H1pFBh_B3.png) If we want to URL encode a string using the urlencode() function, we cannot do it directly because it does not accept a string as an argument. We can pass the argument in the form of a dictionary. Dictionary is a data type in Python that stores the data in a key-value pair. We have to import the urlib library, and we will pass our query string to the urlencode() function of the parse module of the urlib library. In the output, we will get the required encoded URL. In the output, the encoded URL will be in the form where spaces will be replaced by + symbol and : symbol is replaced by & symbol because it is the format of the encoded URL of the query string. # HTTP 13 ###### tags: `Web` ![](https://hackmd.io/_uploads/ryz3u2_Bn.png) ![](https://hackmd.io/_uploads/H1ccKhOr2.png) # HTTP 14 ###### tags: `Web` ![](https://hackmd.io/_uploads/HySMinOHh.png) ![](https://hackmd.io/_uploads/rk2IhhdSh.png) # HTTP 15 ###### tags: `Web` ![](https://hackmd.io/_uploads/rJ2AR3dBh.png) ![](https://hackmd.io/_uploads/BJ7_JadSn.png) # HTTP 16 ###### tags: `Web` ![](https://hackmd.io/_uploads/rJwk7adSn.png) ![](https://hackmd.io/_uploads/B1_0zT_r3.png) # HTTP 17 - GET NULL BYTE ###### tags: `Web` ![](https://hackmd.io/_uploads/SyNjQHoHn.png) Searched for null byte in URL encode ![](https://hackmd.io/_uploads/rJdbVHor3.png) ![](https://hackmd.io/_uploads/Hk-08BiS2.png) # HTTP 18 ###### tags: `Web` ![](https://hackmd.io/_uploads/BJvUPHoSn.png) First, I try `key=please%00%00` but it's not work, then I try to encode it twice like this ![](https://hackmd.io/_uploads/Skli5BoS3.png) And bingo ![](https://hackmd.io/_uploads/B1A35roB3.png) # HTTP 19 - GET ARRAY ###### tags: `Web` ![](https://hackmd.io/_uploads/HJHZaHiH3.png) To send a parameter as an array, you can use the following syntax: ``` name[0]=value1&name[1]=value2. ``` ![](https://hackmd.io/_uploads/Hk5leUiB3.png) When use curl we need to remember that special characters that have particular meanings to the shell. So we need to change the meaning of a character is to precede it with a backslash (\\) ![](https://hackmd.io/_uploads/SkobxLorn.png) With python: I googled the syntax to send array params in python ![](https://hackmd.io/_uploads/B1LgO8oHn.png) But it's not work. ![](https://hackmd.io/_uploads/rkgK2OIir2.png) Then I found out that the URL sent was incorrect. So I change the code a little bit and bingo. ![](https://hackmd.io/_uploads/Sy79tIjB3.png) **NOTE:** We can debug the URL sent with `.url` as follows: ```python print(urllib.parse.unquote(res.url)) ``` # HTTP 20 - GET HASH/DICTIONARY ###### tags: `Web` ![](https://hackmd.io/_uploads/SkwZR8iH2.png) To send the parameter as a hash/dictionary, you can use the following syntax: ``` name[key]=value ``` ![](https://hackmd.io/_uploads/SJ7ebwsS3.png) With curl: ![](https://hackmd.io/_uploads/ryffMPjHn.png) With python: ![](https://hackmd.io/_uploads/Bks6fviBn.png) # HTTP 21 - CUSTOM METHOD HACK ###### tags: `Web` ![](https://hackmd.io/_uploads/r1mM4Dor3.png) Use option `-X` to specify method ![](https://hackmd.io/_uploads/H1p8NPsSh.png) In python I use `Request` and `urlopen` in urllib library to make this custom method - "HACK" ![](https://hackmd.io/_uploads/HkpWw3sSn.png) # HTTP 22 - X-HTTP-Method-Override ###### tags: `Web` ![](https://hackmd.io/_uploads/HJO7OnoSh.png) ## What is HTTP Method Override? In the first line of an HTTP request, the method to be used is specified. Basically, this method indicates what action the user wants to perform on the server. For example, the GET method is commonly applied to read a piece of certain information and the PUT method to add or modify data on ther server. Although there are several other HTTP methods, not all of them are supported on user's devices. For example, very old or very simple browsers, or embedded devices with minimal resouces, may not support all the methods available. So if the server with which these devices communicate need to use one of these unsupported methods, communication would not be possible due to lack of compatibility. In order to allow access to such clients, the solution found was to implement, on the server-side, functions that were capable of receiving a method but interpreting the request with another method. For example: recceive a request with a POST method and interpret it with a PUT method. This concept of `overriding` the method that is being given defines quite well what Method Override is - receiving one method and interpreting another. But how could the server know which requests should have their original methods overwritten? The solution found wwas for the client to pass information in the request, indicating that the method sould be overridden. The information appears in the following forms: - Headers: X-Http-Method-Override, X-HTTP-Method-Override, X-Http-Method, X-HTTP-Method, X-Method-Override; - Parameters in the URL: _method, method, httpMethod, _HttpMethod, and also those in the previous item; - Request body: any of the names mentioned above Thus, if the server supports Method Override, the method will be changed and the request will be processed accurately. The developers have agreed that Method Override should only occur when the original method of the request is POST. However this is not a rule. ## From a security point of view As we have seen, the only function of Method Override is to change the method of the request. With this, an attacker wouldn't be able to do much. It's very likely that the device the attacker uses to perform the attack does not suffer from the same limitations as the ones mentioned above and thus support all HTTP methods. So, in this case he could simply send the request using the method accepted by the application and thus communicate directly with it. With this in mind, the question emerges: Why would an attacker use Methos Override instead of sending the method directl? To answer this question, let look at this lab. The lab consists of a simple HTTP server and an Nginx reverse proxy between the client and the server. The Nginx has been configured to only accept requests with the GET, HEAD and POST methods for this lab. Any other method will be prevented from going through to the server. However, the HTTP server still accepts the PUT and DELETE methods.Knowing that it accepts these methods and these methods usually perform some sensitive operations, it would be interesting for an attacker to test them and see what he coudl achieve. This is the essence of Method Overide: to communicate directly with the server, bypassing possible HTTP method filters that are in the way, such as an API gateway, a reverse proxy, or a firewall. ref: https://sidechannel.blog/en/http-method-override-what-it-is-and-how-a-pentester-can-use-it/ ## Send request with header X-HTTP-Method-Override With curl: ![](https://hackmd.io/_uploads/HkTCOpjSh.png) With Python: ![](https://hackmd.io/_uploads/ByJRD6oS2.png) # HTTP 23 - X-Forwarded-For ###### tags: `Web` ![](https://hackmd.io/_uploads/B1J7RHnHn.png) ## What is X-Forwarded-For? The `X-Forwarded-For` (XFF) request header is a de-facto standard header for identifying the originating IP address of a client connecting to a web server through a proxy server. When a client connects directly to a server, the client's IP address is sent to the server (and is often written to server access logs). But if a client connection passes through any [forward or reverse proxy](https://en.wikipedia.org/wiki/Proxy_server), the server only sees the final proxy's IP address, which is often of little use. That's especially true if the final proxy is a load balancer which is part of the same installation as the server. So, to provide ma more-useful client IP address to the server, the `X-Forwarded-For` header is used. ## Security an privacy concerns This header, by design, exposes privacy-sensitive information, such as the IP address of the client. Therefore the user's privacy must be kept in mind when deploying this header. The `X-Forwarded-For` header is untrustworthy when no trusted reverse proxy (e.g., a load balancer) is between the client and server. If the client and all proxies are beign and well-behaved, then the list of IP addresses in the header has the meaning as mentioned before. But if there's a risk the client or any proxy is malicious or misconfigured, thin it's possible any part (or the entirety) of the header may have been spoofed. Using untrustworthy values can result in rate-limiter avoidance, access-control bypass, memory exhaustion, or other negative security or availability consequences. ## Send request with header X-Forwarded-For With curl: ![](https://hackmd.io/_uploads/Syd0P8nS2.png) With Python: ![](https://hackmd.io/_uploads/ry3cw82Sh.png) # HTTP 24 - X-Forwarded-Host ###### tags: `Web` ![](https://hackmd.io/_uploads/SyeAuInr3.png) ## What is X-Forwarded-Host? The X-Forwarded-Host (XFH) header is a de-facto standard header for identifying the original host requested by the client in the Host HTTP request header. Host names and ports of reverse proxies (load balancers, CDNs) may differ from the origin server handling the request, in that case the X-Forwarded-Host header is useful to determine which Host was originally used. This header is used for debugging, statistics, and generating location-dependent content and by design it exposes privacy sensitive information, such as the IP address of the client. Therefore the user's privacy must be kept in mind when deploying this header. ## Send request with header X-Forwarded-Host ![](https://hackmd.io/_uploads/rkJrKU2rh.png) # HTTP 25 - URL normalization ###### tags: `Web` ![](https://hackmd.io/_uploads/H18dFI2Sn.png) ## What is URL/URI normalization? URL/URI normalization is the proces by which URIs are modified and standardlized in a consistent maner For more information: https://en.wikipedia.org/wiki/URI_normalization We can use curl and tell it not to normalize the url by using `--path-as-is` flag: ![](https://hackmd.io/_uploads/rykwNDhBn.png) With Python: ``` import requests requests.packages.urllib3.disable_warnings() url = "https://ptl-c5852c4a-d2f7e63f.libcurl.so/pentesterlab/../pentesterlab" s = requests.Session() req = requests.Request(method='GET', url=url) prep = req.prepare() prep.url = url r = s.send(prep, verify=False) print(r.text) ``` In short, the above code will mainly use the `prepare()` function to standardize the request, but it has not yet been sent. Then we will replace the canonicalized URL part with the URL we want. Finally send the request using the `send()` function. # HTTP 26 ###### tags: `Web` ![](https://hackmd.io/_uploads/HkXWLO3S3.png) ![](https://hackmd.io/_uploads/ryYSUdnH3.png) ![](https://hackmd.io/_uploads/H12YIdnH3.png) # HTTP 27 ###### tags: `Web` ![](https://hackmd.io/_uploads/BkRN5O3r3.png) ## Use URL encode to send request ![](https://hackmd.io/_uploads/r13nddhS2.png) ![](https://hackmd.io/_uploads/HyB8KO3r2.png) ![](https://hackmd.io/_uploads/r15GcOnSn.png) ## Use option --request-target in curl ![](https://hackmd.io/_uploads/SkI43u2rh.png) # HTTP 28 - Multipart ###### tags: `Web` ![](https://hackmd.io/_uploads/SySeTO2Sh.png) ## What is Multipart? HTTP is the foundationg of today's internet. Not only is it used to browse webpages, but it's also used to interact with APIs and even live stream video. It doesn't matter what kind of content you want to send; HTTP will happily transmit anything. You only need to let the receiver know what kind of data you're sending by setting a proper `Content-Type` header. But what if you need to send data of multiple types in a single request? Say, your cat's name and picture? This is where multipart content type comes into play :3 The multipart content type is widespread. You might not realize it, but you've most likely used it many times in your life. The most common use for multipart requests is web forms. When you have a from with only text-based input elements - like text inputs, checkboxes, date pickers, etc. - your browser sends that data as an `application/x-www-form-urlendcoded` payload, which essentially means encoding the inputs as `key=value` pairs. However, as soon as the form contains an input that cannot be encoded as text, such as a file input, the browser uses the `multipart/form-data` content type. The other situation where a multipart content type is used is when sending emails with attachments. A regular email is just text, and so it's sent with `text/plain`, or sometimes the `text/html` content type. But PDF or image attachments aren't text - they're binary data formats, and so a multipart message is used to send both the email's text and the attachment. ## Detailed example When you sending an HTML form through a browser in an HTTP call, the data contents can be sent in as request body below two formats. - application/x-www-form-urlencoded - multipart/from-data For most of the cases, **application/x-www-form-urlencoded** can be used. **application/x-www-form-urlencoded** is not much efficient for - Sending Files or images - Sending large quantities of binary data or text which contains non-ASCII characters For example, let's say that the below data needs to be sent. - Name - Age Then **application/x-www-form-urlencoded** can be used to send the above data. But let's say that you also need to send the profile photo of the user in the request as well. So the data is now as below - Name - Age - Photo In the above case, it will not be efficient to use **application/x-www-form-urlencoded** content-type. **multipart/form-data** should be used in this case. Why is that? We will answer this question once we have to understood the format of **application/x-www-form-urlencoded** and **multipart/from-data** **application/x-www-form-urlencoded** encodes each non-ASCII byte to 3 bytes. Basically **application/x-www-form-urlencoded** content-type request body is like a fiant querry string. Similar to the query string in a URI it is a key-value pair having the below format ```python key1=value1&key2=value21&key2=value22&key3=value3 ``` While sending **application/x-www-form-urlencoded**, all the non-alphanumeric charactoers are URL encoded in the below format: ```python %WW ``` Where WW is the ASCII code of the alphanumeric character represented in hexadecimal format. As all the non-alphanumeric characters in the binary data are URL encodedd where 1 byte is converted to 3 bytes. So size is increased three folds. So if you sending a file or image which is a lot of binary data then your payload will be very big, almost thrice the size of the actual payload. Hence it is inefficient for sending large binary files or large NON-ASCII data. Now let's understand the format of **multipart/form-data** The format of **multipart/form-data** is as below ``` -- Content-Disposition: form-data; name="" Content-Type: [DATA] -- Content-Disposition: form-data; name=""; filename="" Content-Type: [DATA] ---- ``` The above format is divided into two parts. Also, - Each part is separated by a delimiter or boundary - Each part contains its own headers to describe the type of data - Content-Disposition header to describe the type of data. Contains the name field. This field contains the key name. If the part is a file and it will also a filename field - Each part will also contain its own data. Let’s say we are sending the below data as part of multipart/form-data request - name = John - age =23 - photo = Some binary data And let’s say the delimiter or boundary is ``` xyz ``` Then the format will be as below ``` --xyz Content-Disposition: form-data; name="name" Content-Type: text/plain John --xyz Content-Disposition: form-data; name="age" Content-Type: text/plain 23 --xyz Content-Disposition: form-data; name="photo"; filename="photo.jpeg" Content-Type: image/jpeg [JPEG DATA] --xyz-- ``` As **multipart/form-data** will send the binary data as it is, that is why it is used for sending files and large binary data. Now the question is. Why not use form-data all the time then? The reason is that for small data the additional requirement of boundary string and headers will outweigh any optimizations. ## Full HTTP multipart request look like ``` POST /cgi-bin/qtest HTTP/1.1 Host: aram User-Agent: Mozilla/5.0 Gecko/2009042316 Firefox/3.0.10 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://aram/~martind/banner.htm Content-Type: multipart/form-data; boundary=2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f Content-Length: 514 --2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f Content-Disposition: form-data; name="datafile1"; filename="r.gif" Content-Type: image/gif GIF87a.............,...........D..; --2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f Content-Disposition: form-data; name="datafile2"; filename="g.gif" Content-Type: image/gif GIF87a.............,...........D..; --2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f Content-Disposition: form-data; name="datafile3"; filename="b.gif" Content-Type: image/gif GIF87a.............,...........D..; --2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f-- ``` ## Send request using HTTP multipart Set the header Content-Type of the request to multipart/form-data ![](https://hackmd.io/_uploads/BytzDjhH2.png) ![](https://hackmd.io/_uploads/Bk7KDi2r2.png) # HTTP 29 - Send file ###### tags: `Web` ![](https://hackmd.io/_uploads/SkIHKi3H3.png) With curl: ![](https://hackmd.io/_uploads/H1Cojo2Hn.png) With python: Basically, if you specify a files parameter (a dictionary), then requests will send a **multipart/form-data** POST instead of a **application/x-www-form-urlencoded** POST. ![](https://hackmd.io/_uploads/ryvMynnrh.png) # HTTP 30 ###### tags: `Web` ![](https://hackmd.io/_uploads/HJlKdCpHh.png) ## What is directory traversal? **Directory traversal** (also known as file path traversal) aims to access files and directories that are stored outside the web root folder. By manipulating variables that reference files with **"dot-dot-slash (../)"** sequences and its variations or by using absolute file paths, it may be possible to access arbitrary files and directories stored on file system including application source code or configuration and critical system files. It should be noted that access to files is limited by system operational access control (such as in the case of locked or in-use files on the Microsoft Windows operating system). This attack is also known as “dot-dot-slash”, “directory traversal”, “directory climbing” and “backtracking”. ## Send filename contain ../ With curl: We cannot name the file ../ because the operating system does not allow it. So I tried to encode it to e%2e%2e%2f but it's not work. ![](https://hackmd.io/_uploads/ryffRGRSh.png) And then I found this https://security.stackexchange.com/questions/177307/path-traversal-via-filename/264147#264147?newreg=b3d20a4caa2748f49213396a9aa0d157 Syntax: ``` curl Your-URL -F "filename=@key.txt;filename=../../key.txt" ``` Here the first #filename is local file name and the second #filename is a HTTP request. This is a quick way to test path traversal in a file upload functionality. ![](https://hackmd.io/_uploads/SyZw0G0Hh.png) With python we can set the filename before send like this: Syntax ``` files = {'filename': ('../atleastonebyte', open('atleastonebyte','rb'))} ``` ![](https://hackmd.io/_uploads/SkVwH4kUn.png) # HTTP 31 ###### tags: `Web` ![](https://hackmd.io/_uploads/S1AEIEJUn.png) Sending XML in the body of a POST message ![](https://hackmd.io/_uploads/rkGvD41In.png) With Python: ![](https://hackmd.io/_uploads/HJSUuEkU2.png) # HTTP 32 ###### tags: `Web` ![](https://hackmd.io/_uploads/ryw45N1L2.png) ![](https://hackmd.io/_uploads/Sy63qEy83.png) ![](https://hackmd.io/_uploads/rkoMoVJI2.png) # HTTP 33 ###### tags: `Web` ![](https://hackmd.io/_uploads/r1j_nEJLn.png) ![](https://hackmd.io/_uploads/rkin6V1I3.png) # HTTP 34 - Escape data in XML ###### tags: `Web` ![](https://hackmd.io/_uploads/ry83AHk8n.png) ## What is the XML format? XML (Extensible Markup Language) is a markup language used to definde and struture data in a human-readable and machine-readable format. It consists of elements, attributes, and text content enclosed int tags, which describe the struture and content of the data. ## Why is XML better than HTML? XML is better than HTML in certain contexts because it is more flexible and extensible. XML allows for creating custom tags and using namespaces to define unique elements and attributes, while HTML has a fixed set of tags. Additionally, XML can be used for a wide range of applications beyond web development, such as data exchange, configuration files, and document markup. ## How do you escape data in XML? To escape data in XML, you must replace any special characters with their corressponding escape sequences. While the single quote (') character in XML can only be escaped with ', it can be escaped with either ' or ' in HTML. In addition, a few additional escape sequences used in HTML but not in XML include those for non-breaking spaces. Online tool: https://www.lambdatest.com/free-online-tools/xml-escape With Curl: ![](https://hackmd.io/_uploads/HJb_ZUJL2.png) With Python: ![](https://hackmd.io/_uploads/SkIpWUJLh.png) # HTTP 35 ###### tags: `Web` ![](https://hackmd.io/_uploads/SkJYP6rUn.png) ![](https://hackmd.io/_uploads/HJAAv6H83.png) # HTTP 36 ###### tags: `Web` ![](https://hackmd.io/_uploads/rkfBOTB83.png) ![](https://hackmd.io/_uploads/SJvCdpSL3.png) # HTTP 37 ###### tags: `Web` ![](https://hackmd.io/_uploads/Hk9ZqTSL3.png) ![](https://hackmd.io/_uploads/SypdiaSIh.png) # HTTP 38 ###### tags: `Web` ![](https://hackmd.io/_uploads/ryjqxRS8n.png) ![](https://hackmd.io/_uploads/ryS10pBI2.png) **NOTE:** Các key và value trong JSON phải được bỏ trong dấu nháy kép "" # HTTP 39 ###### tags: `Web` ![](https://hackmd.io/_uploads/B1YwEAr82.png) In this challenge, I'll send the data in JSON format like this: ```py {"key":"please\""} ``` ![](https://hackmd.io/_uploads/H1Tp80rUn.png) # HTTP 40 ###### tags: `Web` ![](https://hackmd.io/_uploads/SyWZKASIh.png) # What is YAML? YAML stands for YAML Ain't Markup Language, but it originally stood for Yet Another Markup Language. YAML is a human-readable data seriallization language, just like XML, JSON. **Serialization** is a process where one application or service that has different data structures and is written in a different set of technologies can transger data to another application using a standard format. In other words, seriallization is about translating, converting, and wrapping up a data structure in another format. The data in the new format can be stored in a file or transmitted to another application or service over a network YAML is a widely used format for writing configuration files for different DevOps tools, programs and applications ![](https://hackmd.io/_uploads/HyX_2CBI2.png) # HTTP 41 ###### tags: `Web` ![](https://hackmd.io/_uploads/rJ__QJvUn.png) ![](https://hackmd.io/_uploads/r1wUikDIh.png) # HTTP 42 ###### tags: `Web` ![](https://hackmd.io/_uploads/Sy5knJv83.png) ## What is basic authentication? Basic authentication is a very simple authentication scheme that is built into the HTTP protocol. The client sends HTTP requests with the Authorization header that contains the `Basic` word followed by a space and a based64-encoded `username:password` strings. For example, a header containing the demo/p@55s0rd credentials would be encoded as: ``` Authorization: Basic ZGVtbzpwQDU1dzByZA== ``` ## Sending Curl Request with Basic Authentication Credentials To send basic auth credentuaks with Curl, use the "-u login: password" command-line option. Curl automatically converts the login:password pair into a Base64-encoded string and adds the "Authorization: Basic [token]" header to the request ![](https://hackmd.io/_uploads/BJpAQlv8n.png) # HTTP 43 ###### tags: `Web` ![](https://hackmd.io/_uploads/S1h34gDL3.png) ## What is WebSocket ?? HTTP and WebSocket both are communication protocols used in clien-server communication. **HTTP Protocol:** HTTP is unidirectional where the client sends the request and the server sends the response. Let's take an example when a user sends a request to the server - This request goes in the form of HTTP or HTTPS - After receiving a request, server send the response to the client - Afer sending the response, the connection get closed - Each HTTP or HTTPS request establish the new connection to the server every time and after getting tthe response the connection gets terminated by itself ![](https://hackmd.io/_uploads/SyGj9mw8h.png) **WebSocket:** WebSocket is bidirectional, a full-duplex protocal that is used in the same scenario of client-server communication, unlike HTTP it starts from ws:// or wss://. It is a stateful protocol, which means the connection between client and server will keep alive until it is terminated by client or server. ![](https://hackmd.io/_uploads/ByFSh7wI2.png) When can a web socket be used: - Real-time web application: used a web socket to show the data at the client end, which continuously being sent by the backend server. In WebSocket, data is continuously pushed/transmitted into the same connection which is already open, that is why WebSocket is faster and improves the application performance. - Gaming application: In a Gaming application, you might focus on that, data is continuously received by the server, and without refreshing thu UI, it will take effect on the screen. UI gets automatically refreshed without even establishing the new connection, so it is very helpful in a Gaming application. - Chat application: Chat applications use WebSockets to establish the connection only once for exchange, publishing, and broadcasting the message among the subscribers. It reuses the same WebSocket connection, for sending and receiving the message and for one-to-one message transfer. When not to use a WebSocket: WebSocket can be used if we want any real-time updated or continuous streams of data that are being transmitted over the network. If we want to fetch old data, or want to get the data only once to process it with an application we should go with HTTP protocol, old data which is not required very frequently or fetched only once can be queried by the simple HTTP request, so in this scenario, it’s better not use WebSocket. ## Using Python ![](https://hackmd.io/_uploads/BJqpBLuUn.png) ## Using HTML and JavaScript ![](https://hackmd.io/_uploads/HyarcU_Ih.png)