---
# System prepended metadata

title: 'Progress Report 1 (Feb 8, 2022)'

---

# Progress Report 1 (Feb 8, 2022)




## Team Members:
- Chiao Lu
    - UID: 204848946
    - email: josephlu85@engineering.ucla.edu
- Boyan Ding
    - UID: 205331471
    - email: dboyan@cs.ucla.edu
- Jiaxin Ge
    - UID: 905526210
    - email: jig099@g.ucla.edu
- Yuyue Wang
    - UID: 205728527
    - email: yuyue@cs.ucla.edu

## Project Title: Insighter
Our team's Git Repo can be found [here](https://github.com/erdnase1902/restler-fuzzer/tree/main).
## Anticipated Finding
<!-- From Microsoft [RESTler Git Repo](https://github.com/microsoft/restler-fuzzer):
> RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.

One critical limitation of RESTler is that the backend (e.g. Express.js or Flask) is treated as a blackbox, and error can only be observed with HTTP status code (e.g. 500 for Internal Server Error). As a result, 
1. Warnings triggered in the backend is not known to  RESTler.
2. Error type (e.g. `DivisionByZeroError` or `FileNotFoundError`) is not known to RESTler as it only receives `500: Internal Server Error`.
3. If no response received, why? Is the backend still running? -->

We would like to investigate the possibility providing an entrypoint into the backend for [RESTler](https://github.com/microsoft/restler-fuzzer), so that RESTler can open the blackbox of the REST API server being tested (like what AFL does).

## Example of Subject Program
The example subject program we found is the demo server in [Restler's repo](https://github.com/erdnase1902/restler-fuzzer/tree/main/demo_server). We are able to modify and run this server with Restler.
```bash=
vagrant@ubuntu:~/restler-fuzzer/demo_server$ python3 demo_server/app.py 
2022-02-08 23:38:36,084 - __main__ - INFO - >>>>> Starting development server at http://localhost:8888/api/ <<<<<
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
2022-02-08 23:38:36,095 - werkzeug - INFO -  * Running on http://localhost:8888/ (Press CTRL+C to quit)
```
### Swagger UI
![Swagger Screenshot](https://i.imgur.com/p0VoQxS.png)


## Baseline Tool: Restler
We were able to build Restler on our machines and run it against the [demo server](https://github.com/erdnase1902/restler-fuzzer/tree/main/demo_server).

### Building Restler
```bash=
vagrant@ubuntu:~/restler-fuzzer$ python ./build-restler.py --dest_dir /home/vagrant/restler-bin/
Generating a new RESTler binary drop...
Publishing dotnet core apps...
...
Testing compilation of Python files...
Copying all python files...
Removing compilation build directory...
Copying all python files...
vagrant@ubuntu:~/restler-fuzzer$ echo $?
0
```
### Running Restler
#### Compile swagger spec
```bash=
vagrant@ubuntu:~/restler-test$ ../restler-bin/restler/Restler compile --api_spec ../restler-fuzzer/demo_server/swagger.json 
Starting task Compile...
Task Compile succeeded.
Collecting logs...
```
#### Test
```bash=
vagrant@ubuntu:~/restler-test$ ../restler-bin/restler/Restler test --grammar_file $HOME/restler-test/Compile/grammar.py --dictionary_file $HOME/restler-test/Compile/dict.json --settings $HOME/restler-test/Compile/engine_settings.json --no_ssl
Starting task Test...
Using python: 'python3' (Python 3.8.10)
Request coverage (successful / total): 6 / 6
No bugs were found.
Task Test succeeded.
Collecting logs...
```
#### Fuzz-Lean
```bash=
vagrant@ubuntu:~/restler-test$ ../restler-bin/restler/Restler fuzz-lean --grammar_file $HOME/restler-test/Compile/grammar.py --dictionary_file $HOME/restler-test/Compile/dict.json --settings $HOME/restler-test/Compile/engine_settings.json --no_ssl
Starting task FuzzLean...
Using python: 'python3' (Python 3.8.10)
Request coverage (successful / total): 6 / 6
Bugs were found!
Bug buckets:

InvalidDynamicObjectChecker_20x: 2
InvalidDynamicObjectChecker_500: 1
PayloadBodyChecker_500: 1
Task FuzzLean succeeded.
Collecting logs...
```
#### Fuzz
```bash=
vagrant@ubuntu:~/restler-test$ ../restler-bin/restler/Restler fuzz --grammar_file $HOME/restler-test/Compile/grammar.py --dictionary_file $HOME/restler-test/Compile/dict.json --settings $HOME/restler-test/Compil
e/engine_settings.json --no_ssl --time_budget 1
Starting task Fuzz...
Using python: 'python3' (Python 3.8.10)
Request coverage (successful / total): 6 / 6
Bugs were found!
Bug buckets:

InvalidDynamicObjectChecker_20x: 2
InvalidDynamicObjectChecker_500: 1
PayloadBodyChecker_500: 1
Task Fuzz succeeded.
Collecting logs...
```
## Related Work
### Restler
Restler is the first statful REST API fuzzer that analyzes the API specification of a cloud service and generates sequences of requests that automatically test the service through its API. 

It was first proposed by [Atlidakis et al.](https://patricegodefroid.github.io/public_psfiles/icse2019.pdf) (ICSE 2019) which emphasized two critical test sequences generation rules: inferring producer-consumer dependencies among request types to generate valid test inputs (e.g. request A must be sent before request B) and analyzing dynamic feedback from previous test responses to prune the large search space of possible request sequences. 

Restler successfully found 28 bugs in GitLan and several bugs in each of  the Azure and Office365 cloud services; However, it has several limitations as well. First, it can only find bugs defined as unexpected HTTP status code. Secondly, it could only discover vulnerabilitiles that are visible through HTTP status code. In order to improve Restler, a few more researches were conducted.

In 2020, Restler was extended with active property checkers that automatically test and detect violations of security rules by [Atlidakis et al.](https://patricegodefroid.github.io/public_psfiles/icst2020.pdf) (ICST 2020). In this paper, Restler was enabled to catch bugs in the cloud service that allow attackers to violate security rules including Use-after-free rule, Resource-leak rule, Resource-hierarchy rule and User-namespace rule. 

[Godefroid et al.](https://patricegodefroid.github.io/public_psfiles/fse2020.pdf) (FSE 2020) futher extended Restler to find data-processing bugs in cloud service. They enhenced Resterler by intelligently generating data payloads so that it can reach and test deeper service states. 

[Godefroid et al.](https://patricegodefroid.github.io/public_psfiles/issta2020.pdf) (ISSTA 2020) also employed differential testing via Restler in order to prevent breaking changes of APIs. That is, they compared the behavior of different versions on the same inputs against each other, and find regressions in the observed differences. 



<!-- ## Benchmarks and Target Subject Applications
### Subject Programs
- Handcrafted simple Flask server

### Benchmark
Show that our improved method can also capture issues not visible to original RESTler
- warnings
- details of the error (e.g. `500` return code is caused by `DynamicTypeError`)

## Potential Work
RESTler currently uses BFS-Fast, can we try different searching algorithm?
- A*
- Genetic algorithm
- ... -->