# Meeting minutes (2017) / Memory Allocation
返回 [Document Index](https://hackmd.io/s/HJqefxKXg)
## 2017-02-12
- allocator with fuzz testing:
https://github.com/ldotrg/allocator
Please install ==clang== envionment first
```
# Install git and get this tutorial
sudo apt-get --yes install git
git clone https://github.com/google/fuzzer-test-suite.git
./fuzzer-test-suite/tutorial/install-deps.sh # Get deps
./fuzzer-test-suite/tutorial/install-clang.sh # Get fresh clang binaries
# Get libFuzzer sources and build it
svn co http://llvm.org/svn/llvm-project/llvm/trunk/lib/Fuzzer
Fuzzer/build.sh
```
Copy the libFuzzer.a to allocater
```
make test_fuzzer_static
./test_fuzzer_static
```
- [allocater](https://github.com/ldotrg/allocator/blob/master/test_fuzzer.cc)有memory leak
- memory leak compile option manual: [AddressSanitizerLeakSanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer)
- 若是已知的問題,可以寫個設定檔suppr.txt, 這次要忽略的是realloc 40 bytes 的memory leak
```shell=
$ cat suppr.txt
# This is a known leak.
leak:realloc
```
- Compile and [linking flag](https://github.com/google/sanitizers/wiki/AddressSanitizerFlags)
```cmake=
FUZZ_CXXFLAGS := -g -fsanitize=address -fsanitize-coverage=trace-pc-guard
FUZZ_CXXFLAGS += -fsanitize=leak
```
- 執行前輸入以下指令
```shell=
$ ASAN_OPTIONS=detect_leaks=1 LSAN_OPTIONS=suppressions=suppr.txt ./test_fuzzer
```
效果如下
memory leak Log
```shell=
ldotrg@b50:~/Workspace/allocator$ ./test_fuzzer
INFO: Seed: 2934944203
INFO: Loaded 1 modules (13 guards): [0x74be10, 0x74be44),
INFO: -max_len is not provided, using 64
INFO: A corpus is not provided, starting from an empty corpus
#0 READ units: 1
=================================================================
==5113==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 40 byte(s) in 1 object(s) allocated from:
#0 0x4c311e in realloc (/home/ldotrg/Workspace/allocator/test_fuzzer+0x4c311e)
#1 0x4f2603 in FuzzMe(unsigned char const*, unsigned long) /home/ldotrg/Workspace/allocator/test_fuzzer.cc:15:20
#2 0x4f2d19 in LLVMFuzzerTestOneInput /home/ldotrg/Workspace/allocator/test_fuzzer.cc:65:3
#3 0x4fc33c in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /home/ldotrg/Workspace/Fuzzer/./FuzzerLoop.cpp:550:13
#4 0x4fc564 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long) /home/ldotrg/Workspace/Fuzzer/./FuzzerLoop.cpp:501:3
#5 0x4fc141 in fuzzer::Fuzzer::RunOne(std::vector<unsigned char, std::allocator<unsigned char> > const&) /home/ldotrg/Workspace/Fuzzer/./FuzzerInternal.h:118:41
#6 0x4fc141 in fuzzer::Fuzzer::ShuffleAndMinimize(std::vector<std::vector<unsigned char, std::allocator<unsigned char> >, std::allocator<std::vector<unsigned char, std::allocator<unsigned char> > > >*) /home/ldotrg/Workspace/Fuzzer/./FuzzerLoop.cpp:480
#7 0x4f51f8 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /home/ldotrg/Workspace/Fuzzer/./FuzzerDriver.cpp:565:6
#8 0x4f2d90 in main /home/ldotrg/Workspace/Fuzzer/./FuzzerMain.cpp:20:10
#9 0x7fcfecd6882f in __libc_start_main /build/glibc-GKVZIf/glibc-2.23/csu/../csu/libc-start.c:291
SUMMARY: AddressSanitizer: 40 byte(s) leaked in 1 allocation(s).
INFO: a leak has been found in the initial corpus.
INFO: to ignore leaks on libFuzzer side use -detect_leaks=0.
MS: 0 ; base unit: 0000000000000000000000000000000000000000
0xa,
\x0a
artifact_prefix='./'; Test unit written to ./leak-adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
Base64: Cg==
```
## 2017-01-14
* [Formal verification of Sleepable Read-copy-update using CBMC](https://github.com/ldr709/verify_srcu)
* Liang Jung 的初步分析
- cbmc: 無法檢查所有 state machine
- gcovr: 只能看code有跑到哪邊，coverage 完全仰賴執行
- spin： 需要將既有的 C code 轉換成 PROMELA
* allocator 實驗
* https://github.com/thestinger/allocator
* [2016年的實驗](https://hackmd.io/s/SJzgJj4-x) 提升sting-alloc huge allocation 的 huge.c使用量
* (2016-12-24) 上禮拜測試時調整chuck 大小 4096kb -> 4096 MB or 更大或是增加shirk 跟expand量雖然拉長test_huge跑的時間，卻無法有效提升huge.c的程式使用量。
* 後來使用–html --html-details可看到各.c檔中function的使用狀況，可發現huge_move_expand這個funcion未使用到，導致整體huge.c只用到65%左右
* [libFuzzer](http://llvm.org/docs/LibFuzzer.html)
* [ThreadSanitizerCppManual](https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual)
* TODO
- [x] 整合 libFuzzer 到 allocator (東儒)
> 先從此使用教學開始：[fuzzer-test-suit](https://github.com/google/fuzzer-test-suite/blob/master/tutorial/libFuzzerTutorial.md)
:::danger
After install clang, when you use `apt-get update`, it will appear apt-get: /usr/local/lib/libstdc++.so.6: version `GLIBCXX_3.4.20' not found message.
Please do not reboot, use following command to fix this.
```shell
sudo cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/local/lib/libstdc++.so.6
```
:::
- [ ] formal verification 目標重新設定 (jserv)
* [BPF and formal verifycation](https://www.sccs.swarthmore.edu/users/16/mmcconv1/pl-reflection.html)
## 2017-01-07
* 提升sting-alloc huge allocation 的 huge.c使用量
測試發現當CHUNK_SIZE = 16MB且test_huge.c 40行 expand量增加到(CHUNK_SIZE * 128k )則可以提升huge.c至82%左右，但是test_huge就跑不完了。
須調整test_huge測試方式。
https://github.com/whosyourdadd/allocator/tree/test0107
最終調整 huge.c cover range 65%=>82%=>90%
可以到90%
```clike=
#define CHUNK_SIZE ((size_t)4096 * 4096)
In test_huge.c
p = realloc(p, CHUNK_SIZE * 123456);
```
產生報告時需要加入 --html --html-details
```
gcovr -r . --html --html-details -o example.html
```
可以鎖定到程式 那一個function 那一行code有沒有執行
從報告發現 ```huge_move_expand```此函數都沒被執行
因此只要製造讓 huge_realloc 能採進去huge_move_expand的方式就行
```clike=
void *huge_realloc(struct thread_cache *cache, void *ptr, size_t old_size, size_t new_real_size) {
if (new_real_size > old_size) {
if (!huge_no_move_expand(ptr, old_size, new_real_size)) {
return ptr;
}
return huge_move_expand(cache, ptr, old_size, new_real_size);
} else if (new_real_size < old_size) {
huge_no_move_shrink(ptr, old_size, new_real_size);
}
return ptr;
}
```
這邊採進去的方式請阿湯跟體積 幫忙解釋
huge_no_move_expand
huge_move_expand
這兩個的差異
### google test in linux
http://www.cnblogs.com/xuning/p/3760378.html
參考上面的網址整理簡單的重點
1. 下載[gtest](https://github.com/google/googletest)
2. 可以刪掉一些不必要的資料夾(可有可無的動作)剩下以下這四個即可
3.編譯與執行
```shell=
cd make
make
./sample1_unittest
```
產出結果如下
4. 自行修改makefile讓自己所使用的程式到用gtest的UI而測試的程式語法大略如下
```clike=
#include "sqrt.h"
#include "gtest/gtest.h"
TEST(SQRTTest,Zero){
EXPECT_EQ(0,sqrt(0));
}
TEST(SQRTTest,Positive){
EXPECT_EQ(100,sqrt(10000));
EXPECT_EQ(1000,sqrt(1000009));
EXPECT_EQ(99,sqrt(98100));
}
TEST(SQRTTest,Negative){
int i = -1;
EXPECT_EQ(0,sqrt(i));
}
```
### google test + sting-alloc
https://github.com/whosyourdadd/gtest_Alloc
* libFuzzer: http://llvm.org/docs/LibFuzzer.html
Sign in with Wallet
Connect another wallet