---
# System prepended metadata

title: 卒研

---

# 卒研
> 方針: go-kawakami.oit.ac.jp のコピーを作り、それをベースに改良する。

## 作るものと欲しい機能
順次追加するかも
1. ライブ画像配信用に、別ポートで立ち上げるwebサーバ
    - 想定どおりの要求の場合はAccessLogを書かない
2. 既存のApacheを置き換える、WAF(ウェブアプリケーションファイアウォール)機能つきwebサーバ
    -  URLをチェックし、怪しい要求が来たら対処する
        - rejectする
        - dropする
        - やたら大きなファイルを送りつけて仕返しする(笑)

---


### 1/14報告分
- ホワイトリスト自動作成UI
    - ドキュメントルート以下を全探索してホワイトリストに登録する
    - shellスクリプトでもなんでも可
- リスト外ファイル並びにクエリのやたら長いアクセスをブロック
    - 短時間で複数回のアクセスがあった場合は一定時間そのIPアドレスに対して何も返さない
- Python ベンチマーク用クライアント作成
    - HTTPのkeepaliveに対応したもの
    - できれば、Websocketに対応したものも合わせて
    - 結果次第でwebsocketかHTTPか決定

### 1/22報告分
- 実験し、うまく行かなかった場合その理由を考察することが大事
    - ファイルオープンでの処理負荷に着目し、その部分を改良するプログラムを制作した
    - しかし処理負荷は軽減できなかった
    - このことより、負荷はファイルオープンによって発生しているものではなく、HTTPのリクエスト処理に負荷がかかっていると思われる
    - Websocketを使用し、HTTPのリクエスト・レスポンスの手順を廃することで負荷の軽減を達成することができた(書ければ)

## 2週間目標

### 10/29 or 31報告予定分
- [x] go-kawakami.oit.ac.jp のコピーサーバーを作る
- [x] Node.jsで、最低限要求に応答できるwebサーバを作ってみる。

#### 追加でやったこと
- [x] Node.jsで、特定のファイルを返すwebサーバを作る

### 11/25〜29報告予定分
- [x] エラーが発生しても落ちないためを行の追加 (furukawa, ikegami)
    - go-kawakamiのnoderun以下にあるファイルを参照
- [x] 返すファイルを変更しやすいようにプログラム書き換え (furukawa, ikegami)
    - 設定ファイルを作るもしくは、行数が短ければプログラムコードの上の方
- [ ] Webサーバ用ベンチマークテストツールを探してみて実行 (furukawa, ikegami)
    - GoogleのV8エンジンは素晴らしいらしい
- [ ] Node.jsのガベージコレクションの頻度などについて調査 (ikegami, furukawa)
    - ガベージコレクション実行中はプログラムが長時間停止してしまうらしい

### 12/9〜13報告予定分
- [ ] Webサーバ用ベンチマークテストツールを探してみて実行(続き)
    - **a1秒間に500アクセス以上できるかどうか**
- [ ] HTTPS対応(Let's Encrypt)
    - HTTPと混在するとchromeが弾くようになるため
## memo
```
150.89.255.21 - - [03/Feb/2020:23:41:59 +0900] "GET /img/usage.jpg HTTP/1.1" 200 57627 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:41:59 +0900] "GET /img/usage_en.jpg HTTP/1.1" 200 55206 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:41:59 +0900] "GET /img/howitworks_en.jpg HTTP/1.1" 200 69505 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:41:59 +0900] "GET /img/a1.jpg HTTP/1.1" 200 89552 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:42:00 +0900] "POST /msg/today-msg.txt HTTP/1.1" 200 340 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:42:00 +0900] "GET /ramdsk/camera.jpg?page=1580740912233 HTTP/1.1" 200 43749 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:42:00 +0900] "POST /msg/top-msg.txt HTTP/1.1" 200 281 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:42:00 +0900] "POST /msg/today-msg.txt HTTP/1.1" 200 340 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:42:00 +0900] "GET /ramdsk/hourly.json?_=1580740910958 HTTP/1.1" 404 491 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
150.89.255.21 - - [03/Feb/2020:23:42:00 +0900] "GET /ramdsk/daily.json?_=1580740910959 HTTP/1.1" 404 491 "http://150.89.255.61/index.html?ja" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"

```

```
196.52.84.16 - - [08/Jan/2020:06:09:50 +0900] "gamagaeru.com" ":443" "GET /admin/ HTTP/1.1" 301 67 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:09:52 +0900] "www.gamagaeru.com" ":443" "GET /admin/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:09:55 +0900] "gamagaeru.com" ":443" "GET /shop/admin/ HTTP/1.1" 301 72 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:09:56 +0900] "www.gamagaeru.com" ":443" "GET /shop/admin/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:09:57 +0900] "gamagaeru.com" ":443" "GET /store/admin/ HTTP/1.1" 301 73 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:09:58 +0900] "www.gamagaeru.com" ":443" "GET /store/admin/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:00 +0900] "gamagaeru.com" ":443" "GET /magento/admin/ HTTP/1.1" 301 75 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:01 +0900] "www.gamagaeru.com" ":443" "GET /magento/admin/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:03 +0900] "gamagaeru.com" ":443" "GET /magento2/admin/ HTTP/1.1" 301 76 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:04 +0900] "www.gamagaeru.com" ":443" "GET /magento2/admin/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:05 +0900] "gamagaeru.com" ":443" "GET /pub/errors/503.php HTTP/1.1" 301 79 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:06 +0900] "www.gamagaeru.com" ":443" "GET /pub/errors/503.php HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:07 +0900] "gamagaeru.com" ":443" "GET /shop/pub/errors/503.php HTTP/1.1" 301 84 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:08 +0900] "www.gamagaeru.com" ":443" "GET /shop/pub/errors/503.php HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:09 +0900] "gamagaeru.com" ":443" "GET /store/pub/errors/503.php HTTP/1.1" 301 85 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:10 +0900] "www.gamagaeru.com" ":443" "GET /store/pub/errors/503.php HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:11 +0900] "gamagaeru.com" ":443" "GET /magento/pub/errors/503.php HTTP/1.1" 301 87 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:12 +0900] "www.gamagaeru.com" ":443" "GET /magento/pub/errors/503.php HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:13 +0900] "gamagaeru.com" ":443" "GET /magento2/pub/errors/503.php HTTP/1.1" 301 88 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:14 +0900] "www.gamagaeru.com" ":443" "GET /magento2/pub/errors/503.php HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:15 +0900] "gamagaeru.com" ":443" "GET / HTTP/1.1" 301 61 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:16 +0900] "www.gamagaeru.com" ":443" "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:19 +0900] "gamagaeru.com" ":443" "GET /shop/ HTTP/1.1" 301 66 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:20 +0900] "www.gamagaeru.com" ":443" "GET /shop/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:21 +0900] "gamagaeru.com" ":443" "GET /store/ HTTP/1.1" 301 67 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:22 +0900] "www.gamagaeru.com" ":443" "GET /store/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:23 +0900] "gamagaeru.com" ":443" "GET /magento/ HTTP/1.1" 301 69 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:24 +0900] "www.gamagaeru.com" ":443" "GET /magento/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:25 +0900] "gamagaeru.com" ":443" "GET /magento2/ HTTP/1.1" 301 70 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
196.52.84.16 - - [08/Jan/2020:06:10:27 +0900] "www.gamagaeru.com" ":443" "GET /magento2/ HTTP/1.1" 404 196 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"
```
```jsonld=
[
  { 
    addr: '::ffff:192.0.2.27',
    unixtime: 1580718039,
    times: 1
  },
  {
    addr: '2001:db8:dead:beef:573:196:1791:1605',
    unixtime: 1580717973,
    times: 7
  }
]

```
```jsonld=
[
  { 
    addr: '::ffff:192.0.2.27',
    unixtime: 1580718150
  },
  {
    addr: '2001:db8:dead:beef:573:196:1791:1605',
    unixtime: 1580718165
  }
]
```
```javascript=
   // 一定間隔で画像データ取得
   setInterval(function(){ //指定した間隔(ミリ秒)で無限ループ
      fs.readFile(config.path, function(err,data){
         imgdata = data; // 取得した画像データをメモリで保持
      });
   },200); //ミリ秒

   // リクエスト処理
   https.createServer(options, function(req, res){
      res.writeHead(200, {'Content-Type' : 'image/jpeg'});
      res.end(imgdata); // 保持している画像データで応答
   }).listen(config.port);
```


```
process.on('uncaughtException', function(err) { //avoid exitting
  var etime = moment().format("YYYY-MM-DD HH:mm:ss ");
  fs.appendFile('./log/error.log', etime + '\r\n' + err + '\r\n');
});
```

```
::ffff:192.0.2.21 - [2020/02/04:03:27:29(9H)] "GET /170417/Dist/images/chap0501.png HTTP/1.1" 404 - "https://www61.example.com:3443/rule.html?hoge=fuga" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
2001:db8:dead:beef:573:196:1791:1605 user [2020/02/04:03:30:42(9H)] "GET /hoge.txt?p=fug HTTP/1.1" 200 7 "-" "curl/7.58.0"

[2020/02/04:04:04:31(9H)] 2001:db8:dead:beef:573:196:1791:1605 Add
[2020/02/04:04:04:33(9H)] ::ffff:192.0.2.21 Blocked
```
## apachebench
### apache
```
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www61.gamagaeru.com (be patient)


Server Software:        Apache/2.4.41
Server Hostname:        www61.gamagaeru.com
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
TLS Server Name:        www61.gamagaeru.com

Document Path:          /ramdsk/camera.jpg
Document Length:        46158 bytes

Concurrency Level:      100
Time taken for tests:   2529.370 seconds
Complete requests:      100000
Failed requests:        50000
   (Connect: 0, Receive: 0, Length: 50000, Exceptions: 0)
Keep-Alive requests:    50000
Total transferred:      2324412450 bytes
HTML transferred:       2309948000 bytes
Requests per second:    39.54 [#/sec] (mean)
Time per request:       2529.370 [ms] (mean)
Time per request:       25.294 [ms] (mean, across all concurrent requests)
Transfer rate:          897.43 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   23 184.0      8    7449
Processing:     0 2504 2503.9   5002    5074
Waiting:        0    1   2.7      1      65
Total:          0 2527 2533.2   5012   12460

Percentage of the requests served within a certain time (ms)
  50%   5012
  66%   5035
  75%   5042
  80%   5045
  90%   5053
  95%   5059
  98%   5067
  99%   5072
 100%  12460 (longest request)
```
### node
```
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www61.gamagaeru.com (be patient)


Server Software:        CosyLab
Server Hostname:        www61.gamagaeru.com
Server Port:            3100
SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES128-GCM-SHA256,2048,128
TLS Server Name:        www61.gamagaeru.com

Document Path:          /ramdsk/camera.jpg
Document Length:        46158 bytes

Concurrency Level:      100
Time taken for tests:   478.951 seconds
Complete requests:      100000
Failed requests:        0
Keep-Alive requests:    0
Total transferred:      4627600000 bytes
HTML transferred:       4615800000 bytes
Requests per second:    208.79 [#/sec] (mean)
Time per request:       478.951 [ms] (mean)
Time per request:       4.790 [ms] (mean, across all concurrent requests)
Transfer rate:          9435.49 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       46  462  20.1    456     630
Processing:     2   17   2.6     17      79
Waiting:        1    9   1.8      9      55
Total:         67  479  20.8    474     656

Percentage of the requests served within a certain time (ms)
  50%    474
  66%    478
  75%    483
  80%    486
  90%    503
  95%    521
  98%    539
  99%    557
 100%    656 (longest request)
```

```javascript=
IncomingMessage {
  (中略)
  httpVersionMajor: 1,
  httpVersionMinor: 1,
  httpVersion: '1.1',
  complete: false,
  headers: {
    host: '[2001:2f8:1024:b3f0:82ee:73ff:feb9:ca64]:3000',
    connection: 'keep-alive',
    pragma: 'no-cache',
    'cache-control': 'no-cache',
    dnt: '1',
    'upgrade-insecure-requests': '1',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36',
    accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    referer: 'http://[2001:2f8:1024:b3f0:82ee:73ff:feb9:ca64]:3000/',
    'accept-encoding': 'gzip, deflate',
    'accept-language': 'ja,en-US;q=0.9,en;q=0.8'
    cookie: '_ga=GA1.1.475259271.1580449298; _gid=GA1.1.156268885.1580449298'
  },
  (中略)
  _dumped: false
}
```