# lambda docker

## Docker

- docker-compose.yaml
- Dockerfile
- message.py

* docker-compose.yaml

version: '3'

services:
  orderbook_docker:
    build: ./
    ports:
      - '80:80'
    #command: tail -f /dev/null #コンテナを立てっぱなしにできる
    networks:
      app_net:
        ipv4_address: 172.30.0.2

networks:
  app_net:
    driver: bridge
    ipam:
     driver: default
     config:
       - subnet: 172.30.0.0/24

* Dockerfile

ARG FUNCTION_DIR="/home/pythontest"

FROM ubuntu:20.04

ARG FUNCTION_DIR

COPY ./app.py /home/pythontest/app.py

RUN apt-get update -y
RUN apt-get install -y g++ make cmake unzip libcurl4-openssl-dev apt-utils
RUN apt-get install -y python3.8
RUN apt-get install -y python3-pip

RUN pip3 install requests pandas tqdm boto3 matplotlib

RUN pip3 install --target ${FUNCTION_DIR} awslambdaric

#COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}
WORKDIR ${FUNCTION_DIR}
# Command can be overwritten by providing a different command in the template directly.
ENTRYPOINT [ "python3", "-m", "awslambdaric" ]
CMD ["app.lambda_handler"]

* `app.py`(シークレットとかはHackMDに公開しないように消してる)

import requests
import pandas as pd
import traceback
import time
import sys
import datetime
import matplotlib.pyplot as plt
import io

webhook_url =  ""
class timeseries_lambda_nortify:    
    def chat_plot(self,ii):
        """
        discordに画像送信
        """
        while True:
            try:
                files_qiita = {
                    "favicon" : ( "favicon.png", ii),
                }
                headers      = {'Content-Type': 'application/json'}
                response     = requests.post(webhook_url, files = files_qiita)
                return
            except Exception as e:
                print(e)
                time.sleep(60)

    def ls_diff2(self):
        """
        #api呼び出し、先物close、index close、LS、Frを集める
        """
        retryt = 10
        while (retryt<=100):
            try:
                #まずはlsデータを取得
                logic_symbols = ["ETHUSDT","BTCUSDT"]
                li1 = []
                for symbol in logic_symbols:
                    lsapi = requests.get(f"https://api.bybit.com/v2/public/account-ratio?symbol={symbol}&period=1h&limit=50").json()       #lsデータの1時間足取得
                    #取引所サーバ側の問題で、1時間足のデータ更新が上手くいっていない場合があるから、念のため最短の5分足の最新データを使う
                    ls_df = pd.DataFrame(lsapi["result"])
                    li1.append(ls_df)
                        
                ls_df = pd.concat(li1)
                ls_df["buy_ratio"] = ls_df["buy_ratio"].apply(float)                            #ls比をfloat型に
                ls_df["timestamp"] = ls_df["timestamp"].apply(int)                              #timestampをint型に
                ls_df["date"] = ls_df["timestamp"].apply(lambda x: pd.to_datetime(x,unit="s"))  #timestampデータから日付型データ作成
                ls_df.sort_values(["symbol","timestamp"],inplace=True)                          #銘柄ごと、時刻ごとに並び替え
                ls_df.reset_index(drop=True,inplace=True)                                       #インデックス振り直し
                
                #次にfunding rate取得
                li1 = []
                for symbol in logic_symbols:
                    #ls_api = requests.get(f"https://api.bybit.com/v2/public/funding/prev-funding-rate?symbol={symbol.replace('USDT','USD')}").json()
                    ls_api = requests.get(f"https://api.bybit.com/derivatives/v3/public/funding/history-funding-rate?category=linear&limit=200&symbol={symbol}").json()
                    #li1.append(ls_api["result"])
                    li1 += ls_api["result"]["list"]
                
                fr_df = pd.DataFrame(li1)
                fr_df.rename(columns={"fundingRate":"fr","fundingRateTimestamp":"timestamp"},inplace=True)
                fr_df["timestamp"] = fr_df["timestamp"].apply(lambda x: int(int(x)/1000))
                fr_df["fr"] = fr_df["fr"].apply(float)
                ls_df = pd.merge(ls_df,fr_df,on=["timestamp","symbol"],how="outer") #ここでlsとfr結合しておく
                
                #次に現先価格データ取得
                li1,li2 = [],[]
                now_t = datetime.datetime.now()
                for symbol in logic_symbols:        
                    for i in range(1,5):                
                        startt = int(int(now_t.timestamp()) - 60*60*199*i)          #1時間足で30日前まで、最新~720個前のデータがほしいけど200個前までしか取れない、だからfromのパラメータを順々にずらしていく
                        pri_spot = requests.get(f"https://api.bybit.com/public/linear/index-price-kline?symbol={symbol}&interval=60&from={startt}&limit=200").json()
                        pri_future = requests.get(f"https://api.bybit.com/public/linear/kline?symbol={symbol}&interval=60&limit=200&from={startt}").json()
                        pri_spot = pd.DataFrame(pri_spot["result"])
                        pri_future = pd.DataFrame(pri_future["result"])
                        li1.append(pri_spot)
                        li2.append(pri_future)
                
                pri_spot = pd.concat(li1)       #現物価格全データ結合
                pri_future = pd.concat(li2)     #先物価格全データ結合
                pri_spot = pri_spot[["symbol","open_time","close"]].copy()      #必要なカラムだけ抽出
                pri_future = pri_future[["symbol","open_time","close"]].copy()
                
                pri_df = pd.merge(pri_future, pri_spot, on=["symbol", "open_time"], suffixes=['', '_s'])    #現物データと先物データを銘柄と時刻が一致しているもので結合、現物価格はclose_sという名前にする
                pri_df["date"] = pri_df["open_time"].apply(lambda x: pd.to_datetime(x,unit="s"))            #timestampを日付型に変更
                pri_df.drop_duplicates(subset=["date","symbol"],inplace=True,keep="last")                   #重複データ削除
                pri_df.sort_values(["symbol","open_time"],inplace=True)                                     #並び替え
                pri_df.reset_index(drop=True,inplace=True)                                                  #インデックス振り直し
                pri_df[["close","close_s"]] = pri_df[["close","close_s"]].applymap(float)                   #価格データをfloat型に変換
                pri_df["dev"] = (pri_df["close_s"]/pri_df["close"]-1)*100
                pri_df["m_dev"] = pri_df["dev"].rolling(720).mean()                                          #これで30日平均のデータになる
                pri_df = pd.merge(pri_df, ls_df, on=["symbol","date"])                               #現先価格データとlsデータの結合
                
                #注文量計算
                pri_df[["dev","m_dev","buy_ratio"]] = pri_df[["dev","m_dev","buy_ratio"]].applymap(lambda x: round(x*100,2))
                pri_df["fr"] = pri_df["fr"]*100
                return pri_df
            except Exception as e:
                traceback.print_exc()
                time.sleep(retryt)
                retryt *= 2
            sys.exit()

    def make_figure(self):
        """
        dfをもとに画像作成
        """
        
        pri_df = self.ls_diff2()

        #銘柄ごとに
        for sy, df in pri_df.groupby("symbol"):
            std_date = df["date"][ df["date"].apply(lambda x: x.hour==16) ].max()
            std_ls = df["buy_ratio"][df["date"]==std_date].iloc[0]
            min_ls, max_ls = df["buy_ratio"].min(), df["buy_ratio"].max()
            start_date, end_date = df["date"].min(), df["date"].max()

            
            df["fr"].fillna(method='ffill',inplace=True)
            fig = plt.figure(figsize=(8,8))
            ax = fig.add_subplot(211)                    
            ax.plot(df["date"],df["close"],label="price")
            ax.set_title(f"{sy}")
            ax.set_ylabel("price [USD]",fontsize=12)
            
            ax1 = ax.twinx()
            ax1.plot(df["date"],df["fr"],label="fr",color="black",linestyle="--")
            ax1.set_ylabel("fr [%]",fontsize=16)
            lines1, labels1 = ax.get_legend_handles_labels()
            lines2, labels2 = ax1.get_legend_handles_labels()
            ax1.legend(lines1+lines2, labels1+labels2, framealpha=1, facecolor="white", loc=(0,0))
            
            ax2 = fig.add_subplot(212)
            ax2.plot(df["date"],df["buy_ratio"],label="buy ratio",color="orange")
            ax2.hlines(y=std_ls, xmin=start_date, xmax=end_date, color="orange",linestyles="--")
            ax2.vlines(x=std_date, ymin=min_ls, ymax=std_ls, color="orange",linestyles="--")
            
            ax2.set_ylabel("buy ratio [%]",fontsize=16)
            ax2.set_xlabel("date",fontsize=12)

            ax3 = ax2.twinx()
            ax3.plot(df["date"],df["dev"], label="index/future",color="black")
            ax3.plot(df["date"],df["m_dev"], label="index/future 30day",color="red",linestyle="--")
            ax3.set_ylabel("price dev [%]",fontsize=16)
            
            lines3, labels3 = ax2.get_legend_handles_labels()
            lines4, labels4 = ax3.get_legend_handles_labels()
            ax3.legend(lines3+lines4, labels3+labels4, framealpha=1, facecolor="white", loc=(0,0))
        
            img_data = io.BytesIO()
            fig.savefig(img_data, format='png')
            plt.close(fig)
            files = img_data.getvalue()        
            self.chat_plot(files)           
        return

def lambda_handler(event, context):
    tn = timeseries_lambda_nortify()
    tn.make_figure()

## Dockerコマンド
Unix、WindosのGitbash前提の書き方

* `docker-compose up --build` `-d`つければバックグラウンド
* `docker ps -a` ID表示、
* `docker rm $(docker ps -aq)`　$()内を変数としてコンテナ一括削除
* `docker rmi $(docker images -aq)`　イメージ一括削除
* `docker exec -it 657 bash` 657は docke psとかで分かるIDの頭3桁
* `docker stop 657`　コンテナ停止

## Lambda 環境構築
* [AWS SAM](https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/serverless-sam-cli-install-windows.html)
* [これ参考に](https://www.farend.co.jp/blog/2022/01/aws-lambda-container-image/)
* インストール後、power shell で`sam --version`でインストール確認
* `sam init`、[これ](https://dev.classmethod.jp/articles/sam-init-enable-x-ray-tracing/)見た感じいったんX-Rayはよくわからんけど追加コストかかるらしいから無効
* `sam init`で権限がない～みたいなエラー。管理者権限で実行してもだめ。[これ](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=powershell#enable-long-paths-in-windows-10-version-1607-and-later)を参考に以下のコマンド打ったらいけた。

New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" `
-Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force

* `sam init`

PS C:\Database\WorkDocker\saminit> sam init

You can preselect a particular runtime or package type when using the `sam init` experience.
Call `sam init --help` to learn more.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Multi-step workflow
        3 - Serverless API
        4 - Scheduled task
        5 - Standalone function
        6 - Data processing
        7 - Infrastructure event management
        8 - Serverless Connector Hello World Example
        9 - Multi-step workflow with Connectors
        10 - Lambda EFS example
        11 - Machine Learning
Template: 1

Use the most popular runtime and package type? (Python and zip) [y/N]: N

Which runtime would you like to use?
        1 - dotnet6
        2 - dotnet5.0
        3 - dotnetcore3.1
        4 - go1.x
        5 - graalvm.java11 (provided.al2)
        6 - graalvm.java17 (provided.al2)
        7 - java11
        8 - java8.al2
        9 - java8
        10 - nodejs16.x
        11 - nodejs14.x
        12 - nodejs12.x
        13 - python3.9
        14 - python3.8
        15 - python3.7
        16 - ruby2.7
        17 - rust (provided.al2)
Runtime: 14

What package type would you like to use?
        1 - Zip
        2 - Image
Package type: 2

Based on your selections, the only dependency manager available is pip.
We will proceed copying the template using pip.

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: N

Project name [sam-app]: orderbook2

[1](https://dev.classmethod.jp/articles/lambda-container-pkgfmt-with-sam-cli/)

[pypl](https://pypi.org/project/awslambdaric/)
[aws資格情報](https://docs.aws.amazon.com/ja_jp/cli/latest/userguide/cli-configure-envvars.html)Y

* まず、管理者実行のpowershellで以下の環境変数設定

$Env:AWS_ACCESS_KEY_ID=""
$Env:AWS_SECRET_ACCESS_KEY=""
$Env:AWS_DEFAULT_REGION="ap-northeast-1"

* `sam build`
* `sam local invoke` これでローカル稼働テスト
* `sam deploy --guided` これで対話式ガイドでデプロイ

終わり