# prometheus remote write client

# Introduction

TL;DR: 要用google的protobuf，並且用snappy壓縮，再加上sigv4 auth傳過去

### Reference

obsolete example

[Example Python Prometheus remote write client](https://gist.github.com/robskillington/fb82ee5c737b79a3bc891df3dce7a9aa)

protobuf教學

[](https://ithelp.ithome.com.tw/articles/10250131)

# 前置作業

### 安裝protobuf compiler

官方下載

[https://github.com/protocolbuffers/protobuf/releases](https://github.com/protocolbuffers/protobuf/releases)

或是mac直接 `brew install protobuf`

[https://blog.keniver.com/2020/03/install-protobuf-compiler-on-macos/](https://blog.keniver.com/2020/03/install-protobuf-compiler-on-macos/)

### Python Requirement

- [google snappy](http://google.github.io/snappy/) compresser

`pip install python-snappy`

[python-snappy](https://pypi.org/project/python-snappy/)

- Protobuf Runtime

`pip install protobuf`

[GitHub - protocolbuffers/protobuf: Protocol Buffers - Google's data interchange format](https://github.com/protocolbuffers/protobuf#protobuf-runtime-installation)

[protobuf/python at main · protocolbuffers/protobuf](https://github.com/protocolbuffers/protobuf/tree/main/python)

- google

`pip install google`

總之會用到

- aws sigv4 auth for python

`pip install requests-auth-aws-sigv4`

[requests-auth-aws-sigv4](https://pypi.org/project/requests-auth-aws-sigv4/)

先確保sigv4能通，可用boto3預設credential or STS token

import requests
from requests_auth_aws_sigv4 import AWSSigV4

r = requests.request('POST', 'https://sts.us-west-2.amazonaws.com', 
    data=dict(Version='2011-06-15', Action='GetCallerIdentity'), 
    auth=AWSSigV4('sts'))
print(r.text)

### 下載prometheus remote proto

prometheus自訂的remote proto在此 (latest)

[https://github.com/prometheus/prometheus/tree/main/prompb](https://github.com/prometheus/prometheus/tree/main/prompb)

先下載 [remote.proto](https://github.com/prometheus/prometheus/blob/main/prompb/remote.proto)和[types.proto](https://github.com/prometheus/prometheus/blob/main/prompb/types.proto)

還需要[gogo.proto](https://github.com/gogo/protobuf/blob/master/gogoproto/gogo.proto)

# Development

### compile proto

- 把remote.proto跟types.proto放在目錄下
- create folder `gogoproto` 把gogo.proto放進去
- 執行

# 產生gogoproto/gogo_pb2.py
cd gogoproto
protoc --python_out=. gogo.proto
# 產生 types_pb2.py
protoc -I=./ -I=./gogoproto --python_out=./ gogoproto/gogo.proto ./types.proto
# 產生 remote_pb2.py
protoc -I=. ./remote.proto

### python code

from datetime import datetime
from remote_pb2 import (
    WriteRequest
)
from types_pb2 import(
    TimeSeries,
    Label,
    Labels,
    Sample
)

import calendar
# import logging
import requests
import snappy
from requests_auth_aws_sigv4 import AWSSigV4

def dt2ts(dt):
    """Converts a datetime object to UTC timestamp
    naive datetime will be considered UTC.
    """
    return calendar.timegm(dt.utctimetuple())

def write():
    write_request = WriteRequest()

    series = write_request.timeseries.add()

    # name label always required
    label = series.labels.add()
    label.name = "__name__"
    label.value = "metric_name"
    
    # as many labels you like
    label = series.labels.add()
    label.name = "ssl_cipher"
    label.value = "some_value"

    sample = series.samples.add()
    sample.value = 42 # your count?
    sample.timestamp = dt2ts(datetime.utcnow()) * 1000

    uncompressed = write_request.SerializeToString()
    compressed = snappy.compress(uncompressed)

    url = "amp remote_write endpoint"
    headers = {
        "Content-Encoding": "snappy",
        "Content-Type": "application/x-protobuf",
        "X-Prometheus-Remote-Write-Version": "0.1.0",
        "User-Agent": "metrics-worker"
    }
    try:
        response = requests.post(url, headers=headers, data=compressed,auth=AWSSigV4('aps'))
        # print(response.text)
    except exception as e:
        print(e)

write()