---
# System prepended metadata

title: OpenTelemetry
tags: [SRE]

---

# fastapi
https://signoz.io/blog/opentelemetry-fastapi/
# OpenTelemetry
*opentelemetry.io* 
>High-quality, ubiquitous, and portable telemetry to enable effective observability

# Tempo
>Tempo is the tracing puzzle of the Grafana observability portfolio.
Ref: [Trace — Log Correlation with Grafana Tempo](https://blog.devgenius.io/trace-log-correlation-with-grafana-tempo-311d499ce63a)

![image](https://hackmd.io/_uploads/rJSBcTw5C.png)

# Concept 
[Youtube:初探 OpenTelemetry 工具組：蒐集遙測數據的新標準](https://youtu.be/PT-Bjs6iCug?si=BuRkMRbe5vxW45sg) 

# Note of Video
>初探 OpenTelemetry 工具組：蒐集遙測數據的新標準

![VS--YouTube-OpenTelemetry-62’52”](https://hackmd.io/_uploads/ryg9kUBwiC.png)
(1)[video: Signoz Service Map](https://youtu.be/PT-Bjs6iCug?si=wRM3lUum5O_OcXjs&t=3772)

![VS--YouTube-OpenTelemetry-63’03”](https://hackmd.io/_uploads/SJz5jdviR.png)
(2)[video: Metrics](https://youtu.be/PT-Bjs6iCug?si=pE72mn8a8eXDj1j5&t=3782)

![image](https://hackmd.io/_uploads/HksDkFDoC.png)
(3)[video: Traces](https://youtu.be/PT-Bjs6iCug?si=nVmnbnedS-c7k4R_&t=3804)

**Drill Down**
(3-1) 
點入其中一個Trace ID
![image](https://hackmd.io/_uploads/Hk5lgFvoC.png)
(3-2)
點入其中一個Span ID
![VS--YouTube-OpenTelemetry-64’05”](https://hackmd.io/_uploads/ryHUbFDiC.png)
[detail of sql query](https://youtu.be/PT-Bjs6iCug?si=Vko-x_GIjKeDAd-y&t=3845)






# Steps

(1) 確認Local可生成監控數據(trace/log/metrics)
(2) k8s Application deploy setting
(3) Grafana or others(ex:[SigNoz](https://signoz.io/)) 呈現 use tempo (datasource) 


---

# Relations

```Example Application (OpenTelemetry Agent) <- (OTLP/gRPC)- OpenTelemetry Collector <- Tempo <- (Grafana|Others)``` 

# Python

- `opentelemetry-instrument` is OpenTelemetry Agent
- OTLP is **exporter**
- OpenTelemetry Collector(port:4317)

---

:::danger
*(需重寫以下...for Python解說)*

需新增: dockerfile adjustment, Basic(不改動source code) vs Advanced(動source code)
:::


**OpenTelemetry(OTel)** with Python

[官網參考_Python Application adopt OpenTelemetry](https://opentelemetry.io/docs/languages/python/getting-started/)

## (1) Basic - Run the instrumented app
- Automatic instrumentation
  >Automatic instrumentation captures telemetry at the edges of your systems, such as inbound and outbound HTTP requests.

- opentelemetry-distro **package**
  >Install the opentelemetry-distro package, which contains the OpenTelemetry API, SDK and also the tools opentelemetry-bootstrap and opentelemetry-instrument...

## (2) Advanced - Modify Code for manual instrumentation to know what’s going on in the application




Code Snippet - Traces
:::spoiler
```python3=
from random import randint
from flask import Flask

from opentelemetry import trace

# Acquire a tracer
tracer = trace.get_tracer("diceroller.tracer")

app = Flask(__name__)

@app.route("/rolldice")
def roll_dice():
    return str(roll())

def roll():
    # This creates a new span that's the child of the current one
    with tracer.start_as_current_span("roll") as rollspan:
        res = randint(1, 6)
        rollspan.set_attribute("roll.value", res)
        return res

```
:::

Code Snippet - Metrics

:::spoiler
```python3=
# These are the necessary import declarations
from opentelemetry import trace
from opentelemetry import metrics

from random import randint
from flask import Flask, request
import logging

# Acquire a tracer
tracer = trace.get_tracer("diceroller.tracer")
# Acquire a meter.
meter = metrics.get_meter("diceroller.meter")

# Now create a counter instrument to make measurements with
roll_counter = meter.create_counter(
    "dice.rolls",
    description="The number of rolls by roll value",
)

app = Flask(__name__)
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@app.route("/rolldice")
def roll_dice():
    # This creates a new span that's the child of the current one
    with tracer.start_as_current_span("roll") as roll_span:
        player = request.args.get('player', default = None, type = str)
        result = str(roll())
        roll_span.set_attribute("roll.value", result)
        # This adds 1 to the counter for the given roll value
        roll_counter.add(1, {"roll.value": result})
        if player:
            logger.warn("{} is rolling the dice: {}", player, result)
        else:
            logger.warn("Anonymous player is rolling the dice: %s", result)
        return result

def roll():
    return randint(1, 6)

```
:::




---

# Kubernetes (k8s)
:::success
For Deploy to Kubernetes
:::
[在不改動 source code 的情況下，將 Opentelemetry 的 Python auto-instrumentation 加入，導入 trace/log/metrics 監控數據生成](https://sean22492249.medium.com/%E5%9C%A8%E4%B8%8D%E6%94%B9%E5%8B%95-source-code-%E7%9A%84%E6%83%85%E6%B3%81%E4%B8%8B-%E5%B0%87-opentelemetry-%E7%9A%84-python-auto-instrumentation-%E5%8A%A0%E5%85%A5-%E5%B0%8E%E5%85%A5-trace-log-metrics-%E7%9B%A3%E6%8E%A7%E6%95%B8%E6%93%9A%E7%94%9F%E6%88%90-4da76e97862b)




