# Detailed Roadmap for Tokio Tracing integration

The project will be implemented in four incremental phases, progressing from foundational setup to advanced tooling and full integration across the Grandine codebase.

flowchart TD
    style A fill:#e6f7ff,stroke:#1890ff,stroke-width:2px
    style B fill:#e6f7ff,stroke:#1890ff,stroke-width:2px
    style C fill:#e6f7ff,stroke:#1890ff,stroke-width:2px
    style D fill:#e6f7ff,stroke:#1890ff,stroke-width:2px

    A[Phase 1: <br>Foundational Setup]
    B[Phase 2: <br>Network Instrumentation]
    C[Phase 3: Core Logic <br>Instrumentation]
    D[Phase 4: <br> Advanced Tooling <br>& FInal Migration]

    A --> B --> C --> D

    subgraph S1 [Phase 1 Details]
        A1(Enable tracing)
        A2(Replace logger)
        A3(Custom formatting <br>and EnvFilter support)
    end
    A --> S1

    subgraph S2 [Phase 2 Details]
        B1(Trace HTTP API)
        B2(Trace P2P)
    end
    B --> S2

    subgraph S3 [Phase 3 Details]
        C1(Fork Choice)
        C2(Validator Duties)
    end
    C --> S3

    subgraph S4 [Phase 4 Details]
        D1(Tokio Console)
        D2(Flamegraphs - <span style="font-size:smaller">optional</span>)
        D3(Final migration)
    end
    D --> S4

### Phase 1: Foundational Setup
Goal: Enable tracing globally and establish a root context for all subsequent diagnostic events.
Requirements: Understand Tokio, Tracing, and Grandine crate organization.

- Enable tracing: Add and configure tracing-subscriber as the main diagnostics backend.
- Replace logger: Modify binary_utils to replace the existing logger setup with tracing-based initialization, ensuring all logs flow through the new system.
- Custom formatting and EnvFilter support: Implement human-friendly output formatting and allow dynamic control over log levels via the RUST_LOG environment variable using EnvFilter.

### Phase 2: Tracing the Network Boundary

Goal: Gain visibility into network boundaries and system entry points.
Requirements: Familiarity with Grandine external interfaces

key components: **HTTP API** & **P2P network**.

Step **2.1**: Instrument the HTTP API
- Key Crate: http_api
- Action: Instrument each function that handles an HTTP endpoint. This will automatically log the start and end of each API call, its duration, and any parameters we choose to include in the span.

Step **2.2**: Instrument the P2P Layer
This is critical for debugging network issues.
- Key Crate: eth2_libp2p
- Action: Add spans and events around network event processing, including peer connections enriched with structured context like peer_id.

### Phase 3: Instrumentation of Core Logic
Goal: Enable observability of consensus-critical components.
Requirements: Understanding the behavior of the **fork choice rule** and **validator duties**.

Step **3.1**: Instrument Fork Choice
- Key Crate: fork_choice_control
- Action: Instrument the key functions that modify or read the fork choice store. This will help debug consensus issues and understand why the node chooses a specific chain head.

Step **3.2**: Instrument Validator Duties
- Key Crate: validator
- Action: Add tracing to the logic that produces blocks and attestations. This will show when a validator is performing its duties and whether it succeeds.

### Phase 4: Advanced Tooling & Full Migration
Goal: Leverage the full power of the tracing ecosystem and complete the migration.
Requirements: Advanced tracing configuration, Tokio runtime options, performance benchmarking, and tooling interpretation.

Step **4.1**: Integrate Tokio Console
This will provide a live, real-time dashboard of all tasks, resources, and their current state.

Step **4.2**: Generate Flamegraphs for Performance **(Optional)**
Action: Introduce tracing-flame to generate flamegraphs from trace data. This is the ultimate tool for visualizing time-based performance and identifying bottlenecks.


Step **4.3**: Final log to tracing Migration
Action: With the core components instrumented, perform a project-wide search and replace of all remaining log:: macros with their tracing:: equivalents. This ensures all diagnostic output is unified under the tracing system.

---
This structured approach ensures a smooth, incremental rollout, providing value at each stage and culminating in a state-of-the-art diagnostics system for the Grandine.

---