---
# System prepended metadata

title: Evaluation
tags: [decompiler]

---

---
tags: decompiler
title: Evaluation
---

# Overview
This document logs all of the evaluation metrics needed and the progress on them. 

## Details:
### Queries
- How to visualize stats? - Keep tables and excel files -> Conversion later
- Is taking average every time the right approach?
    - If no average what could be the other potential approach

### Updates
- Number of multiple rules in the same function
- Table 4:
    - Average in table for rules applied
    - Top 3:
        - Rules that cause most error
        - Most effective solution
    - 
- Rules applied graph
- Rules c

### Do have
- Fixes
    - Number of functions resolved
    - Number of files resolved
- Errors
    - Total number of errors
    - Number of errors in a single file
    - Numbers of errors in a single function
- False positives
- Rules
    - Number of rules applied
    - Number of solutions per fix
    - Changing error form
    - 
- Number 
### Don't have
- Number of blocks iterated
    - Distance between the graph -> Fix and offset reported
- Basic blocks transformed -> Impact of transformation
    - Formulate this
- Correctness
    - Manually verify
    - Leverage confirmed transformations
    - Categorize transofrmations into types that are confirmed -> proved 
- Time taken
    - For creating a rule <- low priorty - appendix
    - for implementing <- low priorty - appendix
    - 
- Efficiency ->very broad
    - No priority
- Performance overhead
    - 
