---
# System prepended metadata

title: Efficacy vs Faithfulness

---

# Efficacy vs Faithfulness
<style>
table th:first-of-type {
    width: 33%;
}
table th:nth-of-type(2) {
    width: 33%;
}
table th:nth-of-type(3) {
    width: 33%;
}
table th:nth-of-type(4) {
    width: 30%;
}
</style>


:::info
**`prompt_template = " {} :"`**
:::

:::warning
**relations with less than 30 test samples are in muted color.**
:::

## GPT-J


### Efficacy vs Faithfulness
![](https://hackmd.io/_uploads/HkqiW1OU3.png)



#### Single vs multitoken subjects
![](https://hackmd.io/_uploads/rkDA-k_Un.png)

### Scatter plots on different $\beta$
![](https://hackmd.io/_uploads/r1nCJ-oth.png)

### with $\beta = 2.25$
![](https://hackmd.io/_uploads/SJu0y-oK2.png)
![](https://hackmd.io/_uploads/B1x5WlZjtn.png)


## GPT2-xl
### Scatter plots on different $\beta$
![](https://hackmd.io/_uploads/ryXrmfjFn.png)

### with $\beta = 2.25$
![](https://hackmd.io/_uploads/rJ5OQMjKh.png)
![](https://hackmd.io/_uploads/rkWaQfoFh.png)

## LLaMa-13B
### Scatter plots on different $\beta$
![](https://hackmd.io/_uploads/ryaH5w3K2.png)


### with $\beta = 5.0$
![](https://hackmd.io/_uploads/H1QB5wnFh.png)
![](https://hackmd.io/_uploads/BJIVqw2th.png)
