# Efficacy vs Faithfulness <style> table th:first-of-type { width: 33%; } table th:nth-of-type(2) { width: 33%; } table th:nth-of-type(3) { width: 33%; } table th:nth-of-type(4) { width: 30%; } </style> :::info **`prompt_template = " {} :"`** ::: :::warning **relations with less than 30 test samples are in muted color.** ::: ## GPT-J ### Efficacy vs Faithfulness ![](https://hackmd.io/_uploads/HkqiW1OU3.png) #### Single vs multitoken subjects ![](https://hackmd.io/_uploads/rkDA-k_Un.png) ### Scatter plots on different $\beta$ ![](https://hackmd.io/_uploads/r1nCJ-oth.png) ### with $\beta = 2.25$ ![](https://hackmd.io/_uploads/SJu0y-oK2.png) ![](https://hackmd.io/_uploads/B1x5WlZjtn.png) ## GPT2-xl ### Scatter plots on different $\beta$ ![](https://hackmd.io/_uploads/ryXrmfjFn.png) ### with $\beta = 2.25$ ![](https://hackmd.io/_uploads/rJ5OQMjKh.png) ![](https://hackmd.io/_uploads/rkWaQfoFh.png) ## LLaMa-13B ### Scatter plots on different $\beta$ ![](https://hackmd.io/_uploads/ryaH5w3K2.png) ### with $\beta = 5.0$ ![](https://hackmd.io/_uploads/H1QB5wnFh.png) ![](https://hackmd.io/_uploads/BJIVqw2th.png)