# Results ## Word-Level Probing: Results (Avg Acc) (2023-10-02) ### Logistic Regression Classifier ![](https://hackmd.io/_uploads/rJ0-8DdxT.png) ![](https://hackmd.io/_uploads/ByRbUwOe6.png) ## Word-Level Probing: Results (F1-Scores) (2023-10-02) ### Logistic Regression Classifier ![](https://hackmd.io/_uploads/SyC-IwOg6.png) ![](https://hackmd.io/_uploads/HJRZIv_l6.png) ## Noise / BLEU correlation results (2023-09-28) ### Sentence-Level: past_present ![](https://hackmd.io/_uploads/HkPUWGml6.png) ![](https://hackmd.io/_uploads/ryvIZf7ea.png) ![](https://hackmd.io/_uploads/S1wL-GXxT.png) ### Word-Level: dep ![](https://hackmd.io/_uploads/B1MvZfmxa.png) ![](https://hackmd.io/_uploads/SkmPWGXxp.png) ![](https://hackmd.io/_uploads/BJzwZzmxa.png) ## Noise / BLEU correlation results (2023-09-25) ### Sentence-Level: past_present ![](https://hackmd.io/_uploads/S1t3eC016.png) ![](https://hackmd.io/_uploads/HJgaeCR1a.png) ![](https://hackmd.io/_uploads/B1wpxR0yp.png) ### Word-Level: dep ![](https://hackmd.io/_uploads/rJhplAR16.png) ![](https://hackmd.io/_uploads/rkgClARk6.png) ![](https://hackmd.io/_uploads/ryHAl0Rya.png) ## Word-Level Probing: F1-Scores (2023-09-10) ### Logistic Regression Classifier ![](https://hackmd.io/_uploads/ByzYYihCh.png) ![](https://hackmd.io/_uploads/HyRFKonAh.png) ### Multi-Layer-Perceptron Classifier ![](https://hackmd.io/_uploads/HkS5Foh03.png) ![](https://hackmd.io/_uploads/B169tinRh.png) ## Word-Level Probing: Results (Avg Acc) (2023-09-10) ### Logistic Regression Classifier ![](https://hackmd.io/_uploads/SyZPZ03R2.png) ![](https://hackmd.io/_uploads/HywD-R30n.png) ### Multi-Layer-Perceptron Classifier ![](https://hackmd.io/_uploads/BkTv-RnAn.png) ![](https://hackmd.io/_uploads/H1xUUtinC2.png) ## Word-Level Probing: Noise-Scores (2023-09-10) ### Logistic Regression Classifier #### sem ![](https://hackmd.io/_uploads/BJgTvwoCh.png) ![](https://hackmd.io/_uploads/By9pvwoR2.png) ![](https://hackmd.io/_uploads/S1v0wviC2.png) ### Multi-Layer-Perceptron Classifier ![](https://hackmd.io/_uploads/rkYJuDoA2.png) ![](https://hackmd.io/_uploads/rJzeuwjAh.png) ![](https://hackmd.io/_uploads/BJ2g_PjR2.png) ## Word-Level Probing: Noise-Scores (2023-09-01) ### Logistic Regression Classifier #### dep ![](https://hackmd.io/_uploads/ryX-c_y02.png) ![](https://hackmd.io/_uploads/rkpM5d10h.png) ![](https://hackmd.io/_uploads/S16Qqdy02.png) #### upos ![](https://hackmd.io/_uploads/SJnV5d1Ah.png) ![](https://hackmd.io/_uploads/HkFS5Oy0h.png) ![](https://hackmd.io/_uploads/SyzU5_kC3.png) #### xpos ![](https://hackmd.io/_uploads/ByaL9OJCh.png) ![](https://hackmd.io/_uploads/SkUvqdJR3.png) ![](https://hackmd.io/_uploads/S1lucdyCn.png) ### Multi-Layer-Perceptron Classifier #### dep ![](https://hackmd.io/_uploads/Hy9ucOkA3.png) ![](https://hackmd.io/_uploads/H1Dtqd1A3.png) ![](https://hackmd.io/_uploads/rJe5cd1Rn.png) #### upos ![](https://hackmd.io/_uploads/Hya5cOJ03.png) ![](https://hackmd.io/_uploads/rywjqOkCh.png) ![](https://hackmd.io/_uploads/HJen5uJAh.png) #### xpos ![](https://hackmd.io/_uploads/r1Fh5uyCh.png) ![](https://hackmd.io/_uploads/ByXp5OJA2.png) ![](https://hackmd.io/_uploads/HkYaquJ0n.png) ## Word-Level Probing: F1-Scores (2023-08-29) ### Logistic Regression Classifier #### Probing Tasks Performance ![](https://hackmd.io/_uploads/rJ08ads6n.png) ![](https://hackmd.io/_uploads/ry6tT_ja3.png) #### Visual vs. Text Encoder ![](https://hackmd.io/_uploads/Hyf3Tujph.png) ![](https://hackmd.io/_uploads/SJphTui62.png) ### Multi-Layer-Perceptron Classifier #### Probing Tasks Performance ![](https://hackmd.io/_uploads/ByVb0_oph.png) ![](https://hackmd.io/_uploads/ByAb0Osp3.png) #### Visual vs. Text Encoder ![](https://hackmd.io/_uploads/S1vG0Oja2.png) ![](https://hackmd.io/_uploads/Byr70Ooa2.png) ## Word-Level Probing: Results (Avg Acc) (2023-08-10) ### Logistic Regression Classifier ![](https://hackmd.io/_uploads/HknZEUMhn.png) ![](https://hackmd.io/_uploads/Hk7zNIfhh.png) ### Multi-Layer-Perceptron Classifier ![](https://hackmd.io/_uploads/Ska5r8z3n.png) ![](https://hackmd.io/_uploads/HkEor8f3h.png) ## Word-Level Probing: Results (2023-08-03) ### Logistic Regression Classifier ![](https://hackmd.io/_uploads/BJq2ZfYs2.png) ![](https://hackmd.io/_uploads/r16kh7tj3.png) ### Multi-Layer-Perceptron Classifier ![](https://hackmd.io/_uploads/HyxiLIznn.png) ![](https://hackmd.io/_uploads/Bkgri8Uf33.png) ## BLEU, Dataset MTTT & WMT: Results (2023-03-29) ### bigram_shift #### cam ![](https://i.imgur.com/JkhoC7m.png) #### l33t ![](https://i.imgur.com/aUQXRqQ.png) #### swap ![](https://i.imgur.com/imN1ASY.png) ### past_present #### cam ![](https://i.imgur.com/VfaMck4.png) #### l33t ![](https://i.imgur.com/nvXBDtt.png) #### swap ![](https://i.imgur.com/Xx78qBK.png) ### object_number #### cam ![](https://i.imgur.com/ryizLJq.png) #### l33t ![](https://i.imgur.com/TgNpphq.png) #### swap ![](https://i.imgur.com/MdBRcev.png) ### subject_number #### cam ![](https://i.imgur.com/5G9lLl0.png) #### l33t ![](https://i.imgur.com/r18StDI.png) #### swap ![](https://i.imgur.com/yZWCuRk.png) ## BLEU, Dataset MTTT: Results (2023-03-29) ![](https://i.imgur.com/L7Xa6IK.png) ## BLEU: Results (2023-03-24) ### bigram_shift #### cam ![](https://i.imgur.com/gxjz3p9.png) #### l33t ![](https://i.imgur.com/GpJDj9P.png) #### swap ![](https://i.imgur.com/Y6VUfPC.png) ### past_present #### cam ![](https://i.imgur.com/m8SNHfT.png) #### l33t ![](https://i.imgur.com/qPrYq4h.png) #### swap ![](https://i.imgur.com/j2upFFW.png) ### object_number #### cam ![](https://i.imgur.com/WP2iW1d.png) #### l33t ![](https://i.imgur.com/ZQF1602.png) #### swap ![](https://i.imgur.com/Rzc2Ooq.png) ### subject_number #### cam ![](https://i.imgur.com/d4Hr63Z.png) #### l33t ![](https://i.imgur.com/07A99VN.png) #### swap ![](https://i.imgur.com/liBR3c8.png) ## BLEU: Results (2023-03-19) ### bigram_shift #### cam ![](https://i.imgur.com/OZJ9HNS.png) #### l33t ![](https://i.imgur.com/Efbzh4Q.png) #### swap ![](https://i.imgur.com/tNJkxFl.png) ### past_present #### cam ![](https://i.imgur.com/lrKVDFQ.png) #### l33t ![](https://i.imgur.com/CCWf2Pf.png) #### swap ![](https://i.imgur.com/1zxfzkV.png) ### object_number #### cam ![](https://i.imgur.com/TbDO24Q.png) #### l33t ![](https://i.imgur.com/ax6m9Wb.png) #### swap ![](https://i.imgur.com/N9yAu1b.png) ### subject_number #### cam ![](https://i.imgur.com/sJxs7nq.png) #### l33t ![](https://i.imgur.com/N5fjX4N.png) #### swap ![](https://i.imgur.com/Xgz6p4e.png) ## BLEU: Results (2023-03-14) ### bigram_shift #### cam ![](https://i.imgur.com/qOHw9P6.png) #### l33t ![](https://i.imgur.com/NaHaPYa.png) #### swap ![](https://i.imgur.com/Cao3GKu.png) ### past_present #### cam ![](https://i.imgur.com/5A5QUwg.png) #### l33t ![](https://i.imgur.com/NToSAxO.png) #### swap ![](https://i.imgur.com/QMaCCtS.png) ### object_number #### cam ![](https://i.imgur.com/4Cv544F.png) #### l33t ![](https://i.imgur.com/Xn854he.png) #### swap ![](https://i.imgur.com/zYnbS6u.png) ### subject_number #### cam ![](https://i.imgur.com/ElS4j0q.png) #### l33t ![](https://i.imgur.com/wAi4Tpk.png) #### swap ![](https://i.imgur.com/Rw543yY.png) ## Sentence-Level: Probing Results (2023-02-16) ### Version 1 ![](https://i.imgur.com/UfooeTw.png) ### Version 2 ![](https://i.imgur.com/CynN42v.png) ## Sentence-Level: Noise (2023-02-16) ### bigram_shift #### cam ![](https://i.imgur.com/KKy4s2i.png) #### l33t ![](https://i.imgur.com/YAu0CQQ.png) #### swap ![](https://i.imgur.com/UvhT9Oq.png) ### past_present #### cam ![](https://i.imgur.com/yXUFt2b.png) #### l33t ![](https://i.imgur.com/pXN4FK3.png) #### swap ![](https://i.imgur.com/ethtlo9.png) ### object_number #### cam ![](https://i.imgur.com/BVnfXeR.png) #### l33t ![](https://i.imgur.com/g0UyrOm.png) #### swap ![](https://i.imgur.com/SlnE7cb.png) ### subject_number #### cam ![](https://i.imgur.com/mFmEtkE.png) #### l33t ![](https://i.imgur.com/uouNl5d.png) #### swap ![](https://i.imgur.com/GptRSRo.png) ## New visualization per Layer (2023-02-09) ### past_present #### cam ![](https://i.imgur.com/iDELGYo.png) #### l33t ![](https://i.imgur.com/ZZ8EzIn.png) #### swap ![](https://i.imgur.com/fZX07w6.png) ### bigram_shift #### cam ![](https://i.imgur.com/9n5Tulg.png) #### l33t ![](https://i.imgur.com/6rI35cG.png) #### swap ![](https://i.imgur.com/MQIdmPQ.png) ## Noise Data; Script Esalesky ### cam ![](https://i.imgur.com/i72BAt9.png) ![](https://i.imgur.com/9yzmCVV.png) #### cam_0.1 Seit der 2007 / 08 tritt Mintšenkova im Seniorenbereich an . #### cam_0.2 Seit der 2007 / 08 tritt Mintšenkova im **Soereibceinnerh** an . #### cam_0.4 Seit der 2007 / 08 tritt **Mtvšnnekoia** im **Sincnerieoeerbh** an . #### cam_0.8 Seit der 2007 / 08 tritt **Mkvešontnia** im Seniorenbereich an . ### l33t ![](https://i.imgur.com/8v6pCoz.png) ![](https://i.imgur.com/hBAqpng.png) #### l33t_0.1 **S3it** der 2007 / 08 **tri7t** **Mintš3nk0va** im Seniorenbereich an . #### l33t_0.2 **Se1t** **d3r** 2007 / 08 **7ritt** **Mintšenk0va** im Seniorenbereich an . #### l33t_0.4 **53i7** **d3r** 2007 / 08 **tr1tt** **Mintšenk0\/a** im **Sen10r3n8ereich** an . #### l33t_0.8 **Se17** **d3r** 2007 / 08 **7r177** **M1n7šenk0\/4** **1m** **53n10r3n83r31ch** **4n** . ### swap ![](https://i.imgur.com/AOVjk9j.png) ![](https://i.imgur.com/nZHvMnl.png) #### swap_0.1 Seit der 2007 / 08 tritt Mintšenkova im Seniorenbereich an . #### swap_0.2 Seit der 2007 / 08 tritt Mintšenkova im Seniorenbereich an . #### swap_0.4 **eS**it d**re** 20**70** / **80** tritt Mintšenkova im Seniorenbereich an . #### swap_0.8 **Seti** der 20**70** / **80** tr**ti**t Mintše**kn**ova im Seniorenber**ie**ch na . ## Dataset NOISE, lineplots, fixed range (2022-01-01) ### OBJ #### swap ![](https://i.imgur.com/o9ckN6M.png) #### cam ![](https://i.imgur.com/lY0KJ4Y.png) ### SUBJ #### swap ![](https://i.imgur.com/2Hrd0Wl.png) #### cam ![](https://i.imgur.com/5rAg2po.png) ## Dataset NOISE, barplots (2022-12-01) ### OBJ #### swap ![](https://i.imgur.com/QY8Ug7B.png) #### cam ![](https://i.imgur.com/dxNL1N4.png) ### SUBJ #### swap ![](https://i.imgur.com/XUDD07R.png) #### cam ![](https://i.imgur.com/7UlhecW.png) ## Dataset Evaluation (2022-10-21) ### SUBJ ![](https://i.imgur.com/J2HYABh.png) ### OBJ ![](https://i.imgur.com/uS7LY0b.png) ## Dataset Evaluation (2022-10-20) ### SUBJ ![](https://i.imgur.com/FPtTI4h.png) ### OBJ ![](https://i.imgur.com/BoMx1EP.png) ## Dataset Evaluation (2022-10-06) ### SUBJ #### Train ![](https://i.imgur.com/0426duc.png) ![](https://i.imgur.com/dyWkccz.png) ![](https://i.imgur.com/gjhm5VK.png) ![](https://i.imgur.com/xwi2BNu.png) ![](https://i.imgur.com/NGncUsg.png) #### Test ![](https://i.imgur.com/1Rv6k7b.png) ![](https://i.imgur.com/1Idgdjy.png) ![](https://i.imgur.com/Tzdie4i.png) ![](https://i.imgur.com/eihO9bh.png) ### OBJ #### Train ![](https://i.imgur.com/2snjkdS.png) ![](https://i.imgur.com/FmacfzX.png) ![](https://i.imgur.com/C2z0ODQ.png) ![](https://i.imgur.com/4A20CMK.png) #### Test ![](https://i.imgur.com/E5oMsk2.png) ![](https://i.imgur.com/NXHLyTG.png) ![](https://i.imgur.com/q9DkifH.png) ![](https://i.imgur.com/FVS2bpa.png) ## Dataset Evaluation, Overlay (2022-09-29) ### Subject ![](https://i.imgur.com/DSszFcC.png) ### Object ![](https://i.imgur.com/EcpIAzJ.png) ## Dataset Evaluation (2022-09-27) ### Subject ![](https://i.imgur.com/1TspGSa.png) ### Object ![](https://i.imgur.com/HE3SW8g.png) ## Dataset Evaluation (2022-08-25) ### Subject ![](https://i.imgur.com/o0apaU3.png) ### Object ![](https://i.imgur.com/9XlHl9K.png) ## Visual vs. Text Model, fixed balanced data (2022-08-15) ### past_pres #### classify 10k sentences ![](https://i.imgur.com/MnMn7wA.png) #### classify 1k sentences ![](https://i.imgur.com/i2t6NdY.png) ### obj_num #### classify 10k sentences ![](https://i.imgur.com/dtAJ4UX.png) #### classify 1k sentences ![](https://i.imgur.com/jolTbj5.png) ### subj_num #### classify 10k sentences ![](https://i.imgur.com/rMXDpDK.png) #### classify 1k sentences ![](https://i.imgur.com/J5t1cMo.png) ### bigram_shift #### classify 10k sentences ![](https://i.imgur.com/GOrnUmi.png) #### classify 1k sentences ![](https://i.imgur.com/lQSIlJO.png) ## Visual vs. Text Model, fixed balanced data (2022-08-08) ### past_pres #### classify 10k sentences ![](https://i.imgur.com/a4i4pXl.png) #### classify 1k sentences ![](https://i.imgur.com/I90sa9I.png) ### obj_num #### classify 10k sentences ![](https://i.imgur.com/hsydYtp.png) #### classify 1k sentences ![](https://i.imgur.com/BdFnAHd.png) ### subj_num #### classify 10k sentences ![](https://i.imgur.com/pUTZvQN.png) #### classify 1k sentences ![](https://i.imgur.com/4Q1mzuO.png) ### bigram_shift #### classify 10k sentences ![](https://i.imgur.com/5lKhGn2.png) #### classify 1k sentences ![](https://i.imgur.com/dob2hia.png) ## Visual vs. Text Model, balanced data (2022-07-29) ### past_pres #### classify 10k sentences ![](https://i.imgur.com/2xCXozc.png) #### classify 1k sentences ![](https://i.imgur.com/ODVcfJt.png) ### obj_num #### classify 10k sentences ![](https://i.imgur.com/nGKrjTW.png) #### classify 1k sentences ![](https://i.imgur.com/u8LMfxc.png) ### subj_num #### classify 10k sentences ![](https://i.imgur.com/2ynhbKO.png) #### classify 1k sentences ![](https://i.imgur.com/eMubOhV.png) ### bigram_shift #### classify 10k sentences ![](https://i.imgur.com/v1UzHDQ.png) #### classify 1k sentences ![](https://i.imgur.com/jx4VIFg.png) ## Visual vs. Text Model, clean data + dummy classifier (2022-07-18) ### past_pres #### classify 10k sentences ![](https://i.imgur.com/AIfak3v.png) #### classify 1k sentences ![](https://i.imgur.com/aIAMVZC.png) ### obj_num #### classify 10k sentences ![](https://i.imgur.com/fxksdAN.png) #### classify 1k sentences ![](https://i.imgur.com/8J5Uo67.png) ### subj_num #### classify 10k sentences ![](https://i.imgur.com/PP8U56p.png) #### classify 1k sentences ![](https://i.imgur.com/G4ZLpdB.png) ### bigram_shift #### classify 10k sentences ![](https://i.imgur.com/hRvaxpR.png) #### classify 1k sentences ![](https://i.imgur.com/CcNwqC1.png) ## Visual vs. Text Model, range[0.45, 1.0] (2022-07-18) ### past_pres #### classify 10k sentences ![](https://i.imgur.com/mrtVeAS.png) #### classify 1k sentences ![](https://i.imgur.com/oCHl7Z2.png) ### obj_num #### classify 10k sentences ![](https://i.imgur.com/sOEOKqx.png) #### classify 1k sentences ![](https://i.imgur.com/ttmH8BS.png) ### bigram_shift #### classify 10k sentences ![](https://i.imgur.com/GUl2Qv4.png) #### classify 1k sentences ![](https://i.imgur.com/ohBnqi2.png) ## Visual vs. Text Model, full view (2022-07-18) ### past_pres #### classify 10k sentences ![](https://i.imgur.com/QaN4IIA.png) #### classify 1k sentences ![](https://i.imgur.com/Deuq3ON.png) ### obj_num #### classify 10k sentences ![](https://i.imgur.com/CJokX8T.png) #### classify 1k sentences ![](https://i.imgur.com/yxgCBOI.png) ### bigram_shift #### classify 10k sentences ![](https://i.imgur.com/DZiVXS3.png) #### classify 1k sentences ![](https://i.imgur.com/GcdJD8X.png) ## Avg. vs first token tensor (2022-07-15) ### past_pres #### classify 10k sentences ##### visual ![](https://i.imgur.com/LPR7HxK.png) ##### text ![](https://i.imgur.com/OYxwaAy.png) #### classify 1k sentences ##### visual ![](https://i.imgur.com/cIm0K5B.png) ##### text ![](https://i.imgur.com/Sty6psP.png) ### obj_num #### classify 10k sentences ##### visual ![](https://i.imgur.com/vAXUaE9.png) ##### text ![](https://i.imgur.com/fp8HH9g.png) #### classify 1k sentences ##### visual ![](https://i.imgur.com/XdxIYDK.png) ##### text ![](https://i.imgur.com/t6tcXw7.png) ### bigram_shift #### classify 10k sentences ##### visual ![](https://i.imgur.com/8Htqqei.png) ##### text ![](https://i.imgur.com/g2qwYS3.png) #### classify 1k sentences ##### visual ![](https://i.imgur.com/0OidmEL.png) ##### text ![](https://i.imgur.com/yTJgYQu.png) ### subj_num (To-Do, classification error) #### classify 10k sentences ##### visual ##### text ### classify 1k sentences #### visual #### text