---
tags: paper-reading
---

# 📃 Aspect-based Sentiment Analysis in Question Answering Forums
Wenxuan Zhang et al. (Hong Kong University)
DAMO Academy, Alibaba Group
Paper: https://aclanthology.org/2021.findings-emnlp.390.pdf
Code: https://github.com/IsakZhang/ABSA-QA

### 1. Introduction
ABSA = ATE + ASC (same task-splitting as our current system)
ABSA-QA (the ABSA task on QA forums, where *QA forum* means an opinion-sharing platform have comment section that allows people to ask questions and answer)


### 2. Methodology

#### Problem & Task Formulation
1. sequence-labeling problem, labels: $\mathcal{Y}^u = \{B, I, E, S\}-\{POS, NEU, NEG\} \cup \{0\}$
2. Given a QA pair, $Q = \{q_1, ..., q_m\}$ and $A = \{a_1, ..., a_n\}$, we aim to detect the discussed aspects and their sentiment polarities by predicting a tag sequence $Y = \{y_1, ..., y_m\}$ for the question text where $y_i \in \mathcal{Y}^u$.
#### Model Architecture

#### <span style="color: #54993C;"> Cross-sentence aspect info fusion </span>
1. 應該是指 q-a 之間是跨語句的關係，背後原理則是兩個句子 X, Y 互相關注（inter-QA attention） 算 attention matrix。$MH-ATT(Q,K,V)$ 是 2017 Vaswani et al. 發表的 multi-head attention 機制。
$\bar{H^{q}} = ATTN(H^q, H^a)$ # using question as `query` to align with key in the answer
$\bar{H^{a}} = ATTN(H^a, H^q)$ # using answer as `query` to align with the `key` in the question

2. Cross-sentence aspect info fusion 裡 Q 的 component 中，given $H^q$ 和 $\bar{H}^q$ ，如何決定最終的 question representation？ 簡單來說就是用 element-wise weighted sum，權重值 $g$ 以 MLP 訓練出來。
最終的 refined- representation $S$ 是 $\tilde{H}$ 的 self-attention 後的版本 $S=ATTN(\tilde{H}, \tilde{H})$。

#### <span style="color: #54993C;"> Answer-guided sentiment prediction </span>
將 Cross-sentence aspect info fusion 裡的 Answer component 的 output repr. 經過 self-attention encoding 後得到 $\bar{p}$，
This $\bar{p}$ summarizes the main opinion info in the answer. $\bar{p}$ after a linear transformation is $p$, and is then concatenated to the $S$ above.
A point-wise CNN operation is applied:
$O$ is the final question representation.

#### <span style="color: #54993C;"> Model training (left side) </span>
使用的 BERT 基底：`bert-base-chinese`
1. ALSC task over $O$
$o_i$: final operation for question token $$

2. ATE task over $S$
To enforce better aspect-aware question repr.
$s_i$: self-attention encoding for question token $i$

3. Joint training
 where $\lambda$ is a hyperparam.

#### <span style="color: #54993C;"> Interaction layer 的權重初始化：QA-Match Pre-training </span>
有點像 contrastive learning 的概念。以原先配好的 qa pairs 作為 positive samples，挑另一組 qa 的 q 配上某個 a 後累積出很多 negative samples。
where $\hat{x}$ is the prediction score of matching (loss fn: cross-entropy)。

### 3. Experiments



#### <span style="color: #54993C;"> Model training </span>
#### <span style="color: #54993C;"> QA-Match Pre-training </span>
以上幾個 mechanisms 中 ATE (joint-learning) 與 QA-matching 都有做 ablation study （比較拆除與加上的效果）。可以看到他們都可以帶來更好的成績。

#### <span style="color: #54993C;"> Answer-guided sentiment prediction </span>
$\bar{S} = [S;P]$ 效果遠好過只使用 S。

Case Analysis

### 5. Conclusions
1. we investigate the aspect-basedsentiment analysis in **question answering forums (ABSA-QA)**, aiming to jointly detect the dis-cussed aspects and their sentiment polarities for a given QA pair.

2. we propose a model with carefully designed **cross-sentence aspect-opinion interaction** to tackle the task. More-over, we utilize **two auxiliary tasks including as-pect term extraction task for learning better aspect-aware representation and QA pair matching task** to pre-train the inter-QA attention components to for **better aligning the question and answer sentence**.