---
# System prepended metadata

title: ICLR Revision Note (General Response)

---

# ICLR Revision Note (General Response)


We thank all reviewers for their insightful questions and valuable feedback. We are particularly encouraged that they consider the tackled problem both interesting and important.
Here we briefly outline the updates to the revised submission based on the reviews. We address individual questions of reviewers in separate responses.

$\newcommand{Ro}{\textcolor{purple}{Mytf}}$
$\newcommand{Rt}{\textcolor{red}{TdLu}}$
$\newcommand{Rth}{\textcolor{blue}{j8n4}}$
$\newcommand{Rf}{\textcolor{green}{S7Fk}}$

Paper Updates:
[General] We completely removed the association between Tuformer and Tucker Tensor Representation from the manuscript to avoid misunderstandings. We also re-organized the structure and added extensive details in the motivation of THSA (and Tuformer) and how THSA (and Tuformer) is derived. ($\Ro$, $\Rt$, $\Rth$, $\Rf$).We also changed the title accordingly.
[Section 3 Tuformer] We revised the wording to avoid misunderstanding of proposed design space.($\Rt$ $\Rth$). 
[Section 4 Experiment] We added the inference run-time comparison($\Rf$), we updated the ALBERT experiments using the largest possible pre-trained models($\Ro$).
[Section 5 Related Work] We added a review of tensor decomposition in machine learning.
