# ICLR 2023 Rebuttal GAT-MTSF (yXLj)

###### tags: `Rebuttal - template`

We thank Reviewer yXLj for the detailed and insightful feedback. We are happy that Reviewer yXLj thinks our paper is easy to follow and has a good quality.

We have updated our manuscript and clarified the claims, as suggested by Reviewer yXLj. Below we address Reviewer yXLj's concerns in detail.

> Concern 1: "However, in this paper, the authors assume that we are given the true future C-step-away values for a subset of time series. This assumption seems to be counter-intuitive and not very realistic. Can the authors comment on the assumption?"

We do not have any assumption regarding access to future C-step-away values of any series in the testing phase. In our experiments, future values are not included in the testing phase. In the training phase, under both semi-supervised and fully supervised settings, it is necessary and correct to have the true response because we need to calculate a supervised loss. This so-called "C-step-away future data" in training phase is actually part of the history data. For example, suppose that the current time is $t$, the $C$-step-away future time is $t+1,...,t+C$, the variable that we want to predict is $Y$ and the inputs to the forecasting model is $X$. Then, in the training phase of the model $f$, under any time $t$, the forward pass is $Loss(f(X_{t-7, t-6, ..., t}), Y_{t+1,...,t+C})$. In the text, we call $Y_{t+1,...,t+C}$ the $C$-step-away future data, which might cause confusions. Actually, it is just future data relative to $X_{t-7, t-6, ..., t}$. In the testing phase, we make sure that there is no overlap between the training data time period and testing data time period, and we use $f(X_{t+C+1, ..., t+C+7})$ to predict the target variable $Y_{t+C+7+1, ..., t+C+7+C}$. The naming may cause confusions. It is also possible that some of the confusions arise from the semi-supervised settings where only some of the nodes in the graph are used to calculate loss in the training phase, which is different from assuming awareness of future values of some series in the testing phase. This semi-supervised setting is closer to the assumption that some nodes' responses (the stock price in this case) are missing in the history training data. So, the assumption underlying the semi-supervised setting is actually more general in the sense that we allow some nodes' reponses to be missing in the training phase. As for the temporal dimension, we strictly ensure that all the time points in the training phase precedes the earliest time point in the testing phase. Thus, there is no data leakage problem.

<!-- For example, suppose that the current time is $t$, the one-step-away future time is $t+1$, the variable that we want to predict is $Y$ and the inputs to the forecasting model is $X$. The history training data we have ranges from the time $t-\tau$ to $t$ (left-right included) and the testing data include data from time $t-7$. Then, in the training phase of the model $f$, the forward pass is $Loss(f(X_{t-\tau, t-\tau+1, ..., t-\tau+6}), Y_{t-\tau+6+1,...,t-\tau+6+C})$. In the paper writing, we call $Y_{t-\tau+6+1,...,t-\tau+6+C}$ the C-step-away future data, which might cause confusions. Actually, it is just future data relative to $X_{t-\tau,...,t-\tau+6}$. In the testing phase, we do not assume any $Y$ in the future exists at all and we use $f(X_{t-7, t-6, ..., t})$ to obtain the prediction on $Y_{t+1}$ and we guarantee that $t-7 > t-\tau+6+C$. -->

Thank you so much for raising this concern and giving us the opportunity to further clarify it.

> Concern 2: "The authors assume that each node in the graph has the same degree. ... I think that requiring each node to have the same degree, e.g., requiring that every node to attend to every other node, is in fact a limitation rather than being general. The general setting should be without assuming anything about the structure of the graph..."

We do not assume that each node in the real graph has the same degree. This impression may arise from both the method we use to process the graph and the simplification of the theory model. It has been shown in literature that the message passing mechnism in graph neural network does not work well when the number of the neighbors of a node is too large, which causes the over-smoothing issue [1]. The intuition of this over-smoothing issue is that those unimportant neighbors' information dilate important neighbors' information so that they increase the noise ratio of the target node and degrade the final performance. Thus, a sparse graph is preferred and the method we use to ensure a sparse graph is to select only the top k neighbors in the message passing based on the correlation ranking. Thus, after taking the top K neighbors, every node has the same degree.

For the concern related to the generality, the reviewer thinks that we should not assume any graph structure, which is the same as our opinion in the paper. As explained in the above paragraph, the top K control naturally makes degree of all nodes the same. This top K control only sparsifies the graph. When the value of K is set to the maximum that is equal to the total number of nodes, we allow every node to attend to other nodes in the graph. It is noteworthy that in this case, the degree to which a node is attended is completely dependent on the attention scores learned by the model. When the attention of a specific node is very close to 0, its information almost does not pass on in the message passing mechanism, equivalent to no edge connected to this node, informing the generality of our work. In comparison, if the K is set to some constants smaller than the total number of nodes, then the node has no freedom to attend to certain nodes in the graph. Furthermore, as we can see, there is no specific graph structure assumption being made in our work. The graph structure is decided by the attention scores learned by the model.

[1] D. Chen, Y. Lin, W. Li, P. Li, J. Zhou, and X. Sun, “Measuring and relieving the over-smoothing problem for graph neural networks from the topological view,” in AAAI, 2020.

> Concern 3: "In general, I feel like the setting in this paper, i.e, problem formulation in section 2.1 and the graph structure in section 2.2, is not really the typical setting for time series forecasting. It feels more like graph-based semi-supervised regression. For example, I couldn't find any characteristics in the problem setting that are special to time series forecasting. Moreover, I think that, in general, semi-supervised setting in more typical for classification rater than forecasting. Please correct me if I am wrong."

As we clarify above, the setting in the paper is actually not the classic semi-supervised setting. It is more of a tweak of the supervised setting, where in the training phase, we use the supervised loss and ensure that the training time steps do not overlap with the testing time period, which is also a classic time series forecasting setting. We call it semi-supervised in the paper is only because that we allow some of the nodes' responses to be missing in the training phase and calculate the supervised loss using a subgraph. Consequently, it is not simply a graph-based semi-supervised regression. Additionally, both the graph structure and the node attributes are specific to time series forecasting in the sense that each node represents a time series and we use a graph-based model to capture the inter-correlations between each variate in the multivariate time series forecasting task. We are not the first to adopt a graph-based setting in time series forecasting. There are several other work with a similar setting [1][2].

We really appreciate the reviewer for raising the question on the semi-supervised setting, which makes us realize that there is confusion in our terminology. We hope the clarification above can clear the confusions. Additionally, we also attch more experiment results in a very typical supervised setting where the only difference with the origianl setting is that the entire graph is used in the training phase.

- [1] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial-
temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 922–929, 2019
- [2] Ailin Deng and Bryan Hooi. Graph neural network-based anomaly detection in multivariate time series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 4027–4035,2021.

> Concern 4: "Because of the above points, I think that there is a gap between what the paper claims to solve and what the paper actually solves. It might be more appropriate if the paper has been framed as providing generalization bound for graph attention network on semi-supervised regression tasks."

We also attach more experiment results that include the forecasting performance in a classic supervised setting and the comparison between our model and the SOTA GAT-based multivariate time series forecasting model. We hope these and the above explanation are enough to clarify that there is no gap between what the paper claims to solve and what the paper actually solves.

> Question: "How did you construct the graph for the stock pricing dataset when you vary the node degrees? Is it kNN graph?"

As our above answer to concern 2 mentioned, the graph is not manually constructed. It is learned by the model using the attention mechanism and sparsified by the top K control.

---
We greatly appreciate Reviewer yXLj for the positive comments and constructive questions. We have addressed all the points raised by the reviewer. Please let us know if there are other questions or concerns. Thank you!