# Time Series Generative Adversarial Networks. NIPS 2019.

[toc]

## Abstract

A good generative model for time-series data should preserve *temporal dynamics*, in the sense that new sequences respect the original relationships between variables across time. The authors propose a novel framework for generating realistic time series data that combines the flexibility of the unsupervised paradigm with the control afforded by supervised training. Through a learned embedding space jointly optimized with both supervised and adversarial objectives, the authors encourage the network to adhere to the dynamics of the training data during sampling.

## Introduction

The temporal setting poses a unique challenge to generative modeling. A model is not only tasked with capturing the distributions of features within each time point, it should also capture the potentially complex dynamics of those variables across time. In modeling multivariate sequential data $\mathbf{x}_{1:T} = (\mathbf{x}_{1}, \ldots, \mathbf{x}_{T})$ , we wish to accurately capture the conditional distribution $p(\mathbf{x}_{t} |\mathbf{x}_{1:t−1})$ of temporal transitions.

### Contribution

First, in addition to the unsupervised adversarial loss on both real and synthetic sequences, we introduce a stepwise supervised loss using the original data as supervision, thereby explicitly encouraging the model to capture the stepwise conditional distributions in the data.

Second, the authors introduce an embedding network to provide a reversible mapping between features and latent representations, thereby reducing the high-dimensionality of the adversarial learning space. This capitalizes on the fact the temporal dynamics of even complex systems are often driven by fewer and lower-dimensional factors of variation.

> "train on synthetic, test on real (TSTR)" framework to the sequence prediction task, the authors evaluate how well the generated data preserves the predictive characteristics of the original.

## Related Work

Autoregressive recurrent networks trained via the maximum likelihood principle are prone to potentially large prediction errors when performing multi-step sampling, due to the discrepancy between closed-loop training (i.e. conditioned on ground truths) and open-loop inference (i.e. conditioned on previous guesses).

Multiple studies have straightforwardly inherited the GAN framework within the temporal setting. The first (C-RNN-GAN) directly applied the GAN architecture to sequential data, using LSTM networks for generator and discriminator.

Representation learning in the time-series setting primarily deals with the benefits of learning compact encodings for the benefit of downstream tasks such as prediction, forecasting, and classification.

## Problem Definition

Let $\mathcal{S}$ be a vector space of static features, $\mathcal{X}$ of temporal features, and let $\mathbf{S} \in \mathcal{S}, \mathbf{X} \in \mathcal{X}$ be random vectors that can be instantiated with specific values denoted $\mathbf{s}$ and $\mathbf{x}$. We consider tuples of theform $(\mathbf{S}, \mathbf{X}_{1:T} )$ with some joint distribution $p$. In the training data, let individual samples be indexed by $n \in {1,\ldots,N}$, so we can denote the training dataset $\mathcal{D} = {(\mathbf{s}_{n}, \mathbf{x}_{n, 1:T_{n}})}^{N}_{n=1}$.

The goal is to use training data $\mathcal{D}$ to learn a density $\hat{p}(\mathbf{S}, \mathbf{X}_{1:T})$ that best approximates $p(\mathbf{S}, \mathbf{X}_{1:T})$. Therefore we additionally make use of the autoregressive decomposition of the joint $p(\mathbf{S}, \mathbf{X}_{1:T}) = p(\mathbf{S}) \prod_{t} p(\mathbf{X}_{t} | \mathbf{S}, \mathbf{X}_{1:t-1})$ to focus specifically on the conditionals $\hat{p}(\mathbf{X}_{t} | \mathbf{S}, \mathbf{X}_{1:t-1})$ that best approximates $p(\mathbf{X}_{t} | \mathbf{S}, \mathbf{X}_{1:t-1})$ at any time $t$.

**Two objectives**. Importantly, this breaks down the sequence-level objective (matching the joint distribution) into a series of stepwise objectives (matching the conditionals).

The first is global

$$
\tag{1}
\min_{\hat{p}} D\left(p(\mathbf{S}, \mathbf{X}_{1:T}) || \hat{p}(\mathbf{S}, \mathbf{X}_{1:T}) \right)
$$

where $D$ is some appropriate measure of distance between distributions. The second is local,

$$
\tag{2}
\min_{\hat{p}} D\left(p(\mathbf{X}_{t} | \mathbf{S}, \mathbf{X}_{1:t-1}) || \hat{p}(\mathbf{X}_{t} | \mathbf{S}, \mathbf{X}_{1:t-1}) \right)
$$

for any $t$. The former takes the form of the Jensen-Shannon divergence. The latter takes the form of the Kullback-Leibler divergence.

## Methodology

TimeGAN consists of four network components: an embedding function, recovery function, sequence generator, and sequence discriminator. The key insight is that the autoencoding components (first two) are trained jointly with the adversarial components (latter two), such that TimeGAN simultaneously learns to encode features, generate representations, and iterate across time.

### Embedding and Recovery Functions

Let $\mathcal{H}_{\mathcal{S}}$, $\mathcal{H}_{\mathcal{X}}$ denote the latent vector spaces corresponding to feature spaces $\mathcal{S}$, $\mathcal{X}$. Then the embedding function $e : \mathcal{S} \times \prod_{t} \mathcal{X} \rightarrow \mathcal{H}_{\mathcal{S}} \times \prod_{t} \mathcal{H}_{\mathcal{X}}$ takes static and temporal features to their latent codes $\mathbf{h}_{\mathcal{S}}$, $\mathbf{h}_{1:T} = e(\mathbf{s}, \mathbf{x}_{1:T})$. Here, the authors implement $e$ via a **recurrent network**

$$
\tag{3}
\mathbf{h}_{\mathcal{S}} = e_{\mathcal{S}}(\mathbf{s}) \\
\mathbf{h}_{t} = e_{\mathcal{X}}(\mathbf{h}_{\mathcal{S}}, \mathbf{h}_{t-1}, \mathbf{x}_{t})
$$

where, $e_{\mathcal{S}}: \mathcal{S} \rightarrow \mathcal{H}_{\mathcal{S}}$ is an embedding network for static feature and $e_\mathcal{X}: \mathcal{H}_{\mathcal{S}} \times \mathcal{H}_{\mathcal{X}} \times \mathcal{X} \rightarrow \mathcal{H}_{\mathcal{X}}$ is a recurrent embedding network for temporal features.

In the opposite direction, the recovery function $r: \mathcal{H}_{\mathcal{S}} \times \prod_{t} \mathcal{H}_{\mathcal{X}} \rightarrow \mathcal{S} \times \prod_{t} \mathcal{X}$ takes static and temporal codes back to their feature representations $\tilde{\mathbf{s}}$, $\tilde{\mathbf{x}}_{1:T} = r(\mathbf{h}_{\mathcal{S}}, \mathbf{h}_{1:T})$. Here, the authors implement $r$ via a **feedforward network**

$$
\tag{4}
\tilde{\mathbf{s}} = r_{\mathcal{S}}(\mathbf{h}_{s}) \\
\tilde{\mathbf{x}}_{t} = r_{\mathcal{X}}(\mathbf{h}_{t})
$$

where, $r_{\mathcal{S}}: \mathcal{H}_{\mathcal{S}} \rightarrow \mathcal{S}$ and $r_{\mathcal{X}}: \mathcal{H}_{\mathcal{X}} \rightarrow \mathcal{X}$ are recovery networks for static and temporal embeddings.

> Note that the embedding and recovery functions can be parameterized by any architecture of choice, with the only stipulation being that they be autoregressive and obey causal ordering (i.e. output(s) at each step can only depend on preceding information). For example, it is just as possible to implement the former with temporal convolutions, or the latter via an attention-based decode

### Sequence Generator and Discriminator

> Instead of producing synthetic output directly in feature space, the generator first outputs into the embedding space.

Let $\mathcal{Z}_{\mathcal{S}}$, $\mathcal{Z}_{\mathcal{X}}$ denote vector spaces over which known distributions are defined, and from which random vectors are drawn as input for generating into $\mathcal{H}_{\mathcal{S}}$, $\mathcal{H}_{\mathcal{X}}$. Then the generating function $g : \mathcal{Z}_{\mathcal{S}} \times \prod_{t} \mathcal{Z}_{\mathcal{X}} \rightarrow \mathcal{H}_{\mathcal{S}} \times \prod_{t} \mathcal{H}_{\mathcal{X}}$ takes a tuple of static and temporal random vectors to synthetic latent codes $\mathbf{\hat{h}}_{\mathcal{S}}$, $\mathbf{\hat{h}}_{1:T} = g(\mathbf{z}_{\mathcal{S}}, \mathbf{z}_{1:T})$. Here, the authors implement $e$ via a **recurrent network**

$$
\tag{5}
\mathbf{\hat{h}}_{\mathcal{S}} = g_{\mathcal{S}}(\mathbf{z}_{\mathcal{S}}) \\
\mathbf{\hat{h}}_{t} = g_{\mathcal{X}}(\mathbf{\hat{h}}_{\mathcal{S}}, \mathbf{\hat{h}}_{t-1}, \mathbf{z}_{t})
$$

where $g_{\mathcal{S}}: \mathcal{Z}_\mathcal{S} \rightarrow \mathcal{H}_\mathcal{S}$ is an generator network for static features, and $g_{\mathcal{X}}: \mathcal{H}_\mathcal{S} \times \mathcal{H}_\mathcal{X} \times \mathcal{Z}_\mathcal{X} \rightarrow \mathcal{H}_\mathcal{X}$ is a recurrent generator for temporal features. Random vector $\mathbf{z}_\mathcal{S}$ can be sampled from a distribution of choice, and $\mathbf{z}_{t}$ follows a stochastic process; here we use the Gaussian distribution and Wiener process respectively.

Finally, the discriminator also operates from the embedding space. The discrimination function $d : \mathcal{H}_\mathcal{S} \times \prod_{t} \rightarrow [0, 1] \times \prod_{t}[0, 1]$ receives the static and temporal codes, returning classifications $\tilde{y}_{\mathcal{S}}$, $\tilde{y}_{1:T} = d(\mathbf{\tilde{h}}_{\mathcal{S}}, (\mathbf{\tilde{h}}_{1:T})$. The $\mathbf{\tilde{h}}_{*}$ notation denotes either real ($\mathbf{h}_{*}$) or synthetic ($\mathbf{\hat{h}}_{*}$) embeddings; similarly, the $\tilde{y}_{*}$ notation denotes classifications of either real ($y_{*}$) or synthetic ($\hat{y}_{*}$) data. Here, the authors implement $d$ via a bidirectional recurrent network with a feedforward output layer,

$$
\tag{6}
\tilde{y}_{\mathcal{S}} = d_{\mathcal{S}}(\mathbf{\tilde{h}}_{\mathcal{S}}) \\
\tilde{y}_{t} = d_{\mathcal{X}}(\mathbf{\overleftarrow{u}}_{t}, \mathbf{\overrightarrow{u}}_{t})
$$

where, $\mathbf{\overrightarrow{u}}_{t} = \overrightarrow{c}_{\mathcal{X}}(\mathbf{\tilde{h}}_{\mathcal{S}}, \mathbf{\tilde{h}}_{t}, \mathbf{\overrightarrow{u}}_{t-1})$ and $\mathbf{\overleftarrow{u}}_{t} = \overrightarrow{c}_{\mathcal{X}}(\mathbf{\tilde{h}}_{\mathcal{S}}, \mathbf{\tilde{h}}_{t}, \mathbf{\overleftarrow{u}}_{t+1})$ respectively denote the sequences of orward and backward hidden states, $\overrightarrow{c}_{\mathcal{X}}$, $\overleftarrow{c}_{\mathcal{X}}$ are recurrent functions, and $d_{\mathcal{S}}$, $d_{\mathcal{X}}$ are output layer classification functions.

Figure 1: (a) Block diagram of component functions and objectives. (b) Training scheme; solid lines indicate forward propagation of data, and dashed lines indicate backpropagation of gradients.

## Jointly Learning to Encode, Generate, and Iterate

First, purely as a reversible mapping between feature and latent spaces, the embedding and recovery functions should enable accurate reconstructions $\tilde{\mathbf{s}}$, $\tilde{\mathbf{x}}_{1:T}$ of the original data $\mathbf{s}$, $\mathbf{x}_{1:T}$ from their latent representations $\mathbf{h}_{\mathcal{S}}$, $\mathbf{h}_{1:T}$. The first objective function is the *reconstruction loss*

$$
\tag{7}
\mathcal{L}_{\text{R}} = \mathbb{E}_{\mathbf{s}, \mathbf{x}_{1:T} \sim p} [||\mathbf{s} - \mathbf{\tilde{s}}||_{2} + \sum_{t}||\mathbf{x}_{t} - \mathbf{\tilde{x}}_{t}||]
$$

Gradients are then computed on the *unsupervised loss*. This is as one would expect-that is, to allow maximizing (for the discriminator) or minimizing (for the generator) the likelihood of providing correct classifications $\hat{y}_{\mathcal{S}}$, $\hat{y}_{1:T}$ for both the training data $\mathbf{h}_{\mathcal{S}}$, $\mathbf{h}_{1:T}$ as well as for synthetic output $\mathbf{\hat{h}}_{\mathcal{S}}$, $\mathbf{\hat{h}}_{1:T}$ from the generator

$$
\tag{8}
\mathcal{L}_{\text{U}} = \mathbb{E}_{\mathbf{s}, \mathbf{x}_{1:T} \sim p} [\text{log} \: y_{\mathcal{S}} + \sum_{t} \text{log} \: y_{t}] + \mathbb{E}_{\mathbf{s}, \mathbf{x}_{1:T} \sim \hat{p}} [\text{log} (1 - \hat{y}_{\mathcal{S}}) + \sum_{t} \text{log}(1 - \hat{y}_{t})]
$$

The authors also train in closed-loop mode, where the generator receives sequences of embeddings of actual data $\mathbf{h}_{1:t−1}$ (i.e. computed by the embedding network) to generate the next latent vector. Gradients can now be computed on a loss that captures the discrepancy between distributions $p(\mathbf{H}_{t} | \mathbf{H}_{\mathcal{S}}, \mathbf{H}_{1:t-1})$ and $\hat{p}(\mathbf{H}_{t} | \mathbf{H}_{\mathcal{S}}, \mathbf{H}_{1:t-1})$. Applying maximum likelihood yields the familiar supervised loss

$$
\tag{9}
\mathcal{L}_{\text{S}} = \mathbb{E}_{\mathbf{s}, \mathbf{x}_{1:T} \sim p} [\sum_{t} ||\mathbf{h}_{t} - g_{\mathcal{X}}(\mathbf{h}_{\mathcal{S}}, \mathbf{h}_{t-1}, \mathbf{z}_{t})||_2]
$$

where $g_{\mathcal{X}}(\mathbf{h}_{\mathcal{S}}, \mathbf{h}_{t-1}, \mathbf{z}_{t})$ approximates $\mathbb{E}_{\mathbf{z}_{t} \sim \mathcal{N}}[\hat{p}(\mathbf{H}_{t} | \mathbf{H}_{\mathcal{S}}, \mathbf{H}_{1:t-1}, \mathbf{z}_{t})]$ with one sample $\mathbf{z}_{t}$-as is standard in stochastic gradient descent.

> In sum, at any step in a training sequence, one assess the difference between the actual next-step latent vector (from the embedding function) and synthetic next-step latent vector (from the generator-conditioned on the actual historical sequence of latents). While $\mathcal{L}_{\text{U}}$ pushes the generator to create realistic sequences (evaluated by an imperfect adversary), $\mathcal{L}_{\text{S}}$ further ensures that it produces similar stepwise transitions (evaluated by ground-truth targets).

**Optimization**. Figure 1(b) illustrates the mechanics of our approach at training. Let $\theta_{e}$, $\theta_{r}$, $\theta_{g}$, $\theta_{d}$ respectively denote the parameters of the embedding, recovery, generator, and discriminator networks. The first two components are trained on both the reconstruction and supervised losses

$$
\tag{10}
\min_{\theta_{e}, \theta_{r}}(\lambda\mathcal{L}_{\text{S}} + \mathcal{L}_{\text{R}})
$$

Next, the generator and discriminator networks are trained adversarially as follows

$$
\tag{11}
\min_{\theta_{g}}(\eta\mathcal{L}_{\text{S}} + \max_{\theta_{d}} \mathcal{L}_{\text{U}})
$$

Figure 2: (a) TimeGAN instantiated with RNNs, (b) C-RNN-GAN, and (c) RCGAN. Solid lines denote function application, dashed lines denote recurrence, and orange lines indicate loss computation.

## Experiments

Benchmarks and Evaluation. We compare TimeGAN with RCGAN and C-RNN-GAN, the two most closely related methods. For purely autoregressive approaches, the authors compare against RNNs trained with teacher-forcing (T-Forcing) as well as professor-forcing (P-Forcing). For additional comparison, the authors consider the performance of WaveNet as well as its GAN counterpart WaveGAN. To assess the quality of generated data, the authors observe three desiderata: (1) diversity-samples should be distributed to cover the real data; (2) fidelity-samples should be indistinguishable from the real data; and (3) usefulness-samples should be just as useful as the real data when used for the same predictive purposes (i.e. train-on-synthetic, test-on-real).

(1) **Visualization**. We apply t-SNE and PCA analyses on both the original and synthetic datasets (flattening the temporal dimension).

(2) **Discriminative Score**. For a quantitative measure of similarity, the authors train a post-hoc time-series classification model (by optimizing a 2-layer LSTM) to distinguish between sequences from the original and generated datasets.

(3) **Predictive Score**. In order to be useful, the sampled data should inherit the predictive characteris-tics of the original. In particular, the authors expect TimeGAN to excel in capturing conditional distributions over time. Therefore, using the synthetic dataset, the authors train a post-hoc sequence-prediction model (by optimizing a 2-layer LSTM) to predict next-step temporal vectors over each input sequence. Then, the authors evaluate the trained model on the original dataset. Performance is measured in terms of the mean absolute error (MAE).

Figure 3: t-SNE visualization on Sines (1st row) and Stocks (2nd row). Each column provides the visualization for each of the 7 benchmarks. Red denotes original data, and blue denotes synthetic.