---
# System prepended metadata

title: 時間序列資料集
tags: [ML, ML / 時間序列, 資料集, 時間序列]

---

時間序列資料集
===
###### tags: `ML / 時間序列`
###### tags: `ML`, `時間序列`, `資料集`

<br>

[TOC]

<br>

## 出生人口預估
- [台灣出生死亡人口數 & 六十年趨勢](https://morrisjfwong.com/blog/台灣出生死亡人口數-六十年趨勢/)
    ![](https://i.imgur.com/yo9PFMx.png)
- [國發會人口圖表](https://ct.org.tw/html/news/3-3.php?cat=10&article=1354805)
    ![](https://i.imgur.com/b24QRJB.png)

<br>

<hr>

<br>

## 高速公路車流量
- [高公局分析90億筆ETC大資料，預測高速公路何時塞車](https://www.ithome.com.tw/article/98069)
    ![](https://i.imgur.com/Y7RnieA.png)
    其他因素：
    - 交通特性
    - 觀光遊憩
    - 天氣
    - 網路輿情
    - 閉路電視
    - 客運與臺鐵人潮
    - 景點遊客人數
    - 住宿率
    - 天氣預報
    - 關鍵字搜尋頻率
    [人事行政局104年行事曆](https://www.319papago.idv.tw/holiday/2015-hr/2015_HR.html)

<br>

<hr>

<br>

## [[Kaggle] Air Passengers](https://www.kaggle.com/rakannimer/air-passengers)
> 每月搭機旅客人數

### 資料分析
- ### [TimeSeries Analysis 📈A Complete Guide 📚](https://www.kaggle.com/andreshg/timeseries-analysis-a-complete-guide)
- ### [Day 24：銷售量預測(2) -- 『時間序列分析』技巧篇](https://ithelp.ithome.com.tw/articles/10195635)
    - 航空公司每月乘客人數資料
        ![](https://i.imgur.com/8uasF3r.png =50%x)
    - 1994~2013年台積電的每日股價資料
        ![](https://i.imgur.com/5Lq1Dgj.png =50%x)

<br>

### read_csv
- ### 載入 CSV (預設參數)
    ```python=
    df = pandas.read_csv('air-passengers.csv')
    ```
    ![](https://i.imgur.com/qVHpcu2.png)
    - 每年 7,8 月是航運最高峰
    - 每年 11,12月或是 2月是航運最低峰
    - 隨著每年航運量提昇，振幅也加大
        - 100人的10％，就是 ±10
        - 500人的10％，就是 ±50

- ### 載入 CSV，並只保留指定欄位
    ```python=
    import pandas
    import matplotlib.pyplot as plt

    df = pandas.read_csv('air-passengers.csv', usecols=[1])
    plt.plot(df)
    plt.show()
    ```
    ![](https://i.imgur.com/J0EzAsQ.png) ![](https://i.imgur.com/NMd4d4h.png)
    
- ### 載入 CSV，並將時間軸設為 raw label
    ```python=
    import pandas
    import matplotlib.pyplot as plt
    
    df = pandas.read_csv('air-passengers.csv', index_col='Month')
    plt.plot(df)
    plt.show()
    ```
    ![](https://i.imgur.com/xECDhHi.png) ![](https://i.imgur.com/TTcKjJi.png)
    
- ### 使用 1 階差分
    ```python=
    import pandas
    import matplotlib.pyplot as plt
    
    df = pandas.read_csv('air-passengers.csv', usecols=[1])

    # 1階差分
    df1 = df.diff(1)
    plt.plot(df1)
    plt.show()
    ```
    ![](https://i.imgur.com/c5gXMKV.png)

- ### 繪製訓練資料＆測試資料
    ```python=
    import pandas

    df_train = pandas.read_csv('air-passengers-train.csv', index_col=[0])
    df_test = pandas.read_csv('air-passengers-test.csv', index_col=[0])

    # plot the training data
    plt.figure(figsize=(12, 5), dpi=100)
    plt.plot(df_train, label='train')

    # prepare the future data
    ## x values
    month = [df_train.index[-1]]
    month.extend(df_test.index)

    ## y values
    passengers = [df_train["#Passengers"][-1]]
    passengers.extend(df_test["#Passengers"])
    
    df_future = pandas.DataFrame({
        'passengers': passengers
    }, index=month)
    plt.plot(df_future, label='test')

    plt.legend(loc='upper left')
    plt.xticks([str(y) + '-01' for y in range(1949, 1962, 1)])
    plt.grid()
    plt.show()
    ```
    [![](https://i.imgur.com/IrfvWqG.png)](https://i.imgur.com/IrfvWqG.png)

<br>

### 預測
- ### [[LSTM] Time Series Prediction with LSTM Recurrent Neural Networks in Python with Keras](https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/)
    - LSTM Network for Regression
        - Train Score: 22.93 RMSE
        - Test Score: 47.53 RMSE
    - LSTM for Regression Using the Window Method
        - Train Score: 24.19 RMSE
        - Test Score: 58.03 RMSE
    - LSTM for Regression with Time Steps
        - Train Score: 23.69 RMSE
        - Test Score: 58.88 RMSE
    - LSTM with Memory Between Batches
        - Train Score: 20.74 RMSE
        - Test Score: 52.23 RMSE
    - Stacked LSTMs with Memory Between Batches
        - Train Score: 20.49 RMSE
        - Test Score: 56.35 RMSE



<br>

<hr>

<br>

## [[Kaggle] Bike Sharing Demand](https://www.kaggle.com/c/bike-sharing-demand/data)

<br>

<hr>

<br>

## [Kaggle] 天氣
### [Tabular Playground Series - Jul 2021](https://www.kaggle.com/c/tabular-playground-series-jul-2021)
> 根據溫度、濕度、感測器，預測空污值（一氧化碳、苯、氮氧化物）

<br>

<hr>

<br>

## [Kaggle] 銷售額
### [Sales Time Series Forecasting](https://www.kaggle.com/c/sales-time-series-forecasting-ca-afcs2020/overview)

### [Store Sales - Time Series Forecasting](https://www.kaggle.com/c/store-sales-time-series-forecasting/overview)

### [M5 Forecasting - Accuracy](https://www.kaggle.com/c/m5-forecasting-accuracy)
> 預測未來 28 天的每日銷售額

### [M5 Forecasting - Uncertainty](https://www.kaggle.com/c/m5-forecasting-uncertainty)
>預測未來 28 天的每日銷售額，並為這些預測做出不確定性估計

### [Walmart Recruiting - Store Sales Forecasting](https://www.kaggle.com/c/walmart-recruiting-store-sales-forecasting/overview)

### [Rossmann Store Sales](https://www.kaggle.com/c/rossmann-store-sales)

<br>

<hr>

<br>

## [Kaggle] 時間-項目 數據
### [Web Traffic Time Series Forecasting](https://www.kaggle.com/c/web-traffic-time-series-forecasting/overview)
> 預測大約 145,000 篇維基百科文章的網絡流量（每日瀏覽量）

<br>

<hr>

<br>

## [[Azure] 自行車共用服務的出租需求](https://docs.microsoft.com/zh-tw/azure/machine-learning/tutorial-automated-ml-forecast)
- MachineLearningNotebooks
    - how-to-use-azureml
        - automated-machine-learning
            - [forecasting-bike-share](https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml/automated-machine-learning/forecasting-bike-share)
                - [bike-no.csv](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/automated-machine-learning/forecasting-bike-share/bike-no.csv)
- 欄位說明
    - [Python Spark MLlib 之決策樹迴歸分析](https://www.itread01.com/content/1544348007.html)
        > 均方誤差RMSE=117.804043648
    - [[Python嗯~機器學習]---用python3來分析共享單車投放量](https://www.twblogs.net/a/5c3f47cabd9eee35b3a66981)
        > R-squared: 0.39423906942549125
        > RMSE: 142.08580203044002
    - [Final Project - Data analysis and visualization in R](https://rstudio-pubs-static.s3.amazonaws.com/98994_613e4a13f448452c937233f146f80d59.html)
- 專有名詞
    - [讓我們看雲去](http://mail.atm.ncu.edu.tw/~hong/atmhmpg/clouds/ahead.htm)
        - 天空小於十分之一的面積有雲時，稱為晴朗（Clear, CLR）
        - 雲覆蓋天空十分之一到十分之五時，稱為疏雲（Scattered, SCT, partly cloudy）
        - 天空的十分之六到十分之九覆蓋有雲時，稱為裂雲（Broken, BRN, cloudy
        - 天空超過十分之九覆蓋著雲時，稱為陰天 （Overcast, OVC）
    - [scattered cloud - 疏雲 - 國家教育研究院雙語詞彙](https://terms.naer.edu.tw/detail/986615/)
        - scattered cloud 疏雲

<br>

## [[Azure] 60 萬筆任意 Web 服務流量的記錄](https://docs.microsoft.com/zh-tw/azure/data-explorer/time-series-analysis)

| TimeStamp                | BrowserVer             | OsVer      | Country        |
|--------------------------|------------------------|------------|----------------|
| 2016-08-21T00:00:10.625Z | Chrome 51.0            | Windows 10 | United States  |
| 2016-08-21T00:00:31.295Z | Internet Explorer 11.0 | Windows 10 | United States  |
| 2016-08-21T00:00:42.879Z | Chrome 52.0            | Windows 10 | United States  |
| 2016-08-21T00:00:47.017Z | Chrome 51.0            | Windows 10 | United States  |
| 2016-08-21T00:01:06.642Z | Chrome 52.0            | Windows 7  | United Kingdom |
| 2016-08-21T00:02:02.893Z | Internet Explorer 11.0 | Windows 7  |                |
| 2016-08-21T00:02:59.909Z | Chrome 52.0            | Windows 7  | United Kingdom |
| 2016-08-21T00:03:03.315Z | Firefox 48.0           | Windows 10 | United States  |
| 2016-08-21T00:03:07.263Z | Firefox 48.0           | Windows 10 | United States  |
| 2016-08-21T00:03:13.471Z | Internet Explorer 11.0 | Windows 7  |                |

[![](https://i.imgur.com/Ajf7R2M.png)](https://i.imgur.com/Ajf7R2M.png)


<br>

<hr>

<br>

## [[UCI] Air Quality Data Set](https://archive.ics.uci.edu/ml/datasets/Air+Quality)
> 資料來源：[A Multivariate Time Series Guide to Forecasting and Modeling (with Python codes)](https://www.analyticsvidhya.com/blog/2018/09/multivariate-time-series-guide-forecasting-modeling-python-codes/)
> #多變量時間序列
> #`from statsmodels.tsa.vector_ar.var_model import VAR`

![](https://i.imgur.com/VFIcVey.png)
| Date      | Time     | CO(GT) | PT08.S1(CO) | NMHC(GT) | C6H6(GT) | PT08.S2(NMHC) | NOx(GT) | PT08.S3(NOx) | NO2(GT) | PT08.S4(NO2) | PT08.S5(O3) | T    | RH   | AH     |
|-----------|----------|--------|-------------|----------|----------|---------------|---------|--------------|---------|--------------|-------------|------|------|--------|
| 3/10/2004 | 18:00:00 | 2.6    | 1360        | 150      | 11.9     | 1046          | 166     | 1056         | 113     | 1692         | 1268        | 13.6 | 48.9 | 0.7578 |
| 3/10/2004 | 19:00:00 | 2      | 1292        | 112      | 9.4      | 955           | 103     | 1174         | 92      | 1559         | 972         | 13.3 | 47.7 | 0.7255 |
| 3/10/2004 | 20:00:00 | 2.2    | 1402        | 88       | 9.0      | 939           | 131     | 1140         | 114     | 1555         | 1074        | 11.9 | 54.0 | 0.7502 |
| 3/10/2004 | 21:00:00 | 2.2    | 1376        | 80       | 9.2      | 948           | 172     | 1092         | 122     | 1584         | 1203        | 11.0 | 60.0 | 0.7867 |
| 3/10/2004 | 22:00:00 | 1.6    | 1272        | 51       | 6.5      | 836           | 131     | 1205         | 116     | 1490         | 1110        | 11.2 | 59.6 | 0.7888 |
| 3/10/2004 | 23:00:00 | 1.2    | 1197        | 38       | 4.7      | 750           | 89      | 1337         | 96      | 1393         | 949         | 11.2 | 59.2 | 0.7848 |
| 3/11/2004 | 0:00:00  | 1.2    | 1185        | 31       | 3.6      | 690           | 62      | 1462         | 77      | 1333         | 733         | 11.3 | 56.8 | 0.7603 |
| 3/11/2004 | 1:00:00  | 1      | 1136        | 31       | 3.3      | 672           | 62      | 1453         | 76      | 1333         | 730         | 10.7 | 60.0 | 0.7702 |
| 3/11/2004 | 2:00:00  | 0.9    | 1094        | 24       | 2.3      | 609           | 45      | 1579         | 60      | 1276         | 620         | 10.7 | 59.7 | 0.7648 |


<br>

<hr>

<br>

## [tensorflow] The weather dataset
> 資訊來源：[Time series forecasting](https://www.tensorflow.org/tutorials/structured_data/time_series)
> 資料下載：https://storage.googleapis.com/tensorflow/tf-keras-datasets/jena_climate_2009_2016.csv.zip

<br>

### [欄位說明](https://keras.io/examples/timeseries/timeseries_weather_forecasting/)
| Index | Features| Format| Description|
|-------|---------|-------|------------|
|  1| Date Time       | 01.01.2009 00:10:00 | Date-time reference|
|  2| p (mbar)        | 996.52| The pascal SI derived unit of pressure used to quantify internal pressure. Meteorological reports typically state atmospheric pressure in millibars. |
|  3| T (degC)        | -8.02 | Temperature in Celsius |
|  4| Tpot (K)        | 265.4 | Temperature in Kelvin|
|  5| Tdew (degC)     | -8.9| Temperature in Celsius relative to humidity. Dew Point is a measure of the absolute amount of water in the air, the DP is the temperature at which the air cannot hold all the moisture in it and water condenses. |
|  6| rh (%)          | 93.3| Relative Humidity is a measure of how saturated the air is with water vapor, the %RH determines the amount of water contained within collection objects. |
|  7| VPmax (mbar)    | 3.33| Saturation vapor pressure|
|  8| VPact (mbar)    | 3.11| Vapor pressure |
|  9| VPdef (mbar)    | 0.22| Vapor pressure deficit |
| 10| sh (g/kg)       | 1.94| Specific humidity|
| 11| H2OC (mmol/mol) | 3.12| Water vapor concentration|
| 12| rho (g/m ** 3)  | 1307.75 | Airtight |
| 13| wv (m/s)        | 1.03| Wind speed |
| 14| max. wv (m/s)   | 1.75| Maximum wind speed |
| 15| wd (deg)        | 152.3 | Wind direction in degrees|

<br>

### read_csv
```python=
import os
import pandas as pd

zip_path = tf.keras.utils.get_file(
    origin='https://storage.googleapis.com/tensorflow/tf-keras-datasets/jena_climate_2009_2016.csv.zip',
    fname='jena_climate_2009_2016.csv.zip',
    extract=True)
csv_path, _ = os.path.splitext(zip_path)

df = pd.read_csv(csv_path)
df
```
![](https://i.imgur.com/FRQBv33.png)


<br>

### 預測
- [[tensorflow] Time series forecasting](https://www.tensorflow.org/tutorials/structured_data/time_series)
- [[Keras] Timeseries forecasting for weather prediction](https://keras.io/examples/timeseries/timeseries_weather_forecasting/)
    > 欄位說明
- [Python 深度學習之循環神經網絡的高級用法](https://pythonmana.com/2021/08/20210816191316709q.html)
    > 《Deep Learning with Python》(第二版，François Chollet 著) 

<br>

<hr>

<br>

## [Shampoo Sales Dataset](https://machinelearningmastery.com/persistence-time-series-forecasting-with-python/)
![](https://i.imgur.com/KtsNzgs.png)

<br>

<hr>

<br>


## FEDOT
- [FEDOT/cases/data/](https://github.com/nccr-itmo/FEDOT/tree/master/cases/data)
    - classification
    - regression
    - time-series
- [Fedot time-series forecasting electricity case](https://github.com/ITMO-NSS-team/fedot_electro_ts_case) (風力發電 + 燃煤發電)
- [FEDOT Timeseries benchmarking](https://github.com/ITMO-NSS-team/Fedot-TS-Benchmark)
    - fred_case
    - river_level_forecasting
    - sensors
    - stock_price
    
   

<br>

<hr>

<br>

## [Sktime](https://pypi.org/project/sktime/)

### [[doc] datasets](https://www.sktime.org/en/stable/api_reference/datasets.html)
- `dir(sktime.datasets)`
    ```=
    'load_PBS_dataset',
    'load_UCR_UEA_dataset',
    'load_acsf1',
    'load_airline',
    'load_arrow_head',
    'load_basic_motions',
    'load_electric_devices_segmentation',
    'load_gun_point_segmentation',
    'load_gunpoint',
    'load_italy_power_demand',
    'load_japanese_vowels',
    'load_longley',
    'load_lynx',
    'load_macroeconomic',
    'load_osuleaf',
    'load_shampoo_sales',
    'load_unit_test',
    'load_uschange'
    ```
### load_airline
```python=
import pandas
y = sktime.datasets.load_airline() # pandas.core.series.Series
pandas.DataFrame(y, index=y.index)
```
![](https://i.imgur.com/d9ougyP.png)

<br>

<hr>

<br>

## 模擬時間序列資料
> #人工生成的時間序列 artificially generated time series
> #合成的時間序列 synthetic time series

### 來源資料
- ### [Time series forecasting with FEDOT. Guide](https://github.com/ITMO-NSS-team/fedot-examples/blob/main/notebooks/latest/3_intro_ts_forecasting.ipynb)
    > artificially generated time series

<br>

### 資料迭代
> sin + (cos1 + cos2) + random_noise

- ### sin
    ![](https://i.imgur.com/PH2Mx6k.png)
- ### cos = cos1 + cos2
    ![](https://i.imgur.com/TdxRs7U.png)
    ![](https://i.imgur.com/esYH6in.png =48%x) ![](https://i.imgur.com/lMZdRmX.png =48%x)
- ### sin + cos
    ![](https://i.imgur.com/eyefoNZ.png)
    ![](https://i.imgur.com/m0sgiKT.png)
- ### random_noise
    ![](https://i.imgur.com/gru9lkF.png)
- ### sin + cos + random_noise
    ![](https://i.imgur.com/GjzHiYM.png)


### 程式碼
```python=
import numpy as np
import matplotlib.pyplot as plt

def generate_synthetic_data(length: int = 2000, periods: int = 10):
    """
    The function generates a synthetic univariate time series

    :param length: the length of the array (even number)
    :param periods: the number of periods in the sine wave

    :return synthetic_data: an array without gaps
    """

    # First component
    sinusoidal_data = np.linspace(-periods * np.pi, periods * np.pi, length)
    sinusoidal_data = np.sin(sinusoidal_data)
    
    # Second component
    cos_1_data = np.linspace(-periods * np.pi/2, periods/2 * np.pi/2, int(length/2))
    cos_1_data = np.cos(cos_1_data) 
    cos_2_data = np.linspace(periods/2 * np.pi/2, periods * np.pi/2, int(length/2))
    cos_2_data = np.cos(cos_2_data)   
    cosine_data = np.hstack((cos_1_data, cos_2_data))
    
    random_noise = np.random.normal(loc=0.0, scale=0.1, size=length)

    # Combining a sine wave, cos wave and random noise
    synthetic_data = sinusoidal_data + cosine_data + random_noise
    return synthetic_data

# Get such numpy array
synthetic_time_series = generate_synthetic_data()

# We will predict 100 values in the future
len_forecast = 100

# Let's dividide our data on train and test samples
train_data = synthetic_time_series[:-len_forecast]
test_data = synthetic_time_series[-len_forecast:]

# Plot time series
plt.figure(figsize=(12, 5), dpi=100)

plt.plot(np.arange(0, len(train_data)), train_data, label = 'Train')
plt.plot(np.arange(len(train_data), len(train_data)+len(test_data)), test_data, label = 'Test')
plt.ylabel('Parameter', fontsize = 15)
plt.xlabel('Time index', fontsize = 15)
plt.legend(fontsize = 15)
plt.title('Synthetic time series')
plt.grid()
plt.show()
```
[![](https://i.imgur.com/TpTvzoS.png)](https://i.imgur.com/TpTvzoS.png)
- 若要匯出為 csv (不帶時間戳記)
    ```python=
    import pandas

    df = pandas.DataFrame(train_data, columns=['value'])
    df.to_csv("synthetic_time_series_train.csv")
    df = pandas.DataFrame(test_data, columns=['value'])
    df.to_csv("synthetic_time_series_test.csv")
    ```
- 若要匯出為 csv (帶時間戳記)
    ```python=
    import pandas

    df = pandas.DataFrame(train_data, columns=['value'])
    index = []
    y = 2000
    m = 1
    for i in range(len(df)):
        index.append('%d-%02d' % (y, m))
        m += 1
        if m > 12:
            y += 1
            m = 1

    df.index = index
    df.to_csv("synthetic_time_series_train.csv")

    # --------------------------------------------------

    df = pandas.DataFrame(test_data, columns=['value'])
    index = []
    for i in range(len(df)):
        index.append('%d-%02d' % (y, m))
        m += 1
        if m > 12:
            y += 1
            m = 1

    df.index = index
    df.to_csv("synthetic_time_series_test.csv")
    ```