# LLMForecaster

Contributors: @aiwalter

## Introduction

This enhacement proposal introduces the idea of having a conversational forecaster based on large language models (LLMs). The idea can be rolled out to a general `LLMBaseEstimator` in order to enable same functionality for other learning problems.

The idea is inspired by [`pandasai`](https://github.com/gventuri/pandas-ai/tree/main)

## Contents
[table of contents]

## Problem statement
Automated machine learning (AutoML) for forecasting is still not mature. Using LLMs enables a new possibility to evolve. Beginners in the field of forecasting often struggle to build advanced forecasting models including pipelining, transformers, grid search etc. and putting everything together.

## Description of proposed solution

### Interface
Scetch of an `LLMForecaster`:

from aeon.forecasting.llm import LLMForecaster
from aeon.datasets import load_airline
from aeon.forecasting.model_selection import temporal_train_test_split
from aeon.performance_metrics.forecasting import mean_absolute_error

y = load_airline()
y_train, y_test = temporal_train_test_split(y, test_size=12)

forecaster = LLMForecaster(query="""
    Build a univariate forecaster using exponential smoothing.
    Also add imputation, detrending and deseasonalization.""",
    ...)
forecaster.fit(y_train)
y_pred = forecaster.predict(fh=12)
mean_absolute_error(y_test, y_pred)

The `LLMForecaster.forecaster_` would hold the forecaster object.

### Functionality
The LLM should always return a single object inherited from `BaseForecaster` and not a sequence of separate steps/object. This could be achieved by adding additional context to the query given by user. Dependent on the LLM output, we could improve the query under the hood and ask the LMM again and again (`max_queries`) until the returned object is a standalone object inheriting from `BaseForecaster`. E.g. we could add `"Put this into a pipeline"` and it will put it into a `TransformedTargetForecaster`. Also possible exceptions due to LLM mistakes could be fed back to fix the issue. The `LLMForecaster` could support different LLMs by means of using `scikit-llm` (see below). We could start using ChatGPT.

### Training of LLM
Optionally enable training of LLMs by means of using [`scikit-llm`](https://github.com/iryna-kondr/scikit-llm) as a unified interface for different LLMs. Models could then be trained on recent `aeon` code. Not sure how computationally expensive this would be to include it into the `LLMForecaster` or not. Otherwise a model can be "trained" by initially giving it some `aeon` code and instructions before actually asking the query given by user.

### Auto ML
Optionally add additionally another layer of automation withauto ML functionality by means of
- infering the seasonality and adding it to the query
- adding meta data of the data to the query, e.g. `DataFrame.index`, `DataFrame.shape`, `DataFrame.dtypes`, `DataFrame.index.freq`, `DataFrame.head()` etc.

### Security
Optionally add a filter for confidential data exposure. There must be FOSS solutions already to e.g. track if there are customer names in the `DataFrame.index` before sending them to any external API.

## Motivation
Unlock low-code forecasting for beginners and quick experimentation for advanced users. Conversational forecasting could also boost attention for the whole `aeon` framework. E.g. `pandasai` was able to raise >6k GitHub stars in a very short time.

## Discussion and comparison of alternative solutions
Alternative solution is that the user gives the query directly to an LLM instead of using the `LLMForecaster`. This has however the disadvantage of e.g not returning a single forecaster but spaghetti code with isolation steps. Also possible errors when executing the code have to be played back manually to the LLM. Further, the LLM might not be up to date with recent `aeon` code.

## Detailed description of design and implementation of proposed solution
[prototypical implementation if applicable]