# Utility Theory

## Understanding Utility and Risk Preferences
In economics, **utility** is an abstract concept that represents the satisfaction or well-being an individual derives from consuming goods and services or engaging in activities. Understanding how individuals evaluate choices involving different levels of risk and reward is crucial for modeling economic behavior.

Utility theory provides the following key insights:
* **Risk Aversion:** Most individuals are risk-averse, meaning they prefer a certain outcome to a risky outcome with the same expected value.
* **Diminishing Marginal Utility:** The additional satisfaction (utility) derived from consuming one more unit of a good or service tends to decrease as consumption increases. This explains why individuals are willing to pay less than the expected value for a gamble.
* **Risk Premium:** The difference between the expected value of a risky outcome and the certain amount an individual would accept instead of taking the risk.

### Intuitive Example
Consider a coin toss game with the following rule:

* You receive $100 for HEAD and $0 for TAIL.

Question: What's the amount you'd be willing to pay to play this game?

* Initial Thought: $50, based on the expected outcome.
* Deeper Insight: Likely less than $50, to *be compensated for the risk* associated with the uncertainty of the outcome.

Conclusively, the amount paid to play the game is equal to *$50 minus Risk-Premium*.

## Formalizing Utility with Utility Functions
Utility functions help quantify an individual's preferences under uncertainty. They possess the following properties:

* **Monotonicity:** Utility functions increase with greater wealth or consumption, reflecting the general preference for more over less.
* **Concavity**: Concave utility functions represent *risk aversion*, as the marginal utility gained from additional consumption decreases.

### Key Takeaways
* Utility functions help us understand how individuals make choices under uncertainty.
* Risk aversion plays a significant role in decision-making.

## Calculating the Risk-Premium

### Certainty-Equivalent
The concavity of a utility function $U$ reveals a core principle: the expected utility of an uncertain outcome is less than the utility of its expected value. Mathematically, this is expressed as:

$$
\mathbb{E}[U(x)] < U(\mathbb{E}[x])
$$

From this, we introduce the **certainty-equivalent value** $x_{CE}$. This is the certain (non-random) amount of wealth an individual is willing to accept instead of taking a gamble. It's found by solving:

$$
U(x_{CE}) = \mathbb{E}[U(x)]
$$


The certainty-equivalent value graphically represents the point where a horizontal line drawn from the expected utility on the utility curve intersects the utility curve itself.

### Defining Risk-Premium
The **risk premium** quantifies how much an individual is willing to pay *over and above* the certainty-equivalent value to completely avoid the risk associated with an uncertain outcome. It has two common forms:

* **Absolute Risk Premium $\pi_A$:** The difference between the expected value of the gamble and its certainty-equivalent value. $$\pi_A = \mathbb{E}[x] - x_{CE}$$
* **Relative Risk Premium $\pi_R$:** The absolute risk premium expressed as a proportion of the gamble's expected value. $$\pi_R = \frac{\pi_A}{\mathbb{E}[x]} = 1 - \frac{x_{CE}}{\mathbb{E}[x]}$$

#### Approximating Risk-Premium
We can get a useful approximation for the risk premium using a Taylor series expansion. Let's denote the expected value of a gamble as $\bar{x}$ and its variance as $\sigma_x^2$.

Expanding the utility function $U(x)$ around $\bar{x}$:

$$
U(x) \approx U(\bar{x}) + U'(\bar{x}) \cdot (x-\bar{x}) + \frac12 U''(\bar{x}) \cdot (x-\bar{x})^2
$$

With the above, we can derive approximations for $x_{CE}$ and the expected utility, leading to a key relationship:

$$
U'(\bar{x}) \cdot (x_{CE}-\bar{x}) \approx \frac12 U''(\bar{x}) \cdot \sigma_x^2
$$

From this, we obtain the approximation for the absolute risk premium:

$$
\pi_A = \bar{x} - x_{CE} \approx -\frac12 \frac{U''(\bar{x})}{U'(\bar{x})} \cdot \sigma_x^2.
$$

### Absolute and Relative Risk-Aversion
* **Absolute Risk Aversion $A(x)$:**
* Defined as $A(x) = -U''(x) / U'(x)$.
* It measures how risk-averse an individual is in absolute terms. Our approximation shows: $$\pi_A \approx \frac12 A(\bar{x}) \cdot \sigma_x^2$$
* **Relative Risk Aversion $R(x)$:**
* Defined as $R(x) = -x \cdot U''(x) / U'(x)$. It measures risk aversion relative to an individual's wealth.
* This lets us approximate the relative risk premium: $$\pi_R = \frac{\pi_A}{\bar{x}} \approx -\frac12 \frac{U''(\bar{x}) \cdot \bar{x}}{U'(\bar{x})} \cdot \frac{\sigma_x^2}{\bar{x}^2} = \frac12 R(\bar{x}) \cdot \sigma_{\frac{x}{\bar{x}}}^2, $$ where $\sigma_{\frac{x}{\bar{x}}}^2$ is the variance of the payoff relative to its expected value.

## Commonly Used Utility Functions

1. **Linear Utility function**
* $U(x) = a+bx$ (where $a$ and $b$ are constants)
* **Implication:** Risk-neutrality. The individual is indifferent between a certain outcome and a gamble with the same expected value.
* **Key Properties:**
* Absolute Risk Aversion $A(x)=0$.
* Certainty-Equivalent: $x_{CE} = \bar{x}$ (equal to the expected value)

2. **Constant Absolute Risk Aversion (CARA)**
* $U(x) = \frac{1-e^{-ax}}{a}$ (where $a$ is a positive constant)
* **Implication:** Represents risk aversion with a constant degree of aversion to risk regardless of wealth level.
* **Key Properties**
* Absolute Risk Aversion: $A(x) = a$
* The coefficient $a$ directly indicates the level of risk aversion. Higher a means greater risk aversion.
* **Special Cases:**
* $a = 0$: Becomes linear, indicating risk neutrality.
* If the outcome $x$ follows a normal distribution $x \sim \mathcal{N}(\mu, \sigma^2)$, then \begin{align*}
\mathbb{E}[U(x)] &= \begin{cases}
\frac{1-e^{-a\mu+\frac{a^2\sigma^2}{2}}}{a} &\text{ for } a \neq 0 \\
\mu &\text{ for } a = 0
\end{cases} \\
x_{CE} &= \mu - \frac{a \sigma^2}{2}, \\
\pi_A &= \mu-x_{CE} = \frac{a \sigma^2}{2}.
\end{align*}

3. **Constant Relative Risk Aversion (CRRA)**
* $U(x) = \frac{x^{1-\gamma}-1}{1-\gamma}$ (where $\gamma \neq 1$)
* **Implication:** Captures risk aversion where aversion is proportional to wealth. Larger wealth often implies less risk aversion.
* **Key Properties:**
* Relative Risk Aversion: $R(x)=\gamma$
* Coefficient $\gamma$ controls the degree of risk aversion. Higher $\gamma$ implies greater risk aversion.
* **Special Cases:**
* $\gamma = 1$: Becomes the logarithmic utility function $U(x) = \ln(x)$.
* $\gamma = 0$: Simplifies to a risk-neutral form.
* If the log of the outcome $ln(x)$ follows a normal distribution $\ln x \sim \mathcal{N}(\mu, \sigma^2)$, then \begin{align*}
\mathbb{E}[U(x)] &= \begin{cases}
\frac{e^{\mu(1-\gamma)+\frac{\sigma^2}{2}(1-\gamma)^2} - 1}{1-\gamma} &\text{ for } \gamma \neq 0 \\
\mu &\text{ for } \gamma = 1
\end{cases} \\
x_{CE} &= e^{\mu + \frac{\sigma^2}{2}(1-\gamma)} \\
\pi_R &= 1 - \frac{x_{CE}}{\bar{x}} = 1 - e^{-\frac{\sigma^2 \gamma}{2}}
\end{align*}

## Optimization Under Uncertainty
When outcomes are uncertain, decision-making goes beyond simply maximizing the expected value. Utility theory helps us incorporate risk preferences. The goal shifts to maximizing:

* **Expected Utility:** The average utility derived across different possible outcomes, weighted by their probabilities: $\mathbb{E}[U(x)]$.
* **Certainty-Equivalent Value:** The risk-adjusted outcome; the sure amount that makes an individual indifferent to a risky prospect.

Let's illustrate this with two portfolio optimization problems:

### Portfolio Optimization with CARA Utility
#### Scenario
* You have $1 to invest for one year.
* **Choices:**
* Risky asset: Annual return follows a normal distribution $\mathcal{N}(\mu, \sigma^2)$.
* Riskless asset: Fixed annual return $r$.
* **Goal:** Find the optimal allocation $\pi$ (proportion of your investment in the risky asset) to maximize your expected CARA utility $U(W) = \frac{1 - e^{-aW}}{a}$.

#### Key Points
* Portfolio wealth W after one year will also follow a normal distribution: $$W \sim \mathcal{N}(1+r+\pi(\mu-r), \pi^2 \sigma^2)$$
* The expected utility of portfolio wealth: $$\mathbb{E}(U(W)) = 1+r+\pi(\mu-r) - \frac{a \pi^2 \sigma^2}{2}.$$
* The optimal fraction of the investment in the risky asset is $\pi^* = \frac{\mu-r}{a \sigma^2}$.

### Portfolio Optimization with CRRA Utility (Merton's Portfolio Problem)

#### Scenario
* Continuous investment horizon.
* **Choices**
* Risky asset: Price $S_t$ follows a geometric Brownian motion process: $$dS_t = \mu S_t dt + \sigma S_t dB_t$$
* Riskless asset: Grows at a constant rate $r$.
* **Goal:** Find the constant allocation $\pi$ to the risky asset that maximizes the expected CRRA utility $U(W_1) = \frac{W_1^{1-\gamma} -1}{1-\gamma}$ of your final wealth $W_1$.

#### Key Points
* To maintain a constant $\pi$, continuous portfolio rebalancing is needed.
* The wealth process $W_t$ evolves according to: $$ d W_t = (r + \pi(\mu-r))W_t dt + \pi \sigma W_t dB_t. $$
* Applying Ito’s Lemma to $\ln W_t$ yields: $$\ln W_t = \int^t_0 (r + \pi(\mu-r) - \frac{\pi^2 \sigma^2}{2})W_u du + \int^t_0 \pi \sigma dB_t,$$ suggesting $\ln W_1 \sim \mathcal{N}(r + \pi(\mu-r) - \frac{\pi^2 \sigma^2}{2}, \pi \sigma)$.
* The expected utility of final wealth is: $$ \mathbb{E}[U(W_1)] = r + \pi(\mu-r) - \frac{\pi^2 \sigma^2 \gamma}{2}. $$
* The optimal fraction $\pi^*$ of the investment in the risky asset, maximizing the expected utility, is: $$ \pi^* = \frac{\mu-r}{\gamma \sigma^2}. $$

## References
- Chapter 7 of the [RLForFinanceBook](https://stanford.edu/~ashlearn/RLForFinanceBook/book.pdf)
- [Utility Theory](https://github.com/coverdrive/technical-documents/blob/master/finance/cme241/Tour-UtilityTheory.pdf) slides for CME 241: Foundations of Reinforcement Learning with Applications in Finance