{%hackmd 5xqeIJ7VRCGBfLtfMi0_IQ %}

# Rayleigh quotient of a diagonal matrix

## Problem

Let

$$
A = \begin{bmatrix}
3 & 0 & 0 \\
0 & 4 & 0 \\
0 & 0 & 5
\end{bmatrix} \text{ and }
\bx = \begin{bmatrix} x \\ y \\ z \end{bmatrix}.
$$

Solve the optimization problem

$$
\begin{array}{ll}
\max & \bx\trans A \bx, \\
\text{subject to} & \|\bx\| = 1.
\end{array}
$$

## Thought

The purpose of this problem is to get a feeling of the Rayleigh quotient theorem, so you are encouraged to solve it by fundamental methods rather than the theorem.

## Sample answer

By direct calculation, the optimization is the same as

$$
\begin{array}{ll}
\max & 3x^2 + 4y^2 + 5z^2, \\
\text{subject to} & x^2 + y^2 + z^2 = 1.
\end{array}
$$

By replacing $x^2 = p_1$, $y^2 = p_2$, and $z^2 = p_3$, we noticed that $p_1, p_2, p_3 \geq 3$ and $p_1 + p_2 + p_3 = 1$. That is, $(p_1, p_2, p_3)$ is a [probability distribution](https://en.wikipedia.org/wiki/Probability_distribution). Now the problem is equivalent to

$$
\begin{array}{ll}
\max & p_3\cdot 3 + p_2\cdot 4 + p_3\cdot 5, \\
\text{subject to} & (p_1,p_2,p_3) \text{ is a probability distribution}.
\end{array}
$$

Under this framework, we are maxizing the weighted average of $3,4,5$ with respect to the weights $p_1, p_2, p_3$. Thus, the weighted average is between the minimum value $3$ and the maximum value $5$. Thus, the maximum value is $5$, achieved by $(x,y,z) = (0,0,1)$ or $(x,y,z) = (0,0,-1)$.

*This note can be found at Course website > Learning resources.*