$$

Rayleigh quotient of a diagonal matrix

Problem

Let

$$A=\left[\begin{array}{ccc}3& 0& 0\\ 0& 4& 0\\ 0& 0& 5\end{array}\right]\text{ and }\mathbf{x}=\left[\begin{array}{c}x\\ y\\ z\end{array}\right].$$

Solve the optimization problem

$$\begin{array}{ll}max& {\mathbf{x}}^{\mathrm{\top }}A\mathbf{x},\\ \text{subject to}& \Vert \mathbf{x}\Vert =1.\end{array}$$

Thought

The purpose of this problem is to get a feeling of the Rayleigh quotient theorem, so you are encouraged to solve it by fundamental methods rather than the theorem.

Sample answer

By direct calculation, the optimization is the same as

$$\begin{array}{ll}max& 3x^2+4y^2+5z^2,\\ \text{subject to}& x^2+y^2+z^2=1.\end{array}$$

By replacing

$x^2=p_1$,

$y^2=p_2$, and

$z^2=p_3$, we noticed that

$p_1,p_2,p_3\ge 3$ and

$p_1+p_2+p_3=1$. That is,

$(p_1,p_2,p_3)$ is a probability distribution. Now the problem is equivalent to

$$\begin{array}{ll}max& p_3\cdot 3+p_2\cdot 4+p_3\cdot 5,\\ \text{subject to}& (p_1,p_2,p_3)\text{ is a probability distribution}.\end{array}$$

Under this framework, we are maxizing the weighted average of

$3,4,5$ with respect to the weights

$p_1,p_2,p_3$. Thus, the weighted average is between the minimum value

$3$ and the maximum value

$5$. Thus, the maximum value is

$5$, achieved by

$(x,y,z)=(0,0,1)$ or

$(x,y,z)=(0,0,-1)$.

This note can be found at Course website > Learning resources.