# Derivative of the inverse of a matrix

So, here I'll show the generalized for of the matrix derivative using the Frechet derivative, and then using it (as a generalised derivative) to derive the formula for the partial derivative relative to a scalar.

As I mentioned in the previous document I showed you, the Frechet derivative although general is actually descriptive. Although there is a chain rule, for example, the Frechet derivative derivations aren't as direct.

We begin by defining $f(A) = A^{-1}$, where $A$ is a matrix. Well, we wish to express $f(A+\Delta A)$ in terms of $f(A)$ for some $\Delta A$ in a neighborhood of $0$.

We begin by proving the following identity:

\begin{equation}
\|h\|_{sp}<1\Rightarrow (\mathbf{I}+h)^{-1} = \sum_{n=0}^\infty (-1)^n h^n,
\end{equation}
where $\|\cdot\|_{sp}$ is the **spectral norm**, defined as the biggest singular value of $h$.

*proof*
First, I want to show that $H=\sum_{n=0}^\infty (-1)^n h^n$ is a matrix, that is, that the series converges.

Well, the first step is to remember the spectral norm is its own operation norm, and that $\|AB\|\leq\|A\|_{op}\|B\|$. Consequently: $\|h^n\|_{sp}\leq \|h\|_{sp}^n$. Therefore, we have that the series is dominated by $\sum_{n=0}^\infty \|h\|_{sp}^n=\frac{1}{1-\|h\|_{sp}}<\infty$. Since we have a series where the norms of the terms are dominated by those of an absolutely convergent series, this series converges absolutely, through Weierstrass' M-test.

Next, I want to show this series corresponds to the inverse. This is trivial, since:

\begin{equation}
(\mathbf{I}+h) \sum_{n=0}^\infty (-1)^n h^n = \sum_{n=0}^\infty (-1)^n h^n+h\sum_{n=0}^\infty (-1)^n h^n=\mathbf{I}+\sum_{n=1}^\infty (-1)^n h^n+\sum_{n=0}^\infty (-1)^n h^{n+1}=\mathbf{I}
\end{equation}

Therefore:

\begin{equation}
\sum_{n=0}^\infty (-1)^n h^n = (\mathbf{I}+h)^{-1},\ Q.E.D
\end{equation}

Well, having proven the identity, we now go to the Frechet derivative. Let us consider:

\begin{equation}
f(A+\Delta A) = (A+\Delta A)^{-1} = A^{-1}(\mathbf{I}+\Delta A A^{-1})
\end{equation}

If we suppose $\|\Delta A A^{-1}\|_{sp}<1$ (achievable by considering $\|\Delta A\|_{sp}<\frac{1}{\|A^{-1}\|_{sp}}$) we have that:

\begin{equation}
f(A+\Delta A) = A^{-1}\sum_{n=0}^\infty (\Delta A A^{-1})^n
\end{equation}

We already see our candidate for a Frechet derivative in:

\begin{equation}
Df[A](\Delta A) = -A^{-1} \Delta A A^{-1}
\end{equation}

And we can show this is indeed the Frechet derivative since:

\begin{equation}
\frac{\|f(A+\Delta A)-f(A)-Df[A](\Delta A)\|_{sp}}{\|\Delta A\|_{sp}} = \left \| \frac{A^{-1}(\Delta A A^{-1})^2}{\|\Delta A\|_{sp}} \sum_{n=0}^\infty (\Delta A A^{-1})^n\right \|_{sp}
\end{equation}

Which is easy to show goes to $0$ as $\Delta A \rightarrow0$.

Therefore, our candidate is the Frechet derivative.

In order to find the partial derivative of a $A$ in regards to the variable $a$, we can easily derive it using the fact the Frechet derivative is an extension for Banach spaces of the concept of the derivative in Euclidean spaces.

Specifically for multiple-variable calculus:

$\frac{\partial f}{\partial a} = Df[A(a)]\left (\frac{\partial A}{\partial a} \right )=-A^{-1} \frac{\partial A}{\partial a} A^{-1}$