## 3.4 Higher Order Partial Derivatives

Recall the following.

- The partial derivatives give us the rate of change of the function in directions parallel to axes in the Euclindean space.
- To compute the partial derivative of $f(x_1,\dots,x_n)$ with respect to $x_j$, we treat other independent variables ($x_k$ with $k\neq j$) as constants, and take the ordinary derivative of $f$ with respect to $x_j$.
- Polynomials, rational functions, trigonometric functions, exponentials, logarithms, and their composites have partial derivatives inside their domains.

Now, we discuss *higher-order* partial derivatives.

:::info

**Second Order Partial Derivatives**: Let $f$ be function of $x$, $y$, and possibly other independent variables. Then
$$f_{xx} :={(f_x)}_x\,,\,\,\, i.e.,\,\,\, \frac{\partial^2 f}{\partial x^2} :=\frac{\partial}{\partial x}\left(\frac{\partial f}{\partial x}\right)$$$$f_{xy} :={(f_x)}_y\,,\,\,\, i.e.,\,\,\, \frac{\partial^2 f}{\partial y \partial x} :=\frac{\partial}{\partial y}\left(\frac{\partial f}{\partial x}\right)\,\,\, \text{(mixed partial)}$$$$f_{yx} :={(f_y)}_x\,,\,\,\, i.e.,\,\,\, \frac{\partial^2 f}{\partial x \partial y} :=\frac{\partial}{\partial x}\left(\frac{\partial f}{\partial y}\right)\,\,\, \text{(mixed partial)}$$
:::
<br>

**Example 1**

Let $f:\mathbb R^2 \to \mathbb R$ be $f(x,y)=x^3+x^2y^3-2y^2$. Find all of its second order partial derivatives.

:::spoiler Answer
Note that $f_x(x,y) = 3x^2+2xy^3$ and $f_y(x,y)=3x^2y^2-4y$.

Therefore, the second order partial derivatives are
$$f_{xx}=6x+2y^3$$$$f_{xy}=6xy^2$$$$f_{yx}=6xy^2$$$$f_{yy}=6x^2y-4\,.$$
:::
<br>

**Example 2**

Let $f:\mathbb R^2 \to \mathbb R$ be $f(x,y)=xye^{y}+\cos(xy)$. Find the second order *mixed partial derivatives* $f_{xy}$ and $f_{xy}$.

:::spoiler Answer
Note that $f_x(x,y) = ye^y - y\sin(xy)$ and $f_y(x,y)=x(e^y+ye^y) - x \sin(xy)$.

Therefore, the second order mixed partial derivatives are
$$f_{xy}=e^y+ye^y-\sin(xy)-yx\cos(xy),$$$$f_{yx}=e^y+ye^y-\sin(xy)-xy\cos(xy)\,.$$
:::
<br>

Did you notice that in both cases $f_{xy} = f_{yx}$? This is not by coincidence. If the function is *nice enough*, then this will always be the case.

:::success

**Theorem** (Clairaut's Theorem)
Suppose $f$ is defined on a disc $D \subset \mathbb R^2$ that contains the point $(a,b)$. If both $f_{xy}$ and $f_{yx}$ are continuous on $D$, then $${f_{xy}}\left( {a,b} \right) = {f_{yx}}\left( {a,b} \right)\,.$$
:::

In fact, this theorem holds for functions of more than two variables: If $f:\mathbb R^n \to
\mathbb R$ be such that $f_{x_jx_k}$ and $f_{x_kx_j}$ are *continuous* on $\mathbb R^n$, then $$f_{x_kx_j}=f_{x_jx_k}\,.$$

**Example 3**

Let $f:\mathbb R^2 \to \mathbb R$ be given by $$f(x,y)=\begin{cases}\frac{xy(x^2-y^2)}{x^2+y^2} & (x,y)\neq (0,0)\\ 0 & (x,y)= (0,0)& \end{cases}\,.$$ It is known that $f$ is continuous and second order partial derivatives exist everywhere including the origin. Find the second order *mixed partial derivatives* at $(0,0)$, i.e., $f_{xy}(0,0)$ and $f_{yx}(0,0)$. Are they equal?

:::spoiler Answer
When $(x,y)\neq (0,0)$, $f_x(x,y)=y \frac{x^4+4x^2y^2-y^4}{(x^2+y^2)^2}$ and $f_y(x,y)=x \frac{x^4-4x^2y^2-y^4}{(x^2+y^2)^2}$.

$f_x$ | $f_y$
:-------------------------:|:-------------------------:
 | 

Note that $f_x(0,y)=-y \implies f_{xy}(0,y)=-1 \implies f_{xy}(0,0)=-1$ and $f_y(x,0)=x \implies f_{yx}(x,0)=1 \implies f_{yx}(0,0)=1$. So, $f_{xy}(0,0) \neq f_{yx}(0,0)$.

But why? Because $f_{xy}(x,y)$ is not continuous at $(0,0)$. (Prove this!)
:::
<br>

We can also define third, fourth, fifth, etc. order partial derivatives such as $$f_{xxx}, f_{xxy}, f_{xyy}, f_{xyx}, \dots, f_{xxxx}, f_{xxxy}, \dots$$ as long as they exist. Computing them amounts to repeated differentiation in the order given by the symbol. We will discuss an example to illustrate this.

**Example 4**

Find $f_{xxxx}$ and $f_{xyx}$ where $f(x,y,z)=x^2yz+ y \sin(x)\,.$

:::spoiler Answer
Taking the derivative with respect to $x$, $f_{x}(x,y,z)=2xyz+y\cos x$. Repeating, $f_{xx}(x,y,z)= 2yz - y \sin x\,.$ Repeating once more, $f_{xxx}(x,y,z)=- y \cos x\,.$ Repeating for the final time $f_{xxxx}(x,y,z)=y \sin x\,.$

Taking the derivative of $f_x$ with respect to $y$, we have $f_{xy}(x,y,z)=2xz+ \cos x$. Taking the derivative with respect to $x$, we obtain $f_{xyx}(x,y,z)=2z - \sin x\,.$
:::
<br>

---
**References**
1. *Chapter 14.3* : Stewart, J. (2012). Calculus (8th ed.). Boston: Cengage Learning.
2. *Chapter 13.2* : Strang G., Calculus (3rd ed.). [https://ocw.mit.edu/](https://ocw.mit.edu/courses/res-18-001-calculus-fall-2023/pages/textbook/)
3. *Chapter 2.2* : Corral, M. (2021). Vector Calculus. [https://www.mecmath.net/](https://www.mecmath.net/)

---
<br>

## 3.5 Linear approximations

Before, we discuss third and higher order partial derivatives, let's see some applications of first and second order partial derivatives.

:::success
When we zoom in on a point $(a,f(a))$ on a smooth curve given by $y=f(x)$ the curve looks more an more like the tangent line $$y-f(a) = f'(a)(x-a).$$
:::


The same phenomena occurs in higher dimensions. For example, we feel that the earth is flat locally but if we zoom out we see that the earth is, in fact, a sphere.


In general, when we zoom-in on a point $(a, b, f(a,b))$ on a *smooth* surface $z=f(x,y)$, the surface looks more an more like the tangent plane to the surface at that point.
<br>

:::success
**Theorem** (Tangent Plane)
Suppose $f$ has continuous partial derivatives. Then the tangent plane to the surface $z=f(x,y)$ at $(a,b, f(a,b))$ is $$z-f(a,b)=f_x(a,b)(x-a)+f_y(a,b)(y-b).$$
:::

:::spoiler Proof
Recall that the a plane passing through $(a, b, f(a,b))$ has the form $$A(x-a)+B(y-b)+C(z-f(a,b))=0.$$Assuming $C \neq 0$, and writing $A'=-A/C$ and $B'=-B/C$, we have
$$z-f(a,b)=A'(x-a)+B'(y-b)\,.$$

Then, the intersection of the plane with $y=b$ must be the $x-$directional tangent line at $(a,b, f(a,b))$, having the slope of $f_x(a,b)$, i.e., $z-f(a,b) = A' (x-a)$. This means $A'=f_x(a,b)$.

Similarly, the intersection of the plane with $x=a$ must be the $y-$directional tangent line at $(a,b, f(a,b))$, having the slope of $f_y(a,b)$, i.e., $z-f(a,b) = B' (y-b)$. This means $B'=f_y(a,b)$.
:::
<br>

**Example 5**

Find the tangent plane to the ellipsoid $2x^2+y^2+z^2=4$ at the point $(1,1,1)$

:::spoiler Answer
Note that the surface is not given in the form $z=f(x,y)$. However, $(1,1,1)$ lies in the upper half $z>0$. Therefore, $z=\sqrt{4-2x^2-y^2}=:f(x,y)$ with partial derivatives $f_x(x,y)=\frac{-2x}{\sqrt{4-2x^2-y^2}}$ and $f_y(x,y)=\frac{-y}{\sqrt{4-2x^2-y^2}}$

Therefore, the tangent plane is $$z-1=f_x(1,1)(x-1)+f_y(1,1)(y-1)=-2(x-1)-1(y-1)$$ or equivalently, $2x+y+z=4$.


:::
<br>

We can use the tangent plane as a first approximation of a function. However, it will be a good approximation only if the function is *nice*.

:::info

**Definition**

Let $D$ be a disc in $\mathbb R^2$. Suppose $f:D\to \mathbb R$ has first order partial derivatives, then the **linear approximation** of $f(x,y)$ at the point $(a,b)\in D$ is $$L_{f,(a,b)}(x,y)=f(a,b)+f_x(a,b)(x-a)+f_y(a,b)(y-b)\,.$$
:::
<br>

**Example 6**

Find the linear approximation of $f(x,y)=xe^{xy}$ at $(1,0)$. Using it find an approximate value for $f(1.001,-0.001)$. Use a computer algebra system to find the error of this approximation.

:::spoiler Answer
Note that $f_x(x,y)=e^{xy}+xye^{xy}$ and $f_y(x,y)=x^2e^{xy}$. So, $$L_f(1,0)=f(1,0)+f_x(1,0)(x-1)+f_y(1,0)(y-0)=1+(x-1)+y=x+y$$ As a result, $f(1.001,-0,001)=1$.

Plugging in to WolframAlpha gives: $f(1.001,-0,001)=0.999999$. So the above linear approximation has a maximal error of $10^{-6}$!!!
:::
<br>

**Example 7**

Find the linear approximation of $$f(x,y)=\begin{cases}\frac{xy}{x^2+y^2} & (x,y) \neq (0,0)\\ 0 & (x,y) = (0,0)\end{cases}$$ at $(0,0)$. Using it find an approximate value for $f(0.001,0.001)$. What is the error of approximation?

:::spoiler Answer
Note that $f_x(0,0)=f_y(0,0)=0$. So, $$L_f(0,0)=f(0,0)+f_x(0,0)(x-0)+f_y(0,0)(y-0)=0$$ As a result, $ f(0.001,0.001) ~ 0 $.

But $f(0.001,0.001)=0.5$ and the error is $0.5$ and it is very large!!!


:::
<br>

There is a fundamental difference between the behaviour of the linear approximation in the two examples above. In the first case, the linear approximation improves closer to the point but in the second case, it does not. In fact, $f(x,x)=0.5$ for all $x$ but the linear approximation at $(0,0)$ give us $L_{f,(0,0)}(x,x)=0$.

This difference is due to the *differentiability* of the function. One can think of it as the existence of a unique tangent plane.

Recall that, in the single variable case, a function is differentiable if there is a unique tangent line, and the tangent line approximation gets better as we zoom in on a point. This idea from one dimension can be generalised to the following.
<br>

:::info
**Definition**

Let $D$ be a disc in $\mathbb R^2$. Suppose $f:D\to \mathbb R$ has first order partial derivatives, then we say that $f(x,y)$ is **differentiable** at the point $(a,b)\in D$ if there exists $\varepsilon_j=\varepsilon_j(x,y)$ for $j=1,2$ such that $\lim_{(x,y)\to (a,b)}\varepsilon_j(x,y) = 0$ for $j=1,2$ and$$f(x,y)=L_{f,(a,b)}(x,y)+\varepsilon_1 (x-a)+\varepsilon_2 (y-b).$$

:::
<br>

There are many examples of functions that are differentiable. The following theorem gives us a sufficient (but not necessary) condition for differentiability of a function.

:::success

**Theorem**

Suppose $f$ is defined on a disc $D \subset \mathbb R^2$ that contains the point $(a,b)$. Suppose $f:D\to \mathbb R$ has first order partial derivatives that are continuous on $D$. Then $f$ is differentiable at $(a,b)$.
:::
<br>

**Example 7**

Discuss the differentiability of $f(x,y)=\ln (x^2+y^2+1)$ on $\mathbb R^2$.

:::spoiler Answer
Note that $f_x(x,y)=\frac{2x}{x^2+y^2+1}$ and $f_y(x,y)=\frac{2y}{x^2+y^2+1}$. Both are continuous on $\mathbb R^2$. According to the previous theorem, $f$ is differentiable everywhere.

:::
<br>

However, the converse of the theroem is not true. There are functions that are differentiable that have discontinuous partial derivatives. One example is the following
$$f(x,y)=\begin{cases}(x^2+y^2)\sin\left(\frac{1}{\sqrt{x^2+y^2}}\right) & \text{ if $(x,y) \ne (0,0)$}\\0 & \text{ if $(x,y) = (0,0)$}.\end{cases}$$$f$ is differentiable at $(0,0)$, i.e., $z=f(x,y)$ has a unique tangent plane at $(0,0)$, but $f_x$ and $f_y$ are not continuous at $(0,0)$. (Visit [this link](https://mathinsight.org/differentiable_function_discontinuous_partial_derivatives) if you are curious.)

Also, the theorem tells us that if a function is not differentiable, then at least one of the partial derivatives should be discontinuous. (Visit [this link](https://mathinsight.org/nondifferentiable_discontinuous_partial_derivatives) for a nice visual explanation of an example.)
<br>

---
**References**
1. *Chapter 14.4* : Stewart, J. (2012). Calculus (8th ed.). Boston: Cengage Learning.
2. *Chapter 13.3* : Strang G., Calculus (3rd ed.). [https://ocw.mit.edu/](https://ocw.mit.edu/courses/res-18-001-calculus-fall-2023/pages/textbook/)
3. *Chapter 2.3* : Corral, M. (2021). Vector Calculus. [https://www.mecmath.net/](https://www.mecmath.net/)

---
<br>

:::danger

**Summary**: Now, we can

- Compute higher order partial derivatives of functions of several variables.
- Provide sufficient conditions for the symmetry of mixed partial derivatives.
- Find the tangent plane to a surface $z=f(x,y)$.
- Use the linerisation to calculate approximate the function values.
- Provide sufficient conditions for the linearisation to be a *good* approximation.

:::