In this lecture, we consider the Fokker-Planck Eqautions associated with control systems that have drift
$$\dot{x} = g_0(x) + \sum_{i=1}^m u_i(t) g_i(x),$$ when we add noise through the control channels. In this setting, while there isn't a general theory of how to shape target densities using purely feedback control, one can still say something useful about the long term behavior of the system, that can be useful in generative modeling approaches such as denoising diffusion, where it is important to understand if noise can excite the system onto the reachable space of a system. For this reason, we present the limited theory that's possible in this case.

## Linear Time Invariant Systems

First, lets consider the case of LTIs. In this case, $g_0(x) = Ax$ and $g_i(x) = b_i$, where $A \in \mathbb{R}^d \times \mathbb{R}^d$, and $b_i \in \mathbb{R}^d$ are vectors.

In this case, when noise is added through the control channels, the system becomes,

$$dX = AX + \sqrt{2}BdW.$$

While we can look at the corresponding Fokker-Planck equation in this case:

$$\partial_t p_t = -\nabla \cdot (Ax ~ p_t)+ \sum_{i} (b^j_i \partial_j )^2 p_t $$

Alternatively, this equation can be expressed as,

$$\partial_t p_t = -\nabla \cdot (Ax ~ p_t)+ \sum_{i} \nabla \cdot (b_ib^t_i \nabla p_t).$$

The solution of this equation can be given in closed form using the associated kernel function:

Using the *Controllability Gramian*

$$\Sigma_t = 2 \int_0^t e^{sA} \, B B^\top \, e^{sA^\top} \, ds = 2 \int_0^t e^{sA} \left(\sum_i b_i b_i^\top \right)e^{sA^\top} \, ds .$$
we define the kernel
$$K(t,x,y)=
\frac{1}{(2\pi)^{n/2} \, (\det \Sigma_t)^{1/2}}
\exp\!\left(
-\frac{1}{2}
\bigl(x - e^{tA} y\bigr)^\top
\Sigma_t^{-1}
\bigl(x - e^{tA} y\bigr)
\right).$$

Then the solution of the PDE $p_t$ can be given by

$$p_t(x) = \int_{\mathbb{R}^d} K(t,x,y) p_0(y)dy.$$

The Kernel is the solution of the PDE for Dirac Delta initial conditions. That is $p_t(x) = K(t,x,y)$ for Dirac valued initial conditions:

$$p_0(x) = \delta(x - y)$$

Therefore, one should interpret the expression $$p_t(x) = \int_{\mathbb{R}^d} K(t,x,y) p_0(y)dy$$ as a form of superposition prinicple, where the effects of different initial conditions -- densities $p_0(y)$ is being superimposed to recover the full solution $p_t(y)$.

### Steady State Behavior
If the Controllability Gramian has an infinite time limit (this requires $A$ to be Hurwitz):

$$\Sigma_{\infty} = 2 \int_0^{\infty} e^{sA} \, B B^\top \, e^{sA^\top} \, ds $$

then one can see that informally,

$$\lim_{t \rightarrow \infty }K(t,x,y) = p_{\infty}(x) =
\frac{1}{(2\pi)^{n/2} \, (\det \Sigma_{\infty})^{1/2}}
\exp\!\left(
-\frac{1}{2}
x ^\top
\Sigma_{\infty}^{-1}
x
\right).$$

This also means that,

$$ \lim_{t \rightarrow \infty} p_t(x) = \int_{\mathbb{R}^d} p_{\infty}(x) p_0(y)dy = p_{\infty}(x) $$

Since the $\int_{\mathbb{R}d} p_0(y)dy = 1$.

Thus, in the LTI case, in contrast to driftless non-linear systems, one has a more analytical grasp of the solutions. On the other hand, controlling the system to arbitrary target probability densities $p_{\infty}$ is a challenge that has not been completely addressed in literature. An option to stabilize to more general Gaussian distributions is to introduced a feedback control law so that the matrix $A$ is replaced by $A+BK$ for some state feedback matrix $K$.

## Positivity of Solutions

An important property that is apparent is that when the controllability Gramaian is invertible, the kernel $K(t,x,y)$ is positive everywhere. This implies that noise causes the system to propogate everywhere in the state space due to controllability through the actuation channels.

## General Nonlinear systems with Drift

In general, for nonlinear systems with drift, its hard to get a analytical characterization of the long term behavior. At best one can say that a stationary distribution $p_{\infty}(x)$ exists. We do not get into the theory of this, and instead settle for something weaker: Is the solution of the corresponding Fokker-Planck equation positive? Alternatively, when does noise propogate the support of the solution everywhere? Clearly, this property has to once again depend on the controllability properties of the system.

We have the following informal claim

**Claim (Informal Positivity Theorem)**: Consider the Fokker-Planck equation

$$\partial_t p_t= \mathcal{Y}^*_0 p_t + \sum_{i=1}^m (\mathcal{Y}_i^2)^* p_t
$$

corresponding to the (Stratonovich) SDE:

$$
\mathrm{d}X
= g_0(X)dt +\sqrt{2}\sum_{i=1}^m g_i(X) \odot dW_i,
$$

If the corresponding control system

$$
\dot{x}
= g_0(x) +\sum_{i=1}^m u_i(t) g_i(x), \tag{ctrsys}
$$

Is globally controllable between any two points:
For any $y,z \in \mathbb{R}^d$ and $t >0$, there exists a control $u(t)$ for eq (ctrsys) such $x(0) = y$ and $x(t) =z$, then the solution
$$p_t(x) >0$$ for all $t>0$ and $x \in \mathbb{R}^d$, assuming $p_0(x)dx = \mathbb{P}(X(0) = dx)$.

### But how does one check globally controllability of a driftless system?

Unfortunately, this happens to be one of the more challenging problems in control theory. Unlike the Kalman rank condition for linear systems, and Lie bracket based tests for driftless nonlinear there is no neccessary and sufficient conditions for global controllability for systems with drift. We are going to stay happy with stating global controllability as an assumption, and march on into generative modeling land.