--- # System prepended metadata title: 'Generative Modeling for Control - Lecture 11: Flow Matching - Open loop to Closed Loop Synthesis' tags: [Generative Modeling for Control] --- # Feedback Control via Flow Matching In this lecture note, we explore a competing perspective on feedback control using generative modeling. This approach avoids the SDE-based framework of denoising diffusion models, and its sufficient to consider ODEs. While we will revisit the denoising principle later in the context of flow matching, we will no longer rely on SDEs. Flow matching is a method introduced in generative modeling that constructs interpolations between measures in a much simpler way. Using interpolations between noise and data samples, one constructs interpolations at the level of probability densities. Interestingly, it also provides a very versatile way to construct control laws. Here, we will look at the most elementary application of flow matching for control. --- ## The Open-Loop to Closed-Loop Problem ### Problem Statement Suppose we have a **family of open-loop controls** for each pair of initial $x_0$ and final conditions $x_T$, given by a map: $$S(x_0, x_T) = (x(t), u(t)) $$ such that the system dynamics are given by: $$ \dot{x} = g_0(x) + \sum_{i=1}^m u_i(t)g_i(x(t)) \equiv g(x, u) $$ with boundary conditions: $$ x(0) = x_0, \quad x(T) = x_T. $$ **Question:** Can we synthesize a **closed-loop control** $u_i(t, x)$ such that: $$ \dot{x} = g_0(x) + \sum_{i=1}^m u_i(t, x)g_i(x(t)) $$ and $x(T) = x_T$ for **any** $x_0 \in \mathbb{R}^d$? In general, this problem is hard since trajectories using different open loop controls might cross, hence preventing realization using a feedback control. While we might not be able to construct a feedback law that achieves the same trajectories as the open-loop controls, we can construct a feedback law that mimics the probabilistic behavior of the open-loop trajectories, as we will see in the following discussion. Let $\rho_0$ and $\rho_T$ be the distributions of the initial and final conditions, respectively. Define: $$ S_t(X_0(\omega), X_T(\omega)) = (X_t(\omega), U_t(\omega)), $$ where $\omega$ represents a sample from the probability space $\Omega$. Let $\rho_t$ be the probability density of $X_t$ under this planner $S(x_0,x_T)$. --- ### Pushforward Measure We briefly recall the pushforward measure notation that will be used subsequently. The **pushforward measure** of $\mathbb{P}$ under $X$, denoted $X_\sharp \mathbb{P}$ or $\mathbb{P}_X$, is defined as: $$ X_\sharp \mathbb{P}(A) = \mathbb{P}(X \in A) \quad \text{for all measurable sets } A \subset E. $$ If $F:\Omega \rightarrow \mathbb{R}^d$ is a function, then formally, $$ \int_{\Omega} F(X(\omega)) d\mathbb{P}(\omega) = \int_{E} F(x) dX_\sharp \mathbb{P}(x). $$ --- ### Deriving the Continuity Equation To derive the continuity equation for $\rho_t$, we use a test function $F \in C^\infty_c(\mathbb{R}^d)$ and evaluate it along $X_t$: $$ \frac{d}{dt}F(X_t(\omega)) = \dot{X}_t(\omega) \cdot \nabla F(X_t(\omega)). $$ Integrating with respect to $\omega$: $$ \frac{d}{dt} \int_{\mathbb{R}^d} F(x) \, d\rho_t(x) = \int_\Omega \dot{X}_t(\omega) \cdot \nabla F(X_t(\omega)) \, d\mathbb{P}(\omega). $$ Using the [**disintegration theorem**](https://en.wikipedia.org/wiki/Disintegration_theorem) for the joint law $(X_t, \dot{X}_t)_\sharp \mathbb{P}$, we have: $$ (X_t, \dot{X}_t)_\sharp \mathbb{P}(dx, dv) = \rho_t(x) \, dx \, d\eta_x(v), $$ where $\eta_x(v)$ is the conditional distribution of $\dot{X}_t$ given $X_t = x$. This allows us to rewrite the integral as: $$ \int_\Omega \dot{X}_t(\omega) \cdot \nabla F(X_t(\omega)) \, d\mathbb{P}(\omega) = \int \int v \cdot \nabla F(x) \, d\eta_x(v)d\rho_t(x) $$ $$ = \int_{\mathbb{R}^d} \nabla F(x) \cdot \mathbb{E}[\dot{X}_t | X = x] \, d\rho_t(x). $$ using $\mathbb{E}[\dot{X}_t | X = x] = \int v d\eta_x(v)$. Given the affine dependence of the vector field $g(x, u)$ on $u$, we have: $$ \mathbb{E}[\dot{X}_t | X = x] = g(x, \mathbb{E}[U_t | X = x]). $$ Setting $u(t, x) = \mathbb{E}[U_t | X = x]$, we obtain: $$ \frac{d}{dt} \int_{\mathbb{R}^d} F(x) \, d\rho_t(x) = \int_{\mathbb{R}^d} \nabla F(x) \cdot g(x, u(t, x)) \, d\rho_t(x). $$ Formally exchanging integrals and derivatives, and applying integration by parts, we recover the **continuity equation**: $$ \partial_t \rho_t(x) + \nabla \cdot \left( g(x, u(t, x)) \rho_t(x) \right) = 0. $$ Therefore, using the **continuity equation** for the density $\rho_t$ and disintegration we identified a feedback law that as the same probabilistic behavior.If $\rho_T = \delta_{x_T}$. Equivalently, $X_T(\omega) = x_T$, then the feedback control law drives the control system to $x_T$ ## Existence to Construction A question remains: **how do we learn the controller $u(t,x)$?** It turns out that the $L_2$ loss provides us with the answer. It is known that $f(x) = \mathbb{E}[Y | X=x]$ minimizes the $L_2$ loss: $$ \mathbb{E}[(Y - f(X))^2]. $$ We can use this principle to learn the feedback controller. Suppose $U_t$ is a random variable representing the open-loop control. Consider the following $L_2$ loss: $$ \mathcal{L}(\theta) = \int_0^T \mathbb{E} \left[ |U_t - u^{\theta}(t, X_t)|^2 \right] dt. $$ The minimizer of this loss is given by the **conditional expectation**: $$ u^{\theta^*}(t, X_t) = \mathbb{E}[U_t | X_t]. $$ This gives us the following algorithm. $$\boxed{\begin{aligned} &\textbf{Algorithm (Flow matching for Closed-loop Synthesis)} \\[6pt] &\textbf{Setup:} \text{ Control-affine system } \dot{x} = g_0(x) + \sum_{i=1}^m u_i\, g_i(x), \\ \\ &1.\ \text{Choose initial distribution } p_0 \text{ and and goal state}~ x_T . \\[4pt] &2.\ \text{Simulate ODE trajectories using Planner $(X_t,U_t) = S(X_0(\omega,x_T)$):} \\ &\qquad\qquad \dot{X}_t = g_0(X_t) + \sum_{i=1}^m U_i(t)g_i(X_t) \\ &\qquad\quad\text{Store samples } \{X^{(i)}_{t_k},U^{(i)}_{t_k}\}_{i=1}^N \text{ at each time step } t_k. \\[4pt] &3.\ \text{Train a network } u_\theta(x,t) \in \mathbb{R}^m \text{ via $L_2$ loss} \\ \\[4pt] &4.\ \text{Stabilise from any initial condition } x(0) \text{ using the ODE:} \\ &\qquad\qquad \dot{x}(t) = g_0(x) + \sum_{i=1}^m u_i(t)\, g_i(x), \\[2pt] &\qquad\qquad u(t) = \, u_\theta(x(t),\, t) \in \mathbb{R}^m. \\[4pt] &\qquad\quad\text{As } t \to T,\ \text{the state } x(t) \to x_T. \\[4pt] \end{aligned}}$$ ## Numerical Example We consider the following example, with 2 control inputs. This system is known to be controllable: $$ \begin{aligned} \dot{x}_1(t) &= u_1(t) \\ \dot{x}_2(t) &= u_2(t) \\ \dot{x}_3(t) &= u_1(t)x_2(t) \end{aligned} $$ An algorithm to steer systems of this form is the following: 1. Move $(x_1, x_2)$ to $(y_1, y_2)$ using constant controls by setting: $$ (u_1(t), u_2(t)) = \left(\frac{y_1 - x_1}{2\pi}, \frac{y_2 - x_2}{2\pi}\right), \quad t \in [0, 2\pi] $$ 2. Next, use sinusoidal signals to adjust the third coordinate. The controls are: $$ (u_1(t), u_2(t)) = \left(\sin(t), \frac{y_3 - x_3(2\pi)}{\pi} \cos(t)\right), \quad t \in (2\pi, 4\pi] $$ Then it can be shown that $\omega(0) = x$ and $\omega(4\pi) = y$. The following animation visualizes the training trajectories used for controlling to the origin. ![open_loop_trajectories_3d](https://hackmd.io/_uploads/r1ruq4m3bl.gif) The result of training a two layer MLP to learn the feedback controller is shown in the following animation. ![closed_loop_trajectories_3d](https://hackmd.io/_uploads/Byzt9NQh-l.gif) --- ## References ### Flow Matching In the uncontrolled settings, flow matching has been introduced in the following papers in the generative modeling literature. [Xingchao Liu, Chengyue Gong, Qiang Liu, *Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow*](https://arxiv.org/abs/2209.03003) [Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, Matt Le, *Flow Matching for Generative Modeling*. ](https://arxiv.org/abs/2210.02747) [ Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden, *Stochastic Interpolants: A Unifying Framework for Flows and Diffusions*.](https://arxiv.org/abs/2303.08797) ### Flow Matching for Control The material in this blog is based on the material of the following paper. [Karthik Elamvazhuthi, *Flow Matching for Measure Transport and Feedback Stabilization of Control-Affine Systems.*](https://arxiv.org/abs/2510.02706v1) See also work on flow matching for stochastic linear systems, [Yuhang Mei, Mohammad Al-Jarrah, Amirhossein Taghvaei, Yongxin Chen, *Flow matching for stochastic linear control systems.*](https://arxiv.org/abs/2412.00617) The open loop controls were generated using the method highlighted in this paper. [Richard M. Murray, and S. Shankar Sastry, *Nonholonomic Motion Planning: Steering Using Sinusoids.*](https://people.eecs.berkeley.edu/~sastry//pubs/OldSastryALL/MurrayNonholonomic1993.pdf)