---
# System prepended metadata

title: 'Collective Decision Making as a Feedback Control Problem III: A Gradient Flow Perspective'
tags: [Collective Decision Making]

---

# The Bees Buzz Down a Gradient Flow

![image](https://hackmd.io/_uploads/BksEcVNpbg.png)


In this note, we take a **gradient flow** approach to solve the collective decision making problem. We once again go back to the two state system we saw in the first post on this. 

We had the equation governing the evolution of the population of bees,

$$
\begin{aligned}
\dot{x}_A(t) &= -u_{AB}x_A(t) + u_{BA}x_B(t),\\
\dot{x}_B(t) &= u_{AB}x_A(t) - u_{BA}x_B(t). 
\end{aligned}
\tag{mfe}
$$

We want the bees to decide between option $A$ and $B$ by choosing an appropriate mean-field feedback law $u_{AB}$ and $u_{BA}$. We presented some constraints in solving this problem if we pose the hard constraint that bees should pick the best option if $A \neq B$, and come to consensus on one of the options if $A = B$. Then we saw how constructively this problem can be solved if the constraints are relaxed in some way.

We take a natural *variational* point of view in this post. What if the bees somehow used an optimization criteria to choose their decision. Towards this end, lets review classical gradient systems:
$$\dot{x} = -\nabla {f}(x)$$where $f(x)$ is some potential function. Under some conditions it is known that this system asymptotically converges to critical points of $f(x)$, and if we are blessed enough, to the global minimzer.

An intuitive extension would be if we had a potential $f(x,\theta)$ that depended on the choices $\theta = (\theta_1, \theta_2)$, and then considered the parametrized gradient system,
$$\dot{x} = -\nabla_x {f}(x,\theta)$$

A problem with this approach is that the actual dynamics of our system are given by the mean field equation (mfe). We want to construct a gradient system that is consistent with these dynamics. That is, there should exist some $u_{AB}$ and $u_{BA}$ such that the closed-loop system looks like our gradient system. Moreover, we require $u_{AB}, u_{BA} \geq 0$, which adds an additional constraint.

## A Projected Gradient Flow Approach

To deal with the issue that we need a system consistent gradient flow lets generalize to specialize. Suppose we have a control system of the form,
$$
\dot{x} = \sum_{i}u_i(t)\, g_i(x) \tag{ctr-sys}
$$ One natural notion of a gradient associated with this system is to consider the feedback law $u_i(t) = \nabla f \cdot g_i$, the [**Lie derivative**](https://en.wikipedia.org/wiki/Lie_derivative) of the potential $f$ along $g_i$. This results in the [non-holonomic gradient system](https://hackmd.io/@clopenloop/rJN69Wv4Wg),
$$
\dot{x} = \sum_{i}\Big[\nabla f \cdot g_i\Big](x)\, g_i(x)
$$ Further, if one wants to constrain the controls to some set $K$, we apply a projection and get
$$u_i(t) = \Pi_{K}(-\mathcal{L}_{g_i} f),$$ where $\Pi_K$ denotes the projection. So the closed-loop system becomes,
$$\dot{x} = \sum_{i}\Big[\Pi_{K}(-\mathcal{L}_{g_i} f)\Big](x)\, g_i(x)
$$ One can think of this as a form of projected gradient system, where at each instant we are projecting the gradient onto the set of admissible velocities or controls.

## Application to Collective Decision Making

### Cost Function

To apply the idea to collective decision making, we need a cost function. Let
$$\ell(x,\theta) = -\theta \cdot x + \frac{\lambda}{2}(1 - \|x\|^2)$$ The first term encourages selection of the best option, and the second term enforces consensus: $1 - \|x\|^2$ achieves its minimum on the simplex exactly when $x_i = 1$ for some $i$.

The gradient of $\ell$ is:
$$\nabla_x \ell = -\theta - \lambda x$$

The mean field dynamics can be written in the form (ctr-sys) as
$$\dot{x} = u_{AB}\, g_1 + u_{BA}\, g_2$$
with
$$g_1 = \begin{bmatrix}-x_A\\x_A\end{bmatrix}, \qquad g_2 = \begin{bmatrix}x_B\\-x_B\end{bmatrix}.$$



The Lie derivative of $\ell$ along $g_1$ is:
$$\mathcal{L}_{g_1}\ell = \nabla_x\ell \cdot g_1 = \begin{pmatrix}-\theta_1 - \lambda x_A \\ -\theta_2 - \lambda x_B\end{pmatrix}\cdot\begin{pmatrix}-x_A\\x_A\end{pmatrix}$$ $$= x_A\left[(\theta_1 - \theta_2) + \lambda(x_A - x_B)\right]$$

Setting $\delta = \theta_1 - \theta_2$ and $\phi(x) = \delta + \lambda(x_A - x_B)$, we have $\mathcal{L}_{g_1}\ell = x_A\phi$ and $\mathcal{L}_{g_2}\ell = -x_B\phi$.



Requiring $K = [0,\infty)$, the projected controls are $u_i = \Pi_K(-\mathcal{L}_{g_i}\ell)$, giving:
$$u_{AB} = \max\{-x_A\phi(x),\, 0\} = x_A\max\{-\phi(x),\,0\}$$ $$u_{BA} = \max\{x_B\phi(x),\, 0\} = x_B\max\{\phi(x),\,0\}$$

where we used $x_A, x_B \geq 0$. 

## Equilibria Characterization

First, lets look at the possible equilibria in the interior of the probability simplex $\Delta = \{x_A + x_B =1 \}$.
In the interior $x_A, x_B > 0$, the scalar dynamics reduce to:
$$\dot{x}_A = -u_{AB}\,x_A + u_{BA}\,x_B = -x_A^2\max\{-\phi,0\} + x_B^2\max\{\phi,0\}$$

At most one of $\max\{-\phi,0\}$, $\max\{\phi,0\}$ is active at any point (not very *bio-like!!*), so this simplifies to:
$$\dot{x}_A = \begin{cases} x_B^2\,\phi & \phi > 0 \\ x_A^2\,\phi & \phi < 0 \end{cases}$$

Clearly, the condition $\dot{x}_A = 0$ in the interior requires $\phi = 0$, giving the unique interior equilibrium:
$$x_A^* = \frac{1}{2} - \frac{\delta}{2\lambda}$$
which lies in $(0,1)$ if and only if $|\delta| < \lambda$.

Next, we look at boundary equilbrium points. At $e_A = (1,0)$: both $u_{AB}g_1$ and $u_{BA}g_2$ are $0$.By symmetry, $e_B = (0,1)$ is also an equilibrium point.

## Stability

Stability of the equilbria can be assesed directly from the sign of $\dot{x}_A$. The sign of $\dot{x}_A$ is determined entirely by the sign of $\phi = 2\lambda(x_A - x_{\rm int}^*)$, 
since the factors $x_B^2$ and $x_A^2$ are non-negative:

$$\dot{x}_A = \begin{cases} x_B^2\,\phi & \phi > 0 \\ x_A^2\,\phi & \phi < 0 \end{cases}$$

**Regime 1: $|\delta| < \lambda$.** The interior equilibrium $x_{\rm int}^* \in (0,1)$ partitions the simplex:
- $x_A > x_{\rm int}^*$: $\phi > 0$, so $\dot{x}_A > 0$ — flow toward $e_A$
- $x_A < x_{\rm int}^*$: $\phi < 0$, so $\dot{x}_A < 0$ — flow toward $e_B$

Both vertices are stable and $x_{\rm int}^*$ is the unstable interior equilibrium between their basins of attraction.

**Regime 2: $\delta \geq \lambda$.** The interior equilibrium $x_{\rm int}^* \leq 0$, so $\phi > 0$ for all 
$x_A \in (0,1)$. Thus $\dot{x}_A > 0$ everywhere — the flow drives the system to $e_A$ globally.

**Regime 3: $\delta \leq -\lambda$.** The interior equilibrium $x_{\rm int}^* \geq 1$, so $\phi < 0$ for all 
$x_A \in (0,1)$. Thus $\dot{x}_A < 0$ everywhere — the flow drives the system to $e_B$ globally.


### Three Regimes

The equilibrium structure depends on the ratio $|\delta|/\lambda$:

| Condition | Interior $x^*$ | Stable equilibria | Behavior |
|---|---|---|---|
| $\|\delta\| < \lambda$ | Inside $(0,1)$, unstable | $e_A$ and $e_B$ | Bistable|
| $\delta \geq \lambda$ | Outside $(0,1)$ | $e_A$ only | Unique decision: $A$ |
| $\delta \leq -\lambda$ | Outside $(0,1)$ | $e_B$ only | Unique decision: $B$ |


The following plots show the projected gradient flow dynamics for the three regimes. Each bee marker represents a swarm starting from a different initial fraction $x_A$. The lower panel shows $\dot x_A$ as a function of $x_A$: where the curve is positive the swarm drifts toward $A$, where negative toward $B$.


![grad_A_wins](https://hackmd.io/_uploads/HJN9GL46Zl.gif)
![grad_B_wins](https://hackmd.io/_uploads/HJ4czINT-x.gif)
![grad_bistable](https://hackmd.io/_uploads/rkV5f8Na-x.gif)

## Achieving Indecision for Low Quality Options


In some situations, one additionally requires that the population remains indecisive if the options are of very poor quality. The current cost function does not capture this. The consensus parameter $\lambda$ is fixed, so the system is always decisive (bistable) or always driven toward one vertex, regardless of whether the options are actually worth committing to.

To address this, we can introduce the modified cost:

$$\ell(x,\theta) = -\theta \cdot x + \frac{\mu(\bar{\theta} - \bar{\theta}^*)}{2}(1 - \|x\|^2)$$

where $\bar{\theta} = (\theta_1 + \theta_2)/2$ is the mean quality of the two options, $\bar{\theta}^*$ is a design parameter representing the minimum quality threshold for commitment, and $\mu > 0$ is a scaling factor. The original cost is recovered when $\bar{\theta} - \bar{\theta}^* = \lambda/\mu$.



The key observation is that $(1 - \|x\|^2)$ is maximized at the center of the simplex ($x_A = x_B = 1/2$) and minimized at the vertices $e_A, e_B$. The sign of $\mu(\bar{\theta} - \bar{\theta}^*)$ therefore controls whether the cost landscape is convex or non-convex over the simplex, and affects the stability of the interior equilibrium.


---

Another interesting variational model in the literature is obtained by looking at the decision making problem as a game. More specially, a *mean-field game*. Interestingly, for some choices of cost, one recovers the control laws of the Seeley model discussed in the previous post. See [Leonardo Stella, Dario Bauso, and Patrizio Colaneri. "Mean-field game for collective decision-making in honeybees via switched systems." ](https://ieeexplore.ieee.org/abstract/document/9529000)