# Multiset checks with multiplicities and LogUp

This note picks up where the previous one on [multiset checks](/b5JjgH5HQye1Rwoxq-zzig) left off. The main motivation for this note is to support a variation on multiset checks where we can encode how many times we expect an element to show up in the set (*i.e.* the *multiplicity*) without having that element showing up that many times. Specifically, the multiset check with multiplicities equation is

$$
\prod_{i=1}^{n-1} \prod_{j=1}^k (\alpha - a_{ij})^{m_{a_{ij}}} = \prod_{i=1}^{n-1} \prod_{j=1}^k (\alpha - b_{ij})^{m_{b_{ij}}}
$$

For example, if $m_{a_{23}} = 2$, then the term $(\alpha - a_{23})$ in the equation is repeated twice: $(\alpha - a_{23})\cdot(\alpha - a_{23})$. Hence, the value $a_{23}$ would appear once in the $\{a_{ij}\}$ multiset, but would need to appear twice in the $\{b_{ij}\}$ multiset (or equivalently, only once, but with an associated multiplicity of $2$). Note also that setting a multiplicity value to $m_{a_{ij}} = 0$ has the effect of "turning off" the associated $(\alpha - a_{ij})$ term.

This is especially useful for range checks, which the [Miden VM docs](https://0xpolygonmiden.github.io/miden-vm/design/range.html) have a good introduction about.

## The problem with multiplicities

Adding multiplicities creates challenges when implementing the new equation in STARKs, as it makes the running product column discussed in the previous note no longer viable. Specifically, the transition constraint for the running product column $p$ would need to be:

$$
p[i+1] = p[i] \times \prod_{j=1}^k \frac{(\alpha - a_{ij})^{m_{a_{ij}}}}{(\alpha - b_{ij})^{m_{b_{ij}}}}.
$$

The right-hand side equation is *not* a polynomial, since both $m_{a_{ij}}$ appears in the exponent position, and is itself a polynomial (and similarly for $m_{b_{ij}}$). LogUp was designed to solve that problem.

## LogUp

The core idea of LogUp is relatively simple. It involves applying the logarithm to both sides of the multiset check with multiplicities equation, followed by a derivative.

> Note that the logarithm and the derivative are concepts defined over the real numbers. The [paper](https://eprint.iacr.org/2022/1530.pdf) goes into the formalism of how to formally think about these operations over finite fields, which we will take for granted in this note.

Start with the multiset check with multiplicities equation,

$$
\prod_{i=1}^{n-1} \prod_{j=1}^k (\alpha - a_{ij})^{m_{a_{ij}}} = \prod_{i=1}^{n-1} \prod_{j=1}^k (\alpha - b_{ij})^{m_{b_{ij}}},
$$

apply the logarithm to both sides (and a few logarithm rules),

$$
\begin{align}
\log \prod_{i=1}^{n-1} \prod_{j=1}^k (\alpha - a_{ij})^{m_{a_{ij}}} &= \log \prod_{i=1}^{n-1} \prod_{j=1}^k (\alpha - b_{ij})^{m_{b_{ij}}} \\
\sum_{i=1}^{n-1} \sum_{j=1}^k \log ((\alpha - a_{ij})^{m_{a_{ij}}}) &= \sum_{i=1}^{n-1} \sum_{j=1}^k \log ((\alpha - b_{ij})^{m_{b_{ij}}}) \\
\sum_{i=1}^{n-1} \sum_{j=1}^k m_{a_{ij}} \log (\alpha - a_{ij}) &= \sum_{i=1}^{n-1} \sum_{j=1}^k m_{b_{ij}} \log (\alpha - b_{ij}), \\
\end{align}
$$

and then apply the derivative to both sides (where $x$ will stand for the "current" and "next" row, the inputs to $a_{ij}$ and $b_{ij}$)

$$
\begin{align}
\frac{d}{dx} \sum_{i=1}^{n-1} \sum_{j=1}^k m_{a_{ij}} \log (\alpha - a_{ij}) &= \frac{d}{dx} \sum_{i=1}^{n-1} \sum_{j=1}^k m_{b_{ij}} \log (\alpha - b_{ij}) \\
\sum_{i=1}^{n-1} \sum_{j=1}^k m_{a_{ij}} \frac{d}{dx} \log (\alpha - a_{ij}) &= \sum_{i=1}^{n-1} \sum_{j=1}^k m_{b_{ij}} \frac{d}{dx} \log (\alpha - b_{ij}) \\
\sum_{i=1}^{n-1} \sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})} &= \sum_{i=1}^{n-1} \sum_{j=1}^k \frac{m_{b_{ij}}}{(\alpha - b_{ij})}.
\end{align}
$$

And that's it! The final form of the LogUp equation is

$$
\sum_{i=1}^{n-1} \sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})} = \sum_{i=1}^{n-1} \sum_{j=1}^k \frac{m_{b_{ij}}}{(\alpha - b_{ij})}
$$

## Implementing LogUp in STARKs

LogUp in STARKs is going to look very similar to standard multiset checks: we will implement a *running sum* auxiliary column instead of a *running product*. Let's see how that would work. But first, we need to rearrange the LogUp equation slightly by subtracting the right-hand side on both sides:

$$
\begin{align}
\sum_{i=1}^{n-1} \sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})} - \sum_{i=1}^{n-1} \sum_{j=1}^k \frac{m_{b_{ij}}}{(\alpha - b_{ij})} &= 0 \\
\sum_{i=1}^{n-1} (\sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})} - \frac{m_{b_{ij}}}{(\alpha - b_{ij})}) &= 0
\end{align}
$$

Our running sum column $s$ is going to be built similarly to the running product column for multiset checks. The first row will always contain $0$, and each new row will start with the previous row's value, add $\sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})}$, and subtract $\sum_{j=1}^k \frac{m_{b_{ij}}}{(\alpha - b_{ij})}$:

| Row | $s$ |
| ---- | ------------- |
| 1 | 0 |
| 2 | $\sum_{j=1}^k \frac{m_{a_{0j}}}{\alpha - a_{0j}} - \frac{m_{b_{0j}}}{\alpha - b_{0j}}$ |
| 3 | $\sum_{i=1}^2 \sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})} - \frac{m_{b_{ij}}}{(\alpha - b_{ij})}$ |
| 4 | $\sum_{i=1}^3 \sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})} - \frac{m_{b_{ij}}}{(\alpha - b_{ij})}$ |
| $\dots$ | $\dots$ |
| $n-1$ | $\sum_{i=1}^{n-2} \sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})} - \frac{m_{b_{ij}}}{(\alpha - b_{ij})}$ |
| $n$ | $\sum_{i=1}^{n-1} \sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})} - \frac{m_{b_{ij}}}{(\alpha - b_{ij})}$ |

### Boundary constraints

In the boundary constraints, we make sure that the first and last row of $s$ contain $0$:

$$
s[1] = s[n] = 0
$$

### Transition constraint

Then, the transition constraint needs to ensure that every row $i$ (except the last) is built by taking the value in the previous row, add in $\sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})}$, and subtract out $\sum_{j=1}^k \frac{m_{b_{ij}}}{(\alpha - b_{ij})}$:

$$
s[i+1] = s[i] + (\sum_{j=1}^k \frac{m_{a_{ij}}}{(\alpha - a_{ij})}) - (\sum_{j=1}^k \frac{m_{b_{ij}}}{(\alpha - b_{ij})}).
$$

However, the current form is not a valid polynomial expression. We need to rearrange the equation so that we remove the division operations for equivalent multiplications:

$$
\begin{align}
& s[i+1] (\prod_{j=1}^k \alpha - a_{ij})(\prod_{j=1}^k \alpha - b_{ij}) = \\
& s[i] (\prod_{j=1}^k \alpha - a_{ij})(\prod_{j=1}^k \alpha - b_{ij}) \\
&+ \sum_{j=1}^k m_{a_{ij}} \prod_{e \neq j}(\alpha - a_{ie}) \prod_{e =1}^k (\alpha - b_{ie}) \\
&- \sum_{j=1}^k m_{b_{ij}} \prod_{e \neq j}(\alpha - b_{ie}) \prod_{e =1}^k (\alpha - a_{ie})
\end{align}
$$

This equation is a little difficult to parse, but it is simply the result of multiplying out all the denominators on both sides. For example, if $k=1$, then the equation would look like:

$$
\begin{align}
&s[i+1](\alpha - a_{i0})(\alpha - b_{i0}) = \\
&s[i](\alpha - a_{i0})(\alpha - b_{i0}) + m_{a_{i0}}(\alpha - b_{i0}) - m_{b_{i0}}(\alpha - a_{i0})
\end{align}
$$

Ultimately, the running sum column implementing LogUp achieved what the running product column could not: a polynomial transition constraint.

### Transition constraint degree

LogUp is a good choice for use cases that can't do without representing multiplicities. However, as we'll see, we pay with a higher transition constraint degree. Specifically, the transition constraint degree is the maximum of

$$
\begin{align}
&1 + \sum_{j=1}^k (\deg a_{ij} + \deg b_{ij}), \\
&\forall j,\quad \deg m_{a_{ij}} + \sum_{e \neq j}\deg a_{ie} + \sum_{e =1}^k \deg b_{ie}, \quad \mathrm{and} \\
&\forall j,\quad \deg m_{b_{ij}} + \sum_{e \neq j}\deg b_{ie} + \sum_{e =1}^k \deg a_{ie}. \\
\end{align}
$$

The biggest difference with the running product column transition constraint degree (other than the fact that we now have multiplicities) is that now the degree of the $a_{ij}$'s and the $b_{ij}$'s *add together*.

Hence, the rule of thumb is that between a standard multiset check and LogUp, prefer to use a standard multiset check *when possible* to lower the transition constraint degree.