---
# System prepended metadata

title: Partial Privacy loss

---

# Partial Privacy loss

Rényi DP:

$$
D^{\alpha}(M(q(D)) \Vert M(q(D'))) \leq \rho
$$
for all adjacent D, D'.

Per-instance Rényi DP:

- Fix D to always be the same database.
- Now, D' is the only thing determining $D^{\alpha}$
- Now, assume $M(q(D))$ is the origin (zero vector).

$$
D^{\alpha}_{\text{per instance}}(0^n \Vert M(q(D'))) \leq \rho
$$

Therefore, the privacy loss of a point is determined only by the value of $M(q(D'))$ for any $D'$.

Rényi divergence between Gaussians:

$$
\alpha \Vert \mu_1 - \mu_2 \Vert^2_2 \over 2 \sigma^2
$$

In our case: $\mu_1 := \nabla_D$ and $\mu_2 := \nabla_{D'}$

For deep learning:

$$
\alpha \Delta^2 \over 2 \sigma^2
$$

because $\Vert \nabla_D - \nabla_{D'} \Vert^2 \leq \Delta^2$ for all adjacent $D, D'$.

For per-instance privacy loss:

$$
\alpha \Vert \nabla_x \Vert^2_2 \over 2 \sigma^2
$$

The per-instance privacy loss of a point is determined only by its gradient norm. (Because $\alpha$ and $\sigma$ are constant).

Note: The per-instance privacy loss is a sensitive quantity and cannot be released or used to calibrate noise. It's NOT possible to achieve any type of DP using per-instance sensitivity.

But, you can use the quantity provided you then privatise ANY computation done using this quantity with global sensitivity and global (worst-case) privacy accounting.

The partial privacy loss is defined as:

$$
{\partial \over \partial x} \Vert \nabla_{\theta} \Vert^2_2 = 2 \nabla_{\theta} \nabla_x \nabla_{\theta}
$$