or
or
By clicking below, you agree to our terms of service.
New to HackMD? Sign up
Syntax | Example | Reference | |
---|---|---|---|
# Header | Header | 基本排版 | |
- Unordered List |
|
||
1. Ordered List |
|
||
- [ ] Todo List |
|
||
> Blockquote | Blockquote |
||
**Bold font** | Bold font | ||
*Italics font* | Italics font | ||
~~Strikethrough~~ | |||
19^th^ | 19th | ||
H~2~O | H2O | ||
++Inserted text++ | Inserted text | ||
==Marked text== | Marked text | ||
[link text](https:// "title") | Link | ||
 | Image | ||
`Code` | Code |
在筆記中貼入程式碼 | |
```javascript var i = 0; ``` |
|
||
:smile: | ![]() |
Emoji list | |
{%youtube youtube_id %} | Externals | ||
$L^aT_eX$ | LaTeX | ||
:::info This is a alert area. ::: |
This is a alert area. |
On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?
Please give us some advice and help us improve HackMD.
Do you want to remove this version name and description?
Syncing
xxxxxxxxxx
Optimization on manifolds
References
Metagoals
"Riemannian Adaptive Optimization Methods"
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Gradient descent
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Manifolds
An \(n\)-manifold: "a 'nice' top. space \((M, \Theta)\) whose every \(p\in M\) has open neighbourhood homeomorphic to (a piece of) \(\mathbb{R}^n\)"
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →\[\phi: U\xrightarrow{\sim} \phi(U)\subset \mathbb{R}^n\]
Manifolds. Missing tools
Smooth manifolds
"A manifold covered with a maximal atlas of smoothly-compatible charts", \((M, \Theta, \mathcal{A})\)
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Smooth functions
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Directional derivatives
\[(f\circ \gamma)'(0).\]
Directional derivatives
\[X_{\gamma, p}f = (f\circ \gamma)'(0)\]
"Directional derivative of \(f\) along \(\gamma\) at \(p\)"
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Directional derivatives
\[X_{\gamma, p}f = (f\circ \gamma)'(0).\]
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Tangent space
\[T_pM = \left\{ X_{\gamma,p}\left| \gamma:I\to M~\text{smooth}, ~\gamma(0)=p \right. \right\}\]
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Coordinate basis
A chart \((U,x)\), \(p\in U\), corresponds to a basis in \(T_p M\), represented by coordinate curves.
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Exponential map
TODO: move to after geodesics
curvegeodesic \(\gamma\) representing \(X\) is unique\[\exp_pX = \gamma(1)~\text{if defined}.\]
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Smooth manifolds. Missing tools
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Given objective \(f:M\to\mathbb{R}\) and \(p_t\in M\), how to pick \(X_t\)?
Derivative of a function
Given smooth \(F:M\to N,\)
"the push-forward of \(F\) at \(p\)" is the linear map \(F_{*,p}:T_p{M}\to T_{F(p)}N\)
\[(\underbrace{F_{*,p}X}_{T_{F(p)}N})\underbrace{g}_{C^\infty(N)} = \underbrace{X}_{T_p M}(\underbrace{g\circ F}_{C^\infty(M)}).\]
Derivative of objective
Given objective \(f:M\to \mathbb{R}\) its derivative at point \(p\in M\) is a linear functional \[T_p{M}\to T\mathbb{R}\cong \mathbb{R}.\]
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Metric manifolds
"A smooth manifold endowed with smooth assignment of (almost) 'inner products' \(g:T_p M\times T_p M\to \mathbb{R}\) at each \(p\)". Notes for details
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →RGD
\[X_t = \operatorname{grad}f(p_t) \equiv f_{*,p_t}^\sharp,\] \[x_{t+1} = \exp_{p_t}(-\alpha X_t).\]
Ready to implement?
Extrinsic representation
Immersion: smooth \(\phi: M\to\mathbb{R}^n\) such that \(\phi_{*,p}\) is injective for every \(p\in M\). Embedding: a diffeomorphic immersion.
[Strong] Whitney: can embed in \(\mathbb{R}^{2\operatorname{dim}M}.\)
Extrinsic representation
\(M\subset\mathbb{R}^d\) be a level-set (\(\{p\left|F(p)=0\right.\}\)).
\(\left.\operatorname{id}\right|_{M}\) is an embedding and
can represent points and tangents via (redundant) \(\mathbb{R}^d\) coordinates,
can induce metric from Euclidean using \(F\),
need projection op.
(prevalent approach, including geoopt)
(examples using local charts?)
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Retractions
\[R_p:T_p M \to M,\] \[R_p(0) = p,\] \[(R_p)_{*,0} "=" \operatorname{id}_{T_p M}.\]
Faster methods
Vector fields
A Bundle is a triple \((E, \pi, M)\), \[E\xrightarrow{\pi} M\]
Vector fields
Tangent bundle: \[E = TM = \cup_p T_p M\] \[\pi = X_{\gamma, p} \mapsto p.\]
Vector fields
A smooth section of a bundle \(E\xrightarrow\pi M\) is a smooth map \[\sigma:M\to E\] such that \[\pi\circ\sigma = \operatorname{id}_M\]
Set of all smooth sections is denoted \(\Gamma(E)\).
Vector fields
A vector field is a smooth section of the tangent bundle \(TM\).
Another example: metric \(g\) is a smooth section of the bundle of bilinear forms (\((0,2)\)-tensor field)
Covariant derivative
"Covariant derivative of vf \(Y\) along \(X\)": \[\nabla_XY\] Wishlist:
Covariant derivative
In chart:
…
(Christoffel symbols)
Levi-Civita connection
…
Vector field along a curve
Extendible if there is \(\tilde{X}\in\Gamma(TM)\) s.t. \(X_\lambda = \tilde{X}_{\gamma(\lambda)}.\)
Parallel translate
If \(X:I \to TM\) is a vf along \(\gamma\) extendible to \(\tilde{X}\) and \[\nabla_X\tilde{X} = 0\] then \(X\) is called autoparallely-transported
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →Vector transport
(here should go some qualitative derivation vector transports for levelsets)
Newton
later addition
Newton
later addition
Newton
later addition
Newton
later addition
Newton
Solve \[\nabla_{X_t}\operatorname{grad}f(p_t) = -\operatorname{grad}f(p_t)\] for \(X_t\in T_{p_t}M\).
\[p_{t+1} = \exp_{p_t}X_t.\]
Accelerated first-order methods
"Coordinates" on manifolds?
Product manifolds
\[M = M_1\times \cdots\times M_n,\]
Product topology, product metric, …
RAdam: adaptivity term for each product component!
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →\(\varkappa\)-Stereographic space
\(\varkappa\)-Stereographic space