Lien de la note Hackmd

Exercice 1

$$\begin{array}{rl}minf(x,y)& =2x+y\\ \text{tel que }3x^2+y^2& \le 4\end{array}$$

Solution

$$\begin{gathered} \begin{array}{rl}3x^2+y^2& \le 4\\ 3x^2+y^2-4& \le 0\\ g(x,y)& =3x^2+y^2-4\end{array} \\ \mathcal{L}(x,y,\alpha )=2x+y+\alpha (3x^2+y^2-4) \\ \{\begin{array}{ll}\frac{\partial \mathcal{L}}{\partial x}=2+6\alpha x=0& \to x=-\frac{1}{3\alpha }\\ \frac{\partial \mathcal{L}}{\partial y}=1+2\alpha y=0& \to y=-\frac{1}{2\alpha }\end{array} \\ \begin{array}{rl}\mathcal{L}(\equiv {\theta }_D(\alpha ))& =2(-\frac{1}{3\alpha })+(-\frac{1}{2\alpha })+\alpha (3(-\frac{1}{3\alpha }{)}^2+(-\frac{1}{2\alpha }{)}^2-4)\\ & =-\frac{2}{3\alpha }-\frac{1}{2\alpha }+\alpha (\frac{1}{3{\alpha }^2}+\frac{1}{4{\alpha }^2}-4)\\ & =-\frac{1}{3\alpha }-\frac{1}{2\alpha }+\frac{1}{3\alpha }+\frac{1}{4\alpha }-4\alpha \\ & =-\frac{1}{3\alpha }-\frac{1}{4\alpha }-4\alpha \\ & =-\frac{7}{12\alpha }-4\alpha \end{array} \\ \begin{array}{rl}{\mathrm{\nabla }}_{\alpha }{\theta }_D(\alpha )& =\frac{7}{12{\alpha }^2}=0\\ & =\frac{7}{12{\alpha }^2}=4\\ & =\frac{1}{4}\frac{7}{12}={\alpha }^2\\ \alpha & =\frac{1}{2}\sqrt{\frac{7}{12}}=\frac{1}{4}\sqrt{\frac{7}{3}}\end{array} \end{gathered}$$

Autre methode, en utilisant la complementarite:

$$\begin{array}{rl}{\alpha }^{\ast }g(x^{\ast },y^{\ast })& =0\\ {\alpha }^{\ast }=(3(-\frac{1}{3{\alpha }^{\ast }})+(-\frac{1}{2{\alpha }^{\ast }}{)}^2)& =0\\ {\alpha }^{\ast }(\frac{1}{3{\alpha }^{{\ast }^2}}+\frac{1}{4{\alpha }^{{\ast }^2}}-4)& =0\\ \frac{1}{3{\alpha }^{\ast }}+\frac{1}{4{\alpha }^{\ast }}-4{\alpha }^{\ast }& =0\\ \frac{7}{12{\alpha }^{\ast }}& =4{\alpha }^{\ast }\\ \frac{1}{4}\frac{7}{12{\alpha }^{\ast }}& ={\alpha }^{{\ast }^2}\\ {\alpha }^{\ast }& =\frac{1}{4}\sqrt{\frac{7}{3}}\end{array}$$

Exercice 2

$$\begin{array}{rl}\underset{x\in {\mathbb{R}}^3}{min}& \frac{1}{2}(x_1^2+x_2^2+x_3^2)\\ \text{tel que }& x_1+x_2+2x_3-1=0\\ & x_1+4x_2+2x_3-3=0\end{array}$$

Sous forme matricielle:

$$\begin{gathered} \boxed{{\displaystyle \begin{array}{rl}min& \frac{1}{2}{x}^{T}x\\ \text{tel que }& Ax-b=0\end{array}}} \\ x=\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right) \\ A=\left(\begin{array}{ccc}1& 1& 2\\ 1& 4& 2\end{array}\right) \\ b=\left(\begin{array}{c}1\\ {}_3\end{array}\right) \end{gathered}$$

Solution

$X=\{S_1,\dots ,S_N\}$ avec proba discrete

$p_i=\mathbb{P}(X=S_i)$ et

$\sum _{i=1}^N{p}_i=1$.

Entropie de Shannon

$$\begin{gathered} H(\mathbb{P}=(p_1,\dots ,p_n))=-\sum _{i=1}^N{p}_i{\log }_2(p_i) \\ {\log }_2(x)=\frac{\ln (x)}{\ln (2)} \end{gathered}$$

La distribution qui maximise l'entropie est la distribution uniforme.

On cherche a maximiser

$$\begin{gathered} H(x)=-\sum _{i=1}^n{x}_i{\log }_2(x_i)\text{pour }x=\left(\begin{array}{c}{x}_1\\ ⋮\\ x_n\end{array}\right)\in {\mathbb{R}}^2 \\ \text{tel que}\sum _{i=1}^n{x}_i=1 \end{gathered}$$

On cherche a minimiser

$$\begin{gathered} -H(x)=\sum _{i=1}^n{x}_i{\log }_2(x_i) \\ \text{tel que }\sum _{i=1}^n{x}_i-1=0 \\ \to h(x)=\sum _{i=1}^n{x}_i-1\text{affine} \\ \begin{array}{rl}\mathcal{L}(x,\beta )& =-H(x)+\beta h(x)\\ & =\sum _{i=1}^n{x}_i{\log }_2(x_i)+\beta (\sum _{i=1}^n{x}_i-1)\end{array} \\ \begin{array}{rl}{\mathrm{\nabla }}_x\mathcal{L}(x,\beta )=0& \iff \mathrm{\forall }i,\{\begin{array}{l}\frac{\partial \mathcal{L}}{\partial x_i}=0\\ \frac{\partial \mathcal{L}}{\partial x_i}=\frac{\partial }{\partial x_i}(x_i{\log }_2(x_i))+\beta \end{array}\\ & \iff \{\begin{array}{l}\frac{d}{dx}(x{\log }_{2}x)={\log }_{2}x+x\frac{d}{dx}{\log }_{2}x\\ \frac{d}{dx}{\log }_{2}x=\frac{d}{dx}\frac{\ln (x)}{\ln (2)}=\frac{1}{x\ln (2)}\end{array}\end{array} \\ \begin{array}{rl}\frac{\partial \mathcal{L}}{\partial x_i}& =\frac{\partial }{\partial x_i}(x_i{\log }_2(x_i))+\beta \\ & ={\log }_2(x_i)+x_i\frac{1}{x_i\ln (2)}+\beta \\ & =\frac{\ln (x_i)+1}{\ln (2)}+\beta =0\end{array} \\ \begin{array}{rl}\frac{\ln x_i+1}{\ln 2}& =-\beta \\ \ln x_i+1& =-\beta \ln 2\\ \ln x_i& =-\beta \ln (2)-1\\ x_i& =e^{-(\beta \ln 2+1)}\mathrm{\forall }i\end{array} \end{gathered}$$

Avec la contrainte:

$$\begin{array}{rl}\sum _{i=1}^n{x}_i& =1\\ \sum _{i=1}{e}^{-(\beta \ln 2+1)}& =1\\ n{e}^{-(\beta )}\\ n{e}^{-(\eta \ln 2+1)}& =1\\ \underset{x_i}{\underbrace{e^{-(\beta \ln 2+1)}}}& =\frac{1}{n}\end{array}$$

La distribution qui maximise l'entropie est donc

$x_i=\frac{1}{n}$

$\mathrm{\forall }i=1\dots n$

$\equiv$ distribution uniforme

:::