---
# System prepended metadata

title: GAN project weekly report
tags: [' project', GAN]

---

---
title: GAN project weekly report
tags: GAN, project
---
# Weekly report
[TOC]

[GAN](https://hackmd.io/iHQIBWpFSS2GigHoDufO3Q?both#Deep-learning-Flow)

---
## 待討論
1. 我們也要每週禮拜二中午12. 前寫完ㄇ？？
    >sure
2. 要訂看完論文的期限ㄇ？？
    >yes, 還是每週訂一個進度嗎?

---
## July 22 - 28

### Joann
- PCA
    - method
- Model Collapse
    >[read more...](https://hackmd.io/iHQIBWpFSS2GigHoDufO3Q?both#Model-Collapse)
- dimensionality reduction
    - PCA
    - t-SNE
- Obtain Optimal $z^*$
    >$z^∗$- search & BEGAN-CS
- Disentangled Reprepesentation Learning
    >由某一個較低維度的latent representation z所產生，而資料也可以藉由控制這個z所生成
    - VAE
    - InfoGAN
- What is CelebA dataset?
    > a large-scale face attributes dataset
- Why unsupervised learning?
    >allow the model to work on its own to discover information and perform more complex processing tasks compared to supervised learning
    - VAE(Variational Autoencoder)
    - InfoGAN
- Distributed vs Disentangled Representation
- Auto-encoder
    >輸入一筆input data經過類神經網路後，也要得到跟 input data一模一樣的data (可以拆成Encoder 和 Decoder 兩個neural network)
- What is a Per-Pixel Loss Function?
    >Used as a metric for understanding differences between images on a pixel level.
- What is Disentangled Representation Learning?
    >Breaks down, or disentangles, each feature into narrowly defined variables and encodes them as separate dimensions.
- What is latent space?
    >abstract multi-dimensional space
- What is Back-Propagation(BP)?
    > a widely used algorithm in training feedforward neural networks for supervised learning.
- [完整筆記在這ㄦ~全部打進來太多惹QQ](https://hackmd.io/MiYVyg6uQvu_4n9MWzt7Iw?both)

Q: 除了寫進度之外要不要確切讀到了些什摩也大概寫一寫ㄚ？？這樣如果有一樣的問題就不用重複再查還可以討論
>補上了~但太多還是先放個大概，完整版附上了 :crying_cat_face: 

---

### Joycelyn's
- Thesis: 
    - **latent vector**: meaningful feature
    - use Principle Component Analysis(**PCA**): essential technique in order to find the latent vector. 
    - **disentangled representations**: 把整張高維圖片拆解降維成 meaningful characteristics, like a nose or a 肉圓.
    - Suggested loss function for
        - discriminator: ReLU function
        - generator: sigmoid function

- Linear Algebra:
    - [**Singular Value Decomposition**](https://www.youtube.com/watch?v=TX_vooSnhm8&list=PLE7DDD91010BC51F8&index=31) （看一半就快完了啦:eyes: 

- Topic brainstorming
    - GAN + facial recognition + NLP -> personal 小護士 :female-doctor: 

- To be continued: 
    - [A Gentle Introduction to Generative Adversarial Networks (GANs)](https://machinelearningmastery.com/what-are-generative-adversarial-networks-gans/) 
    - [thispersondoesnotexist.com](https://www.thispersondoesnotexist.com)
---
### Yang
- CelebA(CelebFaces Attribute 名人人臉屬性數據集)
- BEGAN
    - 新的簡單的GAN結構，使用標準的訓練方式不加訓練trick也能很快且穩定的收斂
    - 對於G及D的能力提出均衡的概念
    - 提供一個超參數，可在圖像的多樣性和生成質量做均衡
    - 提出一種收斂程度的估計，這個機制只在WGAN出現過(靈感來自WGAN)
    - loss function
    - Equilibrium
- WGAN
    - 提出Wasserstein距離代替JS Divergence
- DCGAN
    - 網路結構
    - Deconvolution
    - 訓練方法
    - GAN改進
- PCA
- [完整筆記](https://hackmd.io/ej9lseR4QzaROxy6WPv0eg?both)

---
## July 28 - Aug 4

### Joycelyn's

- Basics:
    - [GAN implementation](https://www.youtube.com/watch?v=DQNNMiAP5lw)
    - [Machine Learning](https://www.youtube.com/watch?v=CXgbekl66jc&list=PLJV_el3uVTsPy9oCRY30oBPNLCo89yu49) - 1, 2, 3, 4
    - **What is Machine Learning?**
        - Looking for a function ==from data==
    - **Machine Learning Framework (流程)**
 ![](https://i.imgur.com/bDbf2hl.jpg)
    > **This is supervised learning.**
    > Step1: Give a set of function (ex. f(cat image) = "cat")
    > Step2: 讓 machine 可以去衡量這個 function 是好還是不好
    > Step3: 讓 machine 自己去找到最好的 function
    


    -  **Machine Learning Task**
        - Regression: The output of the target function f is "scalar"

- Thesis: chapter 3 ~ 5
    - So like if we actually do a research using GAN we would have to invent a new GAN-branch like dear嘉哲？？

---
### Joann
#### thesis : 
- [x] chapter 4~5
- experiment
    - <font color=red>k value suddenly drop, BEGAN collapse shortly</font>
        - $\alpha$ : controls the important of the constraint loss (set to 1 as the default value)
        - k value : how much attention is paid on $\mathcal{L}_G$
        - M value : value of the convergence measure
    - when use M value, the figure seems not observation
- constraint loss
- effective
    - preventing from mode collapse
    - very good performance in generating high-quality images
- mode collapsing
    - the precise timing is unpredicted
    - highly related to : 
        1. the resolution of generated images
        2. th size of training dataset
    - suppressed after adding the constraint loss
- small sized dataset
    - BEGAN-CS can generated high-quality result than original
    - BEGAN collapse earlier than original
    - keep BEGAN from converging to an optimal state
    - BEGAN can **learn salient** and **high-quality disentangled representation** in an <font color=red>unsupervised setting</font>
- advantages of BEGAN-CS
    - approximate $z^*$ on-the-fly
    - $G(z)$ looks similar to $x$
    - exhibit the adjustable disentangled properties

#### basis
- [Machine learning](https://www.youtube.com/watch?v=CXgbekl66jc&list=PLJV_el3uVTsPy9oCRY30oBPNLCo89yu49) : 5/35 episodes 
    - [**<font color=red>ML note</font>**](https://hackmd.io/3JJW0_70SL-fhwavFcmRhg)
- [deep learning theory](https://www.youtube.com/watch?v=KKT2VkTdFyc&list=PLJV_el3uVTsOh1F5eo9txATa4iww0Kp8K) : 0/10 episodes
- [GAN introduction](https://www.youtube.com/watch?v=DQNNMiAP5lw&list=PLJV_el3uVTsMq6JEFPW35BCiOQTsoqwNw) : 0/10 episodes
- supervised(predictive) v.s. unsupervised(descriptive)
    - supervised : to learn a mapping from inputs x to outputs y, given a labeled set of input-output pairs
    - unsupervised : to find “interesting patterns” in the data
        - much less well-defined problem
        - no obvious error metric to use

#### news
[Recognising fake images using frequency analysis](https://news.rub.de/english/press-releases/2020-07-16-information-technology-recognising-fake-images-using-frequency-analysis)

#### terms
- measure of convergence
    - [Convergence of measures](https://encyclopediaofmath.org/wiki/Convergence_of_measures)
    - <font color=red>not to be confused with **convergence in measure**</font>

- spectral normalization
    - [SN-GAN簡介](https://medium.com/@xiaosean5408/sn-gan%E7%B0%A1%E4%BB%8B-spectral-normalization-for-generative-adversarial-networks-f8fd784f2ad)
    - 新的 weight normalization 的方式防止GAN在training時不穩
    - 透過 Lipschitz constant 對每層的輸出做限縮
    - 控制 Lipschitz constant，不需複雜的參數調整
    - 實作簡單並且計算需求少
    - 數學式太多好難理解...(會用到SVD)
---
### yang
- thesis : 
[learning note](https://hackmd.io/Q5SfSlibTnmuK57jMGRs3w)

---
## Aug 4 - 11

### Joycelyn's
- Basic:
    - Machine Learning lecture: 5 6 7 8 9

---
### Joann
- basic
    - [Machine learning](https://www.youtube.com/watch?v=CXgbekl66jc&list=PLJV_el3uVTsPy9oCRY30oBPNLCo89yu49) : 6-12/35 episodes 
        - [Machine Learning note](https://hackmd.io/3JJW0_70SL-fhwavFcmRhg)
        - [deep learning note](https://hackmd.io/xv5OKCzJSjChHfhpqemORg)
---
### yang

---
## Aug 11 - 18
### Joycelyn's
- basic
    - [Machine learning lecture](https://www.youtube.com/watch?v=CXgbekl66jc&list=PLJV_el3uVTsPy9oCRY30oBPNLCo89yu49): 10 11 12 13 14 15

---
### Joann
- basic
    - machine learning lecture : 13-20, 21(14'47)
        - [deep learning note](https://hackmd.io/xv5OKCzJSjChHfhpqemORg)
        - semi-supervised的一半筆記轉到ML

---

---
## Aug 18 - 25
### Joycelyn's
- basic
    - [Machine learning lecture](https://www.youtube.com/watch?v=CXgbekl66jc&list=PLJV_el3uVTsPy9oCRY30oBPNLCo89yu49): 16
    - [Deep Learning Homework](https://drive.google.com/drive/folders/1D0WtaDQWFKiXL9zXNEu3h1aa2V2Z5Dx5?usp=sharing)
### Joann
- basic
    - machine learning lecture : 25
    - [Deep learning lecture](https://www.youtube.com/watch?v=KKT2VkTdFyc&list=PLJV_el3uVTsOh1F5eo9txATa4iww0Kp8K) : 1-3
        - [deep learning note](https://hackmd.io/xv5OKCzJSjChHfhpqemORg)

- deep learning hw
  - hw1 practice(pytorch)
  - [pytorch 語法tutorial note](https://hackmd.io/pY7ABMeuTfqY4cRGdLGfuQ)

---

---
## Aug 25 - Sep 1
### Joycelyn's
- Topic brainstorming
    - [Rewriting the rules of machine-generated art](https://news.mit.edu/2020/rewriting-rules-machine-generated-art-0818)
        - An artificial intelligence tool lets users edit generative adversarial network models with simple copy-and-paste commands.
        - [GAN Paint](http://gandissect.res.ibm.com/ganpaint.html?project=churchoutdoor&layer=layer4), a tool that lets users add and remove features like doors and clouds from a picture. 
            - OMG this is f**king cool !!
        - "If we can rewrite the rules of a GAN directly, the only limit is human imagination."
        - Big datasets are not essential, you can edit the numerical weights in its layers to get the behavior you desire, even if no literal example exists. 
        - "It's like inserting the DNA of a firefly into a plant to make it glow in the dark."
        - “The neural network has different memory banks that function as a set of general rules, relating one set of learned patterns to another, if you could identify one line of memory, you could write a new memory into it.” 
---
### Joann
- basic
    - [deep learning lecture](https://www.youtube.com/watch?v=KKT2VkTdFyc&list=PLJV_el3uVTsOh1F5eo9txATa4iww0Kp8K) : 4 - 7
    - [keras toutorial](https://hackmd.io/JWMXlWCFQQCspIxLlJSKjQ)
- deep learning hw
    - hw1 pytorch practice
    - [pytorch語法tutorial note](https://hackmd.io/pY7ABMeuTfqY4cRGdLGfuQ)
- problem : 
    I cannot use the code `cuda`, e.g. 
    ```python=
    args.cuda:
    device = torch.device('cuda')
    ```
    because my GPU is not the product by NVIDIA
- [上網找的回答(1) 但我看不懂@@](https://www.thinbug.com/q/58408143)
- pytorch官網, 這個不知道是不是可切換成cpu??
    - ![](https://i.imgur.com/MTdQ7Y2.png)


---

---
## Sep 1 - 8

---
### Joann
- basic:
  - [keras](https://keras.io/getting_started/intro_to_keras_for_engineers/)
  - [keras note](https://hackmd.io/JWMXlWCFQQCspIxLlJSKjQ)
  - machine learning lecture : 23, 30
      - (word embedding, RNN part 1)


### Joycelyn's
- [Deep Learning for NLP: ANNs, RNNs and LSTMs explained!](https://towardsdatascience.com/deep-learning-for-nlp-anns-rnns-and-lstms-explained-95866c1db2e4)

---
## Sep 8 - 15
### Joann
- basic : 
    - deep learning lecture : 8
    - machine learning lecture : 31 (26'19)
        - RNN

### Joycelyn's
- Topic brainstorming
    - [AI researchers use heartbeat detection to identify deepfake videos](https://venturebeat.com/2020/09/03/ai-researchers-use-heartbeat-detection-to-identify-deepfake-videos/)
        - “We propose a deepfake source detector that **predicts the source generative model** for any given video. To our knowledge, our approach is the first to conduct a deeper analysis for source detection that interprets residuals of generative models for deepfake videos. Our key finding emerges from the fact that we can **interpret these biological signals as fake heartbeats** that contain a signature transformation of the residuals per model. Thus, it gives rise to a new exploration of these biological signals for not only determining the authenticity of a video, but also classifying its source model that generates the video.”
- [Kaggle competition](https://www.kaggle.com/competitions)



# Topic branstorming
- [Rewriting the rules of machine-generated art](https://news.mit.edu/2020/rewriting-rules-machine-generated-art-0818)
    - An artificial intelligence tool lets users edit generative adversarial network models with simple copy-and-paste commands.
    - [GAN Paint](http://gandissect.res.ibm.com/ganpaint.html?project=churchoutdoor&layer=layer4), a tool that lets users add and remove features like doors and clouds from a picture. 
    - "If we can rewrite the rules of a GAN directly, the only limit is human imagination."
    - Big datasets are not essential, you can edit the numerical weights in its layers to get the behavior you desire, even if no literal example exists. 
    - "It's like inserting the DNA of a firefly into a plant to make it glow in the dark."
    - “The neural network has different memory banks that function as a set of general rules, relating one set of learned patterns to another, if you could identify one line of memory, you could write a new memory into it.” 
- [AI researchers use heartbeat detection to identify deepfake videos](https://venturebeat.com/2020/09/03/ai-researchers-use-heartbeat-detection-to-identify-deepfake-videos/)
    - Russian troll farm: 俄羅斯政治網軍
    - Deepfakes can make epic memes or put Nicholas Cage in every movie, but they can also undermine elections. 
    - Existing deepfake detection models focus on traditional media forensics methods, like tracking unnatural eyelid movements or distortions at the edge of the face.
    - “We propose a deepfake source detector that **predicts the source generative model** for any given video. To our knowledge, our approach is the first to conduct a deeper analysis for source detection that interprets residuals of generative models for deepfake videos. Our key finding emerges from the fact that we can **interpret these biological signals as fake heartbeats** that contain a signature transformation of the residuals per model. Thus, it gives rise to a new exploration of these biological signals for not only determining the authenticity of a video, but also classifying its source model that generates the video.”
    - The first study for detection of [unique GAN fingerprints](https://arxiv.org/abs/1811.08180) was introduced in 2018.
        - GANs have shown increasing success in generating photorealistic images （寫實派影像）
        - learning GAN fingerprints towards **image attribution** and using them to classify an image as real or GAN-generated
        - GAN fingerprints:
            - are stable
            - minor differences in GAN training can result in different results.
            - persist across different image frequencies and patches (like smaller blocks).
            - finetuning (調整) is effective in immunizing five different types of adversarial image perturbations (擾動). 
        - [Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints](https://arxiv.org/pdf/1811.08180.pdf)
    - photoplethysmography PPG
- [Kaggle competition](https://www.kaggle.com/competitions)