Try   HackMD

目標/方向

目標

  1. 圖片導向的風格轉換

時程

https://mermaid.js.org/syntax/gantt.html

06-2907-0607-1307-2007-2708-03討論風格轉換可行性           修改loss function     model比較

待完成

  • 嘗試在CLIP space 做修改，讓細節模糊化
  • 產生一個結果比較的圖片(包含原圖、用文字guide(目標風格文字&結果)、圖片guide(目標風格圖片&結果))

目前結果

原本的content loss不變

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

將CLIP loss的Ptarget(文字導向) 改為一個風格導向圖片y

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

DCLIP():在CLIP embedding space(文字和圖片都被embedded進去的空間）中，generated image和目標風格（文字）的距離

目前問題

原始的做法中有提到 在產出圖片時可能會出現文字本身印在圖片上面

原始的作法對顏色的敏感程度較高 使用圖片修改難以改變色調

原始作法可能會因為CLIP背後的資料集有特定傾向而導致更改的結果有問題 例如一風格幾乎都是肖像畫 按照此風格產出的圖片有高機率會藏人臉

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

而用圖片更改則是有可能會將圖片的內容混進去

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

目前結果

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

參考文獻

Zecon paper

Last changed by 
 
每日關心專題進度
專題
0
82

Read more

討論紀錄

<input class="task-list-item-checkbox" disabled="" type="checkbox"> 針對特定的圖片部分 觀察CLIP score有無變化

Feb 20, 2024
研究文章

擴散模型在圖像風格轉換中的表現一直都很出色，但最大的問題是模型本身速度不快，且擴散模型的隨機性也影響了產出的內容。大部分現有的方法需要對擴散模型進行微調，或者用額外的神經網絡。而我們使用了一種不需要額外訓練，直接使用額外的loss function來試圖將預訓練擴散模型的輸出導向到想要的方向。通過這種方法來提高使用者建構的速度，而不用花太多時間來微調擴散模型，並與其他現有不同的圖像風格轉換方法來做比較。

Dec 13, 2023
論文架構

作法A

Dec 12, 2023
風格辨識

CNN

Oct 15, 2023
Read more from 每日關心專題進度
Published on HackMD