Try   HackMD

資專研究札記by賴昱錡

認識常見照片上色模型架構

date: 2022/6/4

vgg16

GAN

CNN

似乎有用到YOLOv3(v4)技術的專案

CycleGAN

pix2pix

PixelCNN

UNet

residual-encoder-network、ResNet

NoGAN(DeOldify)

CVPR 2020 InstColorization

caffe https://modelzoo.co/model/colorful-image-colorization

Photoshop Neural filter (https://www.adobe.com/tw/sensei.html)

之前整理的wmts&GIS相關連結 https://richardlaiiiii.blogspot.com/2022/04/blog-post.html

教授回信內容的整理

date: 2022/6/5

簡單整理以下的問題:

Q1:請問若要開始進行模型的實作，是否要將嘗試過github或一些paper常用的方法(pix2pix、GAN、U-net)都嘗試過?或者在色彩修復這方面，有沒有什麼實作效果較好or較容易實作的模型，除了之前教授所傳的GAN搭配Unet的實作?
A1:你可以用colorizing (colorization) pretrained model當作關鍵詞，可以找到很多案例討論，
例如：https://github.com/moein-shariatnia/Deep-Learning/tree/main/Image Colorization Tutorial
它就直接將程式碼公開（已經使用GAN、U-net模型），就可以複製到Google Colab來修改執行看看。

Q2:請問利用Qtiles所擷取之影像解析度低的問題有解決的方法嗎? 目前看起來似乎QGIS針對.png及.jpg的輸出品質恐怕無法更好。
A2:影像解析度低主要是選取Zoom Level及原始圖資供應單位提供的影像品質所致，與QGIS關係不大。由於QTiles中圖磚（Tile) Width及Height可以加大，因此可以視前項你選擇的模型輸入資料格式，進行調整。

Q3:如果要比較各地區的訓練樣本帶來的影響，這個想法是否可行?意思就是要比較不同地區(可能是台北市、彰化縣…)的黑白影像(訓練用)，是否會影響到訓練的成效。
A3:應該說訓練資料中就應該包括城市、鄉村及純自然環境的影像（資料），這樣訓練出來模型才能應用到臺灣各地，可以將不同的地景都正確上色。

大四學生針對臺灣茶園AI判釋的報告簡報連結
data augmentation
中央研究院人社中心地理資訊科學研究專題中心

查資料，跑網路上提供的模型，學習ML

date: 2022/6/7

https://emilwallner.medium.com/colorize-b-w-photos-with-a-100-line-neural-network-53d9b4449f8d

https://colab.research.google.com/github/moein-shariatnia/Deep-Learning/blob/main/Image Colorization Tutorial/Image Colorization with U-Net and GAN Tutorial.ipynb#scrollTo=2Yddr6q1F2DC

學了一點numpy跟matplotlib的運用，其實很少啦。

專題課

date: 2022/6/9

google colab note

詳細內容
  1. colab介面是一個ipynb的介面，在每個code cell輸入指令
  2. 資料可以上傳到自己的google drive，使用colab時可以mount自己的google drive:


from google.colab import drive
drive.mount('/content/drive') #/content是你colab的主要資料夾，mount後你googledrive在"drive"
  1. 想要執行linux指令（ls,pwd,nvidia-smi…），只要在你想執行的指令前加上“！”即可:



!ls
!nvidia-smi
!python train.py
  1. 為了加速training速度，建議把你的training data從google drive複製到colab的主要資料夾，因為你google drive的實體位置可能距離colab的實體位置有點距離，colab在load data時會延遲:
!cp -R ./drive/'My Drive'/training_data training_data
  1. colab怪怪的，不能隨便用"!"下cd指令換資料夾，但有個他們稱為magic command的符號，"%"，代替"!"後就可以換資料夾了，指令：%cd。








!ls
# output: colabData/
%%writefile something.py
print("everything's fine.")
!ls
# output: colabData/ something.py
%run something.py
# output: everything's fine.

linux 解壓縮指令
http://note.drx.tw/2008/04/command.html

專題課

2022/6/16
認識auto encoder、convolutional auto encoder

專題課

2022/6/23

  • 批次照片轉灰階程式撰寫、crawling
  • 與教授討論後，教授建議訓練資料中就應該包括 城市、鄉村及純自然環境的影像（資料），這樣訓練出來模型才能應用到臺灣各地，可以將不同的地景都正確上色。
  • 酷酷的網站 https://paperswithcode.com/

看論文

2022/7/1

https://arxiv.org/abs/1603.08511 利用CNN實作上色

翻到以下資源，可能有空會看:

計算機視覺三大國際會議
https://eccv2022.ecva.net/
https://iccv2021.thecvf.com/home
https://cvpr2022.thecvf.com/

Siggraph
ACM siggraph
ACM siggraph Taipei

ICLR
https://iclr.cc/
EX: https://github.com/google-research/google-research/tree/master/coltran (https://paperswithcode.com/method/colorization-transformer)

Deep Koalarization: Image Colorization using CNNs and Inception-ResNet-v2

Kaggle https://www.kaggle.com/ 裡面似乎有蠻多別人寫的code(?

GAN
https://github.com/OvaizAli/Image-Colorization-using-GANs

deepai API
https://deepai.org/machine-learning-model/colorizer

https://github.com/MarkMoHR/Awesome-Image-Colorization

https://wacv2022.thecvf.com/node/83

2022/7/13

原文: https://github.com/MarkMoHR/Awesome-Image-Colorization/blob/master/README.md

  1. Automatic Image Colorization
Paper Source Code/Project Link
Learning Large-Scale Automatic Image Colorization ICCV 2015 [project] [code]
Deep Colorization ICCV 2015
Learning Representations for Automatic Colorization ECCV 2016 [project] [code]
Colorful Image Colorization ECCV 2016 [project] [code]
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification SIGGRAPH 2016 [project] [code]
Learning Diverse Image Colorization CVPR 2017 [code]
Structural Consistency and Controllability for Diverse Colorization ECCV 2018
Coloring With Limited Data: Few-Shot Colorization via Memory Augmented Networks CVPR 2019 [project]

Unofficial:
[code (PyTorch)]
ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution WACV 2020 [code]
Instance-aware Image Colorization CVPR 2020 [code] [project]
Pixelated Semantic Colorization IJCV 2020
Colorization Transformer ICLR 2021 [code]
Focusing on Persons: Colorizing Old Images Learning from Modern Historical Movies ACM MM 2021 [code]
Towards Vivid and Diverse Image Colorization with Generative Color Prior ICCV 2021
SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution (also reference based) AAAI 2022
Bridging the Domain Gap towards Generalization in Automatic Colorization ECCV 2022 [code]
ColorFormer: Image Colorization via Color Memory assisted Hybrid-attention Transformer ECCV 2022
DeOldify Software [code]
  1. User Guided Image Colorization

2.1 Based on color strokes

Image Type Paper Source Code/Project Link
Manga Manga colorization SIGGRAPH 2006 [Project Home]
Unofficial: [code]
Line art / Sketch / Manga LazyBrush: Flexible Painting Tool for Hand-drawn Cartoons Eurographics 2009 [Project Home & Software]

Unofficial:
[code1 (Matlab)]
[code2 (C#)]
[code3]
Line art / Sketch Outline Colorization through Tandem Adversarial Networks 1704.08834 [code]
Line art / Sketch Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks 1705.01908 [code]
Natural Gray-Scale Real-Time User-Guided Image Colorization with Learned Deep Priors SIGGRAPH 2017 [project] [code1] [code2]
Sketch Scribbler: Controlling Deep Image Synthesis with Sketch and Color CVPR 2017
Line art User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks ACM MM 2018 [code]
Line art Style2paints V3 : Two-stage Sketch Colorization SIGGRAPH Asia 2018 [Project]
[Paper's code(V3)]
[Portable software(V4.5)]

Unofficial:
[code1] [code2]
[PaintsTensorFlow]
Natural Gray-Scale Interactive Deep Colorization Using Simultaneous Global and Local Inputs (also palette based) ICASSP 2019
Line art PaintsChainer (Petalica Paint) Online Demo (by Preferred Networks, Inc.) [V3 Demo] [V1 code]
Line art User-Guided Line Art Flat Filling with Split Filling Mechanism CVPR 2021 [Project] [code]
Line art Dual Color Space Guided Sketch Colorization TIP 2021

2.2 Based on reference color image

Image Type Paper Source Code/Project Link
Line art Style2paints V1 : Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN ACPR 2017 [Code]

Unofficial:
[code]
Manga Comicolorization: Semi-Automatic Manga Colorization (also palette based) SIGGRAPH Asia 2017 [code]
Sketch TextureGAN: Controlling Deep Image Synthesis with Texture Patches CVPR 2018 [code]
Natural Gray-Scale Deep Exemplar-based Colorization SIGGRAPH 2018 [code]
Natural Gray-Scale Example-Based Colourization Via Dense Encoding Pyramids (also palette based) Pacific Graphics 2018 [code]
Natural Gray-Scale A Superpixel-based Variational Model for Image Colorization TVCG 2019
Natural Gray-Scale Automatic Example-based Image Colourisation using Location-Aware Cross-Scale Matching TIP 2019
Line art / Sketch Adversarial Colorization Of Icons Based On Structure And Color Conditions ACM MM 2019 [Code]
Line art / Sketch Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence CVPR 2020 [project] Unofficial: [code]
Natural Gray-Scale Stylization-Based Architecture for Fast Deep Exemplar Colorization CVPR 2020 [code]
Manga Manga Filling Style Conversion with Screentone Variational Autoencoder (also palette based) SIGGRAPH Asia 2020 [project]
Line art / Sketch Colorization of Line Drawings with Empty Pupils PG 2020
Line art / Sketch Active Colorization for Cartoon Line Drawings TVCG 2020
Natural Gray-Scale Gray2ColorNet: Transfer More Colors from Reference Image ACM MM 2020
Line art / Sketch Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization WACV 2021
Natural Gray-Scale Globally and Locally Semantic Colorization via Exemplar-Based Broad-GAN TIP 2021
Natural Gray-Scale Yes, "Attention Is All You Need", for Exemplar based Colorization ACM MM 2021
Natural Gray-Scale SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution (also automatic) AAAI 2022
Line art / Sketch Style-Structure Disentangled Features and Normalizing Flows for Diverse Icon Colorization CVPR 2022

2.3 Based on color palette

Image Type Paper Source Code/Project Link
Natural Image Palette-based Photo Recoloring SIGGRAPH 2015 [project]
Manga Comicolorization: Semi-Automatic Manga Colorization (also reference based) SIGGRAPH Asia 2017 [code]
Natural Gray-Scale Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation (also text based) ECCV 2018 [code]
Natural Gray-Scale Example-Based Colourization Via Dense Encoding Pyramids (also reference based) Pacific Graphics 2018 [code]
Natural Gray-Scale Interactive Deep Colorization Using Simultaneous Global and Local Inputs (also strokes based) ICASSP 2019
Scene Sketches](http://sweb.cityu.edu.hk/hongbofu/doc/language-based_sketch_colorization_SA19.pdf) SIGGRAPH Asia 2019 [code] [project]
Natural Gray-Scale L-CoDe: Language-based Colorization using Color-object Decoupled Conditions AAAI 2022
  1. Techniques of Improving Image Colorization
Paper Source Code/Project Link
Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects ICCV 2021 [project] [code(metric)]
Line Art Colorization Based on Explicit Region Segmentation Pacific Graphics 2021 [code]

2022/7/16

認識transformer
https://tw.coderbridge.com/series/2ec9cf0af3f74ed99371952f4849ae33/posts/5c495ca5e46e40bc98ff623e87919c9a
https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)

Colorization transformer
https://arxiv.org/abs/2102.04432?context=cs

用Neural Network實作的教學
https://github.com/emilwallner/Coloring-greyscale-images
https://emilwallner.medium.com/colorize-b-w-photos-with-a-100-line-neural-network-53d9b4449f8d

2022/7/21

https://medium.com/axinc-ai/colorization-a-machine-learning-model-for-colorizing-black-and-white-images-829e35e4f91c
https://github.com/axinc-ai/ailia-models/tree/master/image_manipulation/colorization

以VGG(https://ithelp.ithome.com.tw/articles/10192162)實作的上色模型

https://github.com/richzhang/colorization/blob/master/colorizers/siggraph17.py

2022/7/22

https://github.com/junyanz/interactive-deep-colorization
https://github.com/topics/automatic-colorization
https://github.com/topics/coloring-algorithm

requirements.txt的使用

scikit-image: Python一個影像處理的套件

https://algorithmia.com/ Google算法介紹網站

2022/7/26

建立 ML學習筆記 (基礎太差:( )

2022/7/28

Kaggles的使用教學
https://www.kaggle.com/code/qi4589746/kaggle/notebook

用kaggles的資料庫用CNN實作上色
https://www.kaggle.com/code/adithyakag/image-colorization-cnn
https://colab.research.google.com/drive/1aZV3eRFoU5SQG3-e6zClAOQtPBtTSIIf?usp=sharing

Auto encoder的實作
https://www.kaggle.com/code/brsdincer/image-colorization-process-auto-encoder
colab:
https://colab.research.google.com/drive/1fFTXcTq_b0ilBdX_FDrSQcEXzHe6_5Av?usp=sharing

Combining MobileNetV2 and CNN
https://www.kaggle.com/code/liturgy/image-colorization-combining-mobilenetv2-and-cnn
colab:
https://colab.research.google.com/drive/1llkSEp8spgBzbFod-XAEbozXRqHD5rY6?usp=sharing

colorization-using-cnn-and-inception-resnet-v2
https://www.kaggle.com/code/anshul4150/colorization-using-cnn-and-inception-resnet-v2
colab:

2022/7/29

https://keras.io/
https://www.tensorflow.org/
https://pytorch.org/
https://leemeng.tw/deep-learning-resources.html

keras、tensorflow、pytorch的官網，上面蠻多不錯的專案範例可以實作

2022/7/31

看資訊社學長的簡報，熟悉數學、pytorch、keras等基礎知識
https://slides.com/liusean/machine-learning

Machine Learning resources
https://hackmd.io/@DanielChen/r1SkwBBA-?type=view

2022/8/3

找論文不錯的網站 & keras的中文文檔
https://keras.io/zh
https://deepai.org/

2022/8/8

kera模型架構&程式碼
https://ithelp.ithome.com.tw/articles/10234389?sc=iThelpR

How LSTM works?
https://brohrer.mcknote.com/zh-Hant/how_machine_learning_works/how_rnns_lstm_work.html

kaggles tutorial
https://bit.ly/3zCLVQ4

https://www.kaggle.com/code/alexisbcook/getting-started-with-kaggle/notebook

2022/8/9

https://rawpedia.rawtherapee.com/RGB_and_Lab

Kaggles
colorization-using-cnn-and-inception-resnet-v2
https://colab.research.google.com/drive/1EzXGw0fXAa9klZScA0Cjli8OYnSyBJEm?usp=sharing
結果請看colab或以下照片


kaggle.com/code/rajeevctrl/imagecolorization-labcolorspace-fastai/notebook

了解基本matplotlib的語法 (機器學習中紀錄loss、acc會用到)
https://blog.happycoding.today/pythonbeginner-ep8/

2022/8/10

Real-Time User-Guided Image Colorization with Learned Deep Priors
https://arxiv.org/abs/1705.02999

Colorization using deep learning
https://medium.datadriveninvestor.com/coloring-black-white-images-using-deep-learning-984e6f4ddf14
雖然裡面的程式無法執行 QwQ，但有再介紹將圖片轉換為LAB後的訓練過程 (?

2022/8/11

Richzhang的github page，上面似乎有關於電腦視覺更詳細的介紹 (?
http://richzhang.github.io/colorization/

Deep Koalarization: Image Colorization using CNNs and Inception-ResNet-v2 (Code&Paper)
https://paperswithcode.com/paper/deep-koalarization-image-colorization-using

2022/8/12

還沒找到這個project的相關論文 QwQ
https://github.com/mlhubber/colorize

測試結果見下方的colab:
https://colab.research.google.com/drive/1BlGY7m2DYHswEbaOYl1OFVuv7kH5iT7I?usp=sharing

PySimpleGUI photo colorizer
https://github.com/PySimpleGUI/PySimpleGUI-Photo-Colorizer

2022/8/15

稍微讀過以下的資訊:

Floydhub: 可以租用雲端GPU的線上工具
https://newtoypia.blogspot.com/2018/04/floydhub.html

The machine learning hub
https://mlhub.readthedocs.io/en/latest/

跑過一些模型，熟習matplotlib的語法

學會利用ngrok搭建伺服器
https://hackmd.io/@expectlai/HJwKMuwRc

CNN、Deep-Exemplar-based-Colorization
https://github.com/msracver/Deep-Exemplar-based-Colorization

有神人實作多種版本如alpha、beta、GAN等version的上色模型
https://github.com/emilwallner/Coloring-greyscale-images

2022/8/16

pix2pix實作
https://medium.com/@falconives/day-56-pix2pix-21c7df48922c

U-net實作semantics sementation
https://ithelp.ithome.com.tw/articles/10240314
Colab:
https://colab.research.google.com/drive/1rI6GuEWOu6jpuEvkleBrOQDQ2autXZyu?usp=sharing

實作GAN
https://chih-sheng-huang821.medium.com/pytorch手把手實作-generative-adversarial-network-8adae9a3092b
Colab: https://colab.research.google.com/drive/1KeA9D__qa9UwndT3Y1pvjP9qa7ihAZEV?usp=sharing

廖泫銘教授介紹最近加入GIS專題中心的陳哲安學長（蔡宗翰老師的學生），他目前也都在進行pix2pix、GAN、U-net有關的實驗 (主要應用如semantic segmentation、Style transfer)
之後如果有模型相關的問題，我應該能請教學長他。

2022/8/16

Conditional GAN上色的論文
https://arxiv.org/abs/1611.07004

GAN、U-net approach of colorization
https://towardsdatascience.com/colorizing-black-white-images-with-u-net-and-conditional-gan-a-tutorial-81b2df111cd8

The method of displaying a image in python
https://stackoverflow.com/questions/1413540/showing-an-image-from-console-in-python

https://bit.ly/3AvVj9u

Neural network implemention
https://emilwallner.medium.com/colorize-b-w-photos-with-a-100-line-neural-network-53d9b4449f8d

複習常見的神經網路&電腦視覺的模型
https://ithelp.ithome.com.tw/users/20112126/ironman/1971

2022/8/22

https://richzhang.github.io/InteractiveColorization/

學習Python檔案處理、pillow模組的應用

也寫了簡單將圖片轉為黑白的程式，但pil的convert會將圖轉成典陣圖物件，因此畫質會降低許多 ;-;
只能改成ImageOps












# 此檔案可置於專案該資料夾
from PIL import Image, ImageOps
import os
new_path='.\\grey_version'
if not os.path.exists(new_path):
    os.makedirs(new_path)
for filename in os.listdir('./image'):
    pic=Image.open('.\\image\\'+filename)
    new_pic=ImageOps.grayscale(pic)
    # new_pic=pic.convert('1')
    new_pic.save('.\\grey_version\\'+str(filename[:-4]+'_grey_ver.jpg'))

https://medium.com/axinc-ai/colorization-a-machine-learning-model-for-colorizing-black-and-white-images-829e35e4f91c

2022/8/23

https://techtutorialsx.com/2018/06/02/python-opencv-converting-an-image-to-gray-scale/









# 利用cv2轉成黑白
import cv2
image = cv2.imread('./1.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
cv2.imshow('Origin image', image)
cv2.imshow('Grayscale image', gray)

cv2.waitKey(0)

https://arxiv.org/abs/2204.02980 不同loss對上色效果的影響

真的該來認真研究論文、code了，也應開始試著用自己的datasets訓練
https://paperswithcode.com/search?q_meta=&q_type=&q=Image+Colorization

Inception Resnet
https://ithelp.ithome.com.tw/articles/10277071

2022/8/24、8/25

Colorful Image Colorization

Abstract

  • 過往的模型通常依賴用戶互動、色彩不飽和、且在多個物件上的著色上表現頗差
  • Proposing a fully automatic approach that produces vibrant and realistic colorizations
  • The system is implemented as a feed-forward pass in a CNN at test time and is trained on over a million color images.

Features

To appropriately model the multimodal nature of the problem, we predict a distribution of possible colors for each pixel.Furthermore, we re-weight the loss at training time to emphasize rare colors.

(a)designing an
appropriate objective function that handles the multimodal uncertainty of the
colorization problem and captures a wide diversity of colors.
(b)introducing
a novel framework for testing colorization algorithms, potentially applicable to
other image synthesis tasks.
©setting a new high-water mark on the task by
training on a million color photos.

除了看論文，也寄gmail與學長討論，之後可能會約些時間進行實體會面。

https://arxiv.org/abs/1803.05400v5 中有提及結合CNN的GAN: DCGAN (Deconvolutional GAN)
https://www.tensorflow.org/tutorials/generative/dcgan
之後有空可以看

2022/8/27

kera GAN 手寫數字生成
https://ithelp.ithome.com.tw/articles/10208478

ML的技巧: Batch normalization
https://ithelp.ithome.com.tw/articles/10204106

Instance-aware Colorization
https://arxiv.org/abs/2005.10825v1 (跑不動的Colab: https://github.com/ericsujw/InstColorization/blob/master/README_TRAIN.md )
下方為tensowflow的實現(利用U-net)、展示
https://github.com/ariG23498/instance-aware-colorization-TF

原理:
Given a grayscale image as input, the model starts with detecting the object bounding boxes using an off-the-shelf object detection model. We then crop out every detected instance via bounding boxes and use instance colorization network to colorize instances. However, as the instances'colors may not be compatible with respect to the predicted background colors, we propose to fuse all the instances' feature maps in every layer with the extracted full-image feature map using the proposed fusion module. We can thus obtain globally consistent colorization results.

keras model的存檔，這樣可不用重新訓練模型
https://ithelp.ithome.com.tw/articles/10191627

利用Conditional GAN實現圖像翻譯的paper集合，conditional GAN的介紹可以看 https://www.hmoonotes.org/2020/08/conditional-gan.html
https://paperswithcode.com/paper/image-to-image-translation-with-conditional

Modelzoo: 收集許多ML框架、paper的網站，雖然可能不大有用，但先記下來。

2022/8/29

陳哲安學長上週的報告，包含style transfer (cycleGAN、pix2pix)和semantic segmentation的內容，
https://docs.google.com/presentation/d/1_t9j1h4OkvQXbSVMq4vVfFNTmUvesG0i/edit?usp=sharing&ouid=116385839922469449708&rtpof=true&sd=true

建議
可能之後在挑選模型盡量選擇有訓練腳本的論文，否則要複製論文中的模型需要花相當多時間，結果也不一定好。
https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix (CycleGAN、pix2pix)
https://github.com/taesungp/contrastive-unpaired-translation (CUT) 也是使用他寫好的腳本去訓練自定義的資料集、CUT簡介、CycleGAN簡介

稍微整理一下CycleGAN跟CUT的內容吧:

  • CycleGAN是個可以用來產生不同風格/材質轉換的一個神經網路，主要應用在影像對影像的變換，簡單而言是兩個generator (產生假A、假B)，兩個 discriminator (分辨 A/假A、B/假B)組成，想法是將 A domain (例如馬) 的影像轉成 B domain (例如斑馬)，因為是 unsupervised learning 所以不好保證效果，但可以再從 B domain (例如斑馬) 的圖轉回 A domain (例如馬)，轉回的還原圖要跟原本給的 input 越像越好，類似於 auto-encoder 的概念

2022/8/31

一些整理

  1. 其實在圖像處理套件最主要會用到的就是opencv和maplotlib，另外可以使用seaborn讓matplotlib的作圖更美觀，以及wandb可以紀錄訓練數據。
  2. 如果你的電腦沒有好的獨顯，那在Google colab上訓練會是一個好選擇，但建議升級Google colab pro，最主要是不會斷線以及不會有每週限制的GPU使用時間，再來就是可以用比較好的顯卡，也可以使用大量RAM模式，而學長目前是人社中心的伺服器和Google colab都有使用。
  3. 學長目前的研究主題是土地利用分類 (semantic segmentation) 再加上 style transfer (所以有使用到image-to-image translation相關的模型)，我覺得剛開始一定是最花時間以及最沒有頭緒的，但慢慢閱讀論文並把訓練和測試的流程都熟悉後，我相信會順暢很多，也會從中找到很多成就感。

2022/9/1

Learning Representations for Automatic Colorization
https://arxiv.org/abs/1603.06668v3

Point-interactive image colorization
https://paperswithcode.com/task/point-interactive-image-colorization

2022/9/2

Mask R-CNN (Fast R-CNN的一個分支)
https://arxiv.org/abs/1703.06870

2022/9/7

https://github.com/Time0o/colorful-colorization
https://ithelp.ithome.com.tw/articles/10262541

2022/9/11

Python argparse模組的介紹與教學
https://haosquare.com/python-argparse/

Python 實用標準函式庫 argparse
https://dboyliao.medium.com/python-超好用標準函式庫-argparse-4eab2e9dcc69

2022/9/15

向中研院申請到GPU，學會Wireguard VPN、PuTTy、remote-ssh等軟體的使用。

相關連結:
Putty的基礎教學
https://its.sinica.edu.tw/posts/117169
VScode遠端連上ssh
https://code.visualstudio.com/docs/remote/ssh
Wireguard安裝
https://www.wireguard.com/
Putty安裝
https://putty.org/

GPU instructions

IP: 140.109.161.10

User: laiyuchi

Passwd: 2021laiyuchi@

Wireguard VPN設定檔: https://drive.google.com/file/d/1adjw2Oer5KO6NhjDdnomzg517hNV0QCy/view?usp=sharing

2022/9/16

CycleGAN實作

https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

https://github.com/junyanz/CycleGAN

CycleGAN inage colorization
https://github.com/ArkaJU/Image-Colorization-CycleGAN

Style GAN
https://github.com/NVlabs/stylegan

也許可以用一些style transfer的模型，去進行上色。

2022/9/20

https://arxiv.org/abs/2011.11731

Single Image Colorization Via Modified Cyclegan
https://ieeexplore.ieee.org/document/8803677

2022/9/22

https://sci-hub.se/
https://sci-hub.se/10.1109/icip.2019.8803677

https://blog.csdn.net/lgzlgz3102/article/details/108970997

2022/9/23

Anaconda環境安裝 (Linux)
https://ithelp.ithome.com.tw/articles/10237621
Conda 基本指令 (Linux)
https://ithelp.ithome.com.tw/articles/10237623

科展-利用深度學習將黑白影片色彩化
https://twsf.ntsec.gov.tw/activity/race-1/61/pdf/NPHSF2021-052508.pdf?0.2732005977974896
https://twsf.ntsec.gov.tw/activity/race-2/2022/pdf/TISF2022-190012.pdf

2022/9/29

https://github.com/zeruniverse/neural-colorization

測試: https://github.com/phillipi/pix2pix

確定之後要採用的模型

Pix2pix
paper:
https://arxiv.org/abs/1611.07004
code:
https://github.com/phillipi/pix2pix https://phillipi.github.io/pix2pix/

GAN
paper:
https://arxiv.org/abs/1803.05400
code:
https://github.com/ImagingLab/Colorizing-with-GANs
https://github.com/srihari-humbarwadi/image_colorization_gan_tf2.0

Pytorch CycleGAN and pix2pix
paper:
https://arxiv.org/abs/1703.10593
https://arxiv.org/pdf/1703.10593.pdf

code
https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/master/models/colorization_model.py (Colorization model)

Pytorch GAN & pix2pix implemention
https://github.com/eriklindernoren/PyTorch-GAN
https://www.kaggle.com/code/orkatz2/pytorch-pix-2-pix-for-image-colorization/notebook
https://www.tensorflow.org/tutorials/generative/pix2pix
https://www.tensorflow.org/tutorials/generative/style_transfer
https://github.com/SachaIZADI/colorful-world-pix2pix

tensorflow pix2pix
https://github.com/affinelayer/pix2pix-tensorflow

2022/10/20

https://affinelayer.com/pix2pix/
pix2pix 論文的小解說
https://medium.com/@falconives/day-56-pix2pix-21c7df48922c

pix2pix
https://github.com/topics/pix2pix
https://github.com/phillipi/pix2pix
https://towardsdatascience.com/colorizing-black-white-images-with-u-net-and-conditional-gan-a-tutorial-81b2df111cd8

Stable Diffusion
https://github.com/CompVis/stable-diffusion
https://ommer-lab.com/research/latent-diffusion-models/

https://arxiv.org/abs/1609.04802

2022/10/21










# move.py，置於多資料夾的資料夾中，可將檔案取至 data 資料夾
import os
import shutil
for folder_name in os.listdir("."):
    for file_name in os.listdir(f'.\\{folder_name}'):
        source = ".\\"+folder_name+"\\"+file_name
        destination = ".\\data\\"+folder_name+"_"+file_name
        shutil.copy(source,destination)
        print("copied",folder_name+"_"+file_name)

如何使用 python 處理 yaml 檔案
https://medium.com/bryanyang0528/使用-python-讀取-yaml-檔案-d3f413d7dd6

Google colab 掛接雲端硬碟
https://www.wongwonggoods.com/python/python-colab-mount-google-drive/

2022/10/25

Epoch 4/100 Iteration 400/500 loss_D_fake: 0.52868 loss_D_real: 0.58197 loss_D: 0.55533 loss_G_GAN: 1.24667 loss_G_L1: 10.98473 loss_G: 12.23140

ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data
https://arxiv.org/abs/1904.00592
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial
Network
https://arxiv.org/pdf/1609.04802.pdf

2022/10/30

跑過 pix2pix 的模型
https://colab.research.google.com/drive/1cjMy-THaPf0-AYtFJXctB-IBdhVBAfHQ?usp=

小技巧 (下載 colab 中資料夾的方式)
https://stackoverflow.com/questions/50453428/how-do-i-download-multiple-files-or-an-entire-folder-from-google-colab

2022/11/2

https://www.tensorflow.org/tutorials/generative/style_transfer

2022/11/3

用 CGAN&U-net 跟自己的 datasets 訓練
https://colab.research.google.com/drive/1KzGX8X2_bf1M-z1pu0g0OXCkmcCWiQLY?usp=sharing

pix2pix training

tensorboard tutorial
https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html

用 matplotlib.pyplot 繪製折線圖
https://ithelp.ithome.com.tw/articles/10246384
多條折線
https://blog.csdn.net/daybreak___/article/details/107752519

範例code 

























def print_loss():
    x = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]
    plt.title('Loss record')
    plt.xlabel('Epoch')
    plt.ylabel('Loss')
    plt.plot(x,loss_D_Fake,marker='o',marker_size=3)
    plt.plot(x,loss_D_Real,marker='o',marker_size=3)
    plt.plot(x,loss_d,marker='o',marker_size=3)
    plt.plot(x,loss_G_gan,marker='o',marker_size=3)
    plt.plot(x,loss_G_l1,marker='o',marker_size=3)
    plt.plot(x,loss_g,marker='o',marker_size=3)
    for a,b in zip(x,loss_D_Fake):
        plt.text(a,b,b,ha='center',va='bottom',fontsize=10)
    for a,b in zip(x,loss_D_Real):
        plt.text(a,b,b,ha='center',va='bottom',fontsize=10)
    for a,b in zip(x,loss_d):
        plt.text(a,b,b,ha='center',va='bottom',fontsize=10)
    for a,b in zip(x,loss_G_gan):
        plt.text(a,b,b,ha='center',va='bottom',fontsize=10)
    for a,b in zip(x,loss_G_l1):
        plt.text(a,b,b,ha='center',va='bottom',fontsize=10)
    for a,b in zip(x,loss_g):
        plt.text(a,b,b,ha='center',va='bottom',fontsize=10)
    plt.legend('loss_d_fake','loss_d_real','loss_d','loss_g_gan','loss_g_l1','loss_g')
    plt.show()

2022/11/8

暫時把 11 月的研究札記轉到以下連結:
https://hackmd.io/@expectlai/informatic-project-November

Last changed by 
 
Expect Lai
INFOR35th Network Management
1
1436

Read more

Data structure

Yu-Chi Lai

Jul 8, 2024
資專研究札記By賴昱錡 (Nov.&Dec.)

2022/11/8 Comparison of deep learning methods for colorizing historical aerial imagery https://meetingorganizer.copernicus.org/EGU22/EGU22-7686.html ColorAI -Automatic Image Colorization using CycleGAN https://www.researchgate.net/publication/350957456_ColorAI_-Automatic_Image_Colorization_using_CycleGAN CycleGAN model: https://github.com/junyanz/CycleGAN https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

Dec 29, 2022
建中資訊 Basic Linux

Basic Linux Yu-Chi Lai Intro Linux 介紹 由 Linus Torvalds 於 1991 年首次發布 自由且開放的類 Unix 系統 高安全性、高穩定性、高移植性 常用於伺服器 特色

Dec 26, 2022
Dark Theme

A dark theme for HackMD

Oct 15, 2022
Read more from Expect Lai
Published on HackMD