# DL Surveyception: a survey of overviews, reviews and surveys in deep learning for computer vision, time-series and others
[Summary of slides](https://imgur.com/a/02pZTKh)
## Introduction
In order to catch up with the progress of the field, I did a meta-survey, a survey of surveys, in different areas of deep learning. It allows for understanding the main challenges and trends in particular areas.
## Methodology
I found the following papers by using the following keywords on Google, Google Scholar, Microsoft Academic, SemanticScholar, and others were found through connections on [Connected Papers](https://www.connectedpapers.com/) or other websites. I divide them into papers before 2020, and since 2020. The former includes 14 papers, while the latter includes 15, for a total of 29 papers.
For all the papers I list the title, the date it was last updated (for most Arxiv papers), the number of citations (estimate of how impactful/useful it's been for others), the number of pages (how much content does it pack), and the first author (just to get familiar with some big names in the field). The classification in this section is relatively arbitrary, since the original purpose of this was to do a survey on GANs but then I expanded it to other topics. However, just for reference, the idea is that the blue DL/CV represented more general-purpose surveys, the green AS meant application-specific where it focused on a particular area or application, and the purple GAN represented general papers related to GANs.
<figure>
<img src="https://i.imgur.com/vrlMyaE.png" alt="Surveys Before 2020">
<figcaption style="align: left; text-align:center;">Surveys before 2020.</figcaption>
</figure>
<figure>
<img src="
https://i.imgur.com/e84uHyg.png" alt="Surveys Since 2020">
<figcaption style="align: left; text-align:center;">Surveys since 2020.</figcaption>
</figure>
## Surveyception
I'll try to group up similar reviews and list them here, along with one or two reviews that I found particularly informative, or helpful, for any reason. I'll link the topic in the tables to the slide in the album.
Note: these slides are definitely not for everyone; they're extremely information dense.
Disclaimer: the content of these slides correspond to the respective authors.
### Time-series
| Topic | Year / Month | First Author |
| ------- | ---------- | ------ |
| [Physiological Signals](https://i.imgur.com/0f4JcXA.png) | 2018 / 05 | Faust |
| [EEG Analysis](https://i.imgur.com/Zq6TzcO.png) | 2019 / 01 | Roy |
| [Classification](https://i.imgur.com/o9EFF9m.png) | 2019 / 05 | Fawaz|
### Computer vision
| Topic | Year / Month | First Author |
| ------- | ---------- | ------ |
| [CV using DL in General](https://i.imgur.com/hUELhrd.png) | 2017 / 06 | Voulodimos|
| [CV using DL in General](https://i.imgur.com/TWJt0sO.png) | 2020 / 05 | Khan A. |
### GANs
| Topic | Year / Month | First Author |
| ------- | ---------- | ------ |
| [GANs in General](https://i.imgur.com/xnQMPnz.png) | 2017 / 09 | Creswell|
| [GANs in General](https://i.imgur.com/N5CyJkD.png) | 2018 / 01 | Wang K. |
| [GANs in General](https://i.imgur.com/OwN5VSj.png) | 2019 / 03 | Pan |
| [GANs in General](https://i.imgur.com/QzCdCUJ.png) | 2020 / 06 | Jabbar |
| [GANs in General](https://i.imgur.com/40fMF6i.png) | 2020 / 06 | Gui |
| [GANs in General: Pt. 1](https://i.imgur.com/TZrCqew.png) / [Pt. 2](https://i.imgur.com/k35zPak.png) / [Pt. 3](https://i.imgur.com/UpbG3aq.png) | 2020 / 09 | Saxena |
| [GANs in General](https://i.imgur.com/mHXkA0p.png) | 2020 / 12 | Wang Z. |
| [GANs for Medical Imaging](https://i.imgur.com/ZdggTTP.png) | 2019 / 09 | Yi |
| [GANs Stabilizing](https://i.imgur.com/QI19oys.png) | 2020 / 03 | Wistrak |
| [GANs Loss Functions](https://i.imgur.com/X3rhpWp.png) | 2020 / 05 | Pan |
| [GANs for DeepFakes](https://i.imgur.com/EHQqoUg.png) | 2020 / 06 | Tolosana |
| [GANs for Video](https://i.imgur.com/ntaO9j8.png) | 2020 / 11 | Aldausari |
| [GANs for Image Synthesis: Pt. 1](https://i.imgur.com/OiDxN8l.png) / [Pt. 2](https://i.imgur.com/duLqFJ9.png) | 2020 / 12 | Shamsolmoali |
### Transformers
| Topic | Year / Month | First Author |
| ------- | ---------- | ------ |
| [Efficient Transformers: Pt. 1](https://i.imgur.com/bTshOlz.png) / [Pt. 2](https://i.imgur.com/dlVPD6P.png) / [Pt. 3](https://i.imgur.com/CQVnmTE.png) | 2020 / 09 | Tay|
| [Transformers for Vision: Pt. 1](https://i.imgur.com/r67KWwS.png) / [Pt. 2](https://i.imgur.com/YgYsa1Q.png) / [Pt. 3](https://i.imgur.com/3abuhoC.png) | 2021 / 01 | Khan S. |
| [Transformers for Vision](https://i.imgur.com/BuZhSlz.png) | 2021 / 01 | Han |
### Representation, transfer and not-fully supervised learning
| Topic | Year / Month | First Author |
| ------- | ---------- | ------ |
| [Not Fully-Supervised Learning for Medical Image](https://i.imgur.com/c00cwJK.png) | 2017 / 04 | Cheplygina |
| [Self-Supervised Learning](https://i.imgur.com/cjSOLcZ.png) | 2019 / 02 | Jing |
| [Transfer Learning](https://i.imgur.com/uIitJzJ.png) | 2019 / 05 | Zhang |
| [Transfer Learning](https://i.imgur.com/M1HrFL6.png) | 2019 / 11 | Zhuang |
| [Self-Supervised Learning: Pt. 1](https://i.imgur.com/ymzHSd9.png) / [Pt. 2](https://i.imgur.com/XI1YT5B.png) | 2020 / 07 | Liu |
### Others:
| Topic | Year / Month | First Author |
| ------- | ---------- | ------ |
| [DL for Healthcare in General](https://i.imgur.com/8lOL4nV.png) | 2017 / 05 | Miotto|
| [DL for Edge Computing](https://i.imgur.com/t6PqFsF.png) | 2019 / 08 | Chen |
| [DL for Audio-Visual](https://i.imgur.com/ClRRADj.png) | 2020 / 06 | Zhu |
## Open-source repo020, Accessed: Jan. 28, 2021. [Online]. Available: http://arxiv.org/abs/2001.04758.
[4]M. Wiatrak, S. V. Albrecht, and A. Nystrom, “Stabilizing Generative Adversarial Networks: A Survey,” arXiv:1910.00927 [cs], Mar. 2020, Accessed: Nov. 13, 2020. [Online]. Available: http://arxiv.org/abs/1910.00927.
[5]Z. Wang, Q. She, and T. E. Ward, “Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy,” arXiv:1906.01529 [cs], Jun. 2020, Accessed: Nov. 13, 2020. [Online]. Available: http://arxiv.org/abs/1906.01529.
[6]R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega-Garcia, “DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection,” arXiv:2001.00179 [cs], Jun. 2020, Accessed: Nov. 13, 2020. [Online]. Available: http://arxiv.org/abs/2001.00179.
[7]Y. Tay, M. Dehghani, D. Bahri, and D. Metzler, “Efficient Transformers: A Survey,” arXiv:2009.06732 [cs], Sep. 2020, Accessed: Jan. 22, 2021. [Online]. Available: http://arxiv.org/abs/2009.06732.
[8]P. Shamsolmoali et al., “Image Synthesis with Adversarial Networks: a Comprehensive Survey and Case Studies,” arXiv:2012.13736 [cs, eess], Dec. 2020, Accessed: Jan. 28, 2021. [Online]. Available: http://arxiv.org/abs/2012.13736.
[9]D. Saxena and J. Cao, “Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions,” arXiv:2005.00065 [cs, eess, stat], Oct. 2020, Accessed: Nov. 13, 2020. [Online]. Available: http://arxiv.org/abs/2005.00065.
[10]Z. Pan et al., “Loss Functions of Generative Adversarial Networks (GANs): Opportunities and Challenges,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 4, no. 4, pp. 500–522, Aug. 2020, doi: 10.1109/TETCI.2020.2991774.
[11]X. Liu et al., “Self-supervised Learning: Generative or Contrastive,” arXiv:2006.08218 [cs, stat], Jul. 2020, Accessed: Dec. 16, 2020. [Online]. Available: http://arxiv.org/abs/2006.08218.
[12]B. Lim and S. Zohren, “Time Series Forecasting With Deep Learning: A Survey,” arXiv:2004.13408 [cs, stat], Sep. 2020, Accessed: Dec. 16, 2020. [Online]. Available: http://arxiv.org/abs/2004.13408.
[13]A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, “A Survey of the Recent Architectures of Deep Convolutional Neural Networks,” Artif Intell Rev, vol. 53, no. 8, pp. 5455–5516, Dec. 2020, doi: 10.1007/s10462-020-09825-6.
[14]A. Jabbar, X. Li, and B. Omar, “A Survey on Generative Adversarial Networks: Variants, Applications, and Training,” arXiv:2006.05132 [cs, eess], Jun. 2020, Accessed: Nov. 13, 2020. [Online]. Available: http://arxiv.org/abs/2006.05132.
[15]S. Hooker, “The Hardware Lottery,” arXiv:2009.06489 [cs], Sep. 2020, Accessed: Jan. 28, 2021. [Online]. Available: http://arxiv.org/abs/2009.06489.
[16]J. Gui, Z. Sun, Y. Wen, D. Tao, and J. Ye, “A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications,” arXiv:2001.06937 [cs, stat], Jan. 2020, Accessed: Nov. 13, 2020. [Online]. Available: http://arxiv.org/abs/2001.06937.
[17]N. Aldausari, A. Sowmya, N. Marcus, and G. Mohammadi, “Video Generative Adversarial Networks: A Review,” arXiv:2011.02250 [cs, eess], Nov. 2020, Accessed: Jan. 28, 2021. [Online]. Available: http://arxiv.org/abs/2011.02250.
[18]F. Zhuang et al., “A Comprehensive Survey on Transfer Learning,” Nov. 2019, Accessed: Jan. 13, 2021. [Online]. Available: https://arxiv.org/abs/1911.02685v3.
[19]J. Zhang, W. Li, P. Ogunbona, and D. Xu, “Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition: A Problem-Oriented Perspective,” arXiv:1705.04396 [cs], May 2019, Accessed: Jan. 24, 2021. [Online]. Available: http://arxiv.org/abs/1705.04396.
[20]X. Yi, E. Walia, and P. Babyn, “Generative Adversarial Network in Medical Imaging: A Review,” Medical Image Analysis, vol. 58, p. 101552, Dec. 2019, doi: 10.1016/j.media.2019.101552.
[21]Y. Roy, H. Banville, I. Albuquerque, A. Gramfort, T. H. Falk, and J. Faubert, “Deep learning-based electroencephalography analysis: a systematic review,” arXiv:1901.05498 [cs, eess, stat], Jan. 2019, Accessed: Nov. 13, 2020. [Online]. Available: http://arxiv.org/abs/1901.05498.
[22]Z. Pan, W. Yu, X. Yi, A. Khan, F. Yuan, and Y. Zheng, “Recent Progress on Generative Adversarial Networks (GANs): A Survey,” IEEE Access, vol. 7, pp. 36322–36333, 2019, doi: 10.1109/ACCESS.2019.2905015.
[23]L. Jing and Y. Tian, “Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey,” arXiv:1902.06162 [cs], Feb. 2019, Accessed: Dec. 16, 2020. [Online]. Available: http://arxiv.org/abs/1902.06162.
[24]H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P.-A. Muller, “Deep learning for time series classification: a review,” Data Min Knowl Disc, vol. 33, no. 4, pp. 917–963, Jul. 2019, doi: 10.1007/s10618-019-00619-1.
[25]J. Chen and X. Ran, “Deep Learning With Edge Computing: A Review,” Proceedings of the IEEE, vol. 107, no. 8, pp. 1655–1674, Aug. 2019, doi: 10.1109/JPROC.2019.2921977.
[26]R. Miotto, F. Wang, S. Wang, X. Jiang, and J. T. Dudley, “Deep learning for healthcare: review, opportunities and challenges,” Brief Bioinform, vol. 19, no. 6, pp. 1236–1246, Nov. 2018, doi: 10.1093/bib/bbx044.
[27]O. Faust, Y. Hagiwara, T. J. Hong, O. S. Lin, and U. R. Acharya, “Deep learning for healthcare applications based on physiological signals: A review,” Computer Methods and Programs in Biomedicine, vol. 161, pp. 1–13, Jul. 2018, doi: 10.1016/j.cmpb.2018.04.005.
[28]A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A. Bharath, “Generative Adversarial Networks: An Overview,” IEEE Signal Processing Magazine, vol. 35, no. 1, pp. 53–65, Jan. 2018, doi: 10.1109/MSP.2017.2765202.
[29]V. Cheplygina, M. de Bruijne, and J. P. W. Pluim, “Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis,” Apr. 2018, Accessed: Jan. 13, 2021. [Online]. Available: https://arxiv.org/abs/1804.06353v2.
[30]V. A, D. N, D. A, and P. E, “Deep Learning for Computer Vision: A Brief Review.,” Comput Intell Neurosci, vol. 2018, pp. 7068349–7068349, Feb. 2018, doi: 10.1155/2018/7068349.
[31]K. Wang, C. Gou, Y. Duan, Y. Lin, X. Zheng, and F. Wang, “Generative adversarial networks: introduction and outlook,” IEEE/CAA Journal of Automatica Sinica, vol. 4, no. 4, pp. 588–598, 2017, doi: 10.1109/JAS.2017.7510583.
[32]Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, May 2015, doi: 10.1038/nature14539.
[33]M. Längkvist, L. Karlsson, and A. Loutfi, “A Review of Unsupervised Feature Learning and Deep Learning for Time-Series Modeling,” Pattern Recognition Letters, vol. 42, Jun. 2014, doi: 10.1016/j.patrec.2014.01.008.
[34]S. J. Pan and Q. Yang, “A Survey on Transfer Learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345–1359, Oct. 2010, doi: 10.1109/TKDE.2009.191.
</details>
Sign in with Wallet
Connect another wallet