# Paper 01 High-Resolution Image Synthesis with Latent Diffusion Models

## Summary

The current study in a proposed improvment to diffucion models, AImodels which are used for image synthesis and modification.
The authors introduce a new Diffusion model, dubbed Latent Diffusion Model, that aims to "reach a near-optimal point between complexity reduction and detail preservation". Which they believe they have succeeded in.
The main difference of this 'new' model with the 'old' diffusion models is the transformation of the dataset to the latent space. The authors reduced the dimensionality of the data and combined similar data that would have minimal contribution to the resulting image, thus greatly reducing computation resources.

The authors argue for their methods and their results, by experiments and comparisons with related models in the fields. They acknowledge their limitations compared to GANs in the sampling progress and potential difficulty with high precision images. However, overall they present a new and greatly improved Diffusion Models, especially for more consumer rated GPU/Hardware

## Good Points

The sheer amount of results and examples presented by the authors is really comendable. It makes it easier for the reader to follow along.

It is admirable that they chose to lisence the eniter source code under the MIT lisence. It helps when you want to reproduce and verify their results.

Since the LDM requires a lot less hardware to setup up-and-coming reaserches will have a lot easier time with budget hardware to start their own research into Computer Vision/High-Resolution Imaging. Also goes nicely along with the MIT lisence.

## Weak Points

The authors presumably for the sake of scientific appearence, space and time limitation have resolved to overly complicated language and limited explanations of utilized and mentioned algorithms. This may be a plus for the specialized profesional comunity but makes unprochable to the uninitiated.

They mention GANs and other algorithms without presenting their full name or elaborating/referencing them. In General, the paper has some minor structual incositancies, which can often end up confusing the reader, as some examples are the fact that they use the **DM** acronym for Diffusion Models and Diffusion Probablistic Model, and this can confuse the reader. There is also places where an acronym is used before it is defined.

## Questions

We are left with a lot of question, most of them related to our limited knowledge with the Diffusion Model, and generaly the Deep Learning part of Computer Vision. There is a lot more work for us to get into and understand what is going on.

This paper also bases itself on a lot of previous work that we are not familiar with. For them it already known, and they care little to explain it for the uninisiated. However, that is often how fresh papers in state-of-the-art fields are composed.

## Ideas

Since it can be tiresome for the experienced community to be reintoduced in to their own fields of study, it would be helpfull if the authors composed a short seeprate paragraph or footnote, with further elaboration or easily accesible references on previous needed knowledge, in order to ease the introduction of newly exposed readers to the ideas.

## Other comments

I was able to keep up with the Abstact, Introduction and most of the related work. However, when they went into 3. Methods, I was near completly lost
- Tomas

Bold claim:
- "In contrast to previous work, training diffusion models on such a representation allows for the first time to reach a **near-optimal** point between complexity reduction and detail preservation, greatly boosting visual fidelity."

I was able undersand the abstract, but there were a lot of new ideas and algorithms i had never heard of before in the introduction, which was really dificult and time consuming.
- Dimitris

## Definitions

- [Diffusion Models](https://lilianweng.github.io/posts/2021-07-11-diffusion-models/)

- [UNet](https://towardsdatascience.com/understanding-semantic-segmentation-with-unet-6be4f42d4b47)

- [GAN](https://www.techtarget.com/searchenterpriseai/definition/generative-adversarial-network-GAN) (Generative Adversarial Network)

# Paper 02 Contrastive Multiview Coding

## Summary

The paper seeks out to investigate and study the hypothesis that "a powerful representation is one that models view-invariant factors". To do this they use the technologies of contrastive and predictive learning and their evolvement to Multiview learning. A technique which compares 2 views of the same image and tries to extrapolate useful information. It does so by loosely defining some "good" and "bad" bits, the "good" ones being the ones that are shared between multiple _views_, and the different views being different image channels for this paper, however the definition of a view is very flexible. That way the training of the models based only on the "good" bits provides accurate and realistic results in a fraction of the time it would take to reproduce them with the raw data.

The authors go one step ahead and compare multiple views instead of just 2, stating that the increased number of views yields better results.
Later they back their statement with experiments and they prove that by increasing the views from 2 to 4 the results are far superior and are comparable with state of the art technologies.
For the multi-viewing prosses they present 2 options. One is to take one base view and try to compare all other views with it, which while it's fast it doesn't provide any information about the missing views, only for the main one. Or compare all the possible pairs which provides more information but its computationally heavy.

## Good Points

This paper does a much better work of explaining both the previous works, the acronyms and related work, that does makes it much easier to follow along. When they introduce a previous or concept they write something short and concise about it, so even people who haven't read the previous paper has some idea of what it was about.

## Weak Points

At least compared to the previous paper we summarized [[1]](https://arxiv.org/abs/2112.10752), this current paper uses a lot less visualization to show their results. This in turn can make it a bit harder to know exactly what they are referring to

## Questions

Is there an optimal number of views after which the quality improvement of the results isn't worth the computational time

## Ideas

Add a bit more visual representations, and "real life" examples to further elaborate their point. Maybe a step by step image succession to show the reader the difference in the quality instead of percentages.

## Other Comments

This time I had an easier time following what was going on. When I started reading the methods part I for the most part did not understand all the functions and mathematical methods.

The article started smother that the previous one, and it provided more "examples", which made it easier and more pleasant to follow.

## Definitions

- [Instance Discrimination](https://sh-tsang.medium.com/review-unsupervised-feature-learning-via-non-parametric-instance-discrimination-aa2fda45dcba)
- [Contrastive Predictive Coding](https://paperswithcode.com/method/contrastive-predictive-coding)
- [Contrastive learning](https://www.v7labs.com/blog/contrastive-learning-guide)
- [mutual information maximization](https://stats.stackexchange.com/questions/480918/what-does-maximizing-mutual-information-do)
- [deep representation learning](https://www.glassroom.com/deep-representation-learning)
- [multi-view learning](https://www.sciencedirect.com/science/article/abs/pii/S1566253516302032)

# Paper 03 Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning

## Summary

This paper takes us through the essential step-by-step approach the how to train a model from a general point-of-view, such as common methods for model evaluation and when to use them (such as the holdout method), how to divide a dataset and measure the confidence of the model. It also mentions advantages and disadvantages with certain methods.

It does not aim to be the end-all be-all of "Model Evaluation, Model Selection, and Algorithm Selection", but a starting of point, to get the reader to consider potential problems related to each decision you make.

## Performance Estimation

The performance estimation of machine learning algorithms is divided into three distinct points. Firstly how efficiently the model generalized unseen(future) data. In addition, how well would a model preform with a given hypothesis space and how the algorithm could be tweaked to find the best model for said hypothesis space. Lastly, selecting the optimal algorithm for the given problem and the hypothesis space. A common occurrence when we are trying to estimate the performance is the addition of bias, which will depend on the circumstances and the training set. Although, it may sound counter intuitive the existence of a bias is perfectly acceptable as long as the same bias is persistent throughout the tests. It's worth keeping in mind that unbiased circumstanced and in extension an absolute performance estimation is a very challenging task in the field.

## Holdout Validation

Holdout method is one of the most common practices in machine learning. It's the practice where( in the absence of the testing dataset) the training dataset is divided into two parts. A larger part for training and a smaller for testing, the easiest way is to split the dataset is by taking random values but that can lead to two problems. Firstly, the violation of independence since using a random part of the training set as testing removes vital data from the training and changes proportions of the variables(testing on the data used for training is called Resubstitution Validation and leads to over-fitting. The second problem is the capacity, not using all the available data can lead to pessimistically biased predictions. But, there are ways to counter those problems. The proper procedure for holdout method has four steps:

1. Divide the dataset into two groups using stratification to counter the first problem. Most common approach is 2/3,1/3 but that depends on the dataset and its size.
1. Manually specify the hyper parameter values. There are ways to find the optimal but this method is not well suited for it, so fixed parameters can be also used.
1. Run the algorithm, given that the training dataset is new for the algorithm the results should be unbiased.
1. Add the last part of the dataset to the training and retrain the algorithm, more data can solve the second problem and in general it is proven that they can only improve performance.

## Confidence Intervals via Normal Approximation

If we assume that the prediction follows a normal distrebution, and this means the variance is independent of the mean, this is considered as confidence intervals via Normal approximation method.(A typical value for confidence intervals is 95%)

However the article recommends to instead split the traning-test multiple times, and get the confdence inverval that way.

## Stratification

Stratification is the idea of sorting/grouping object into a distinct group. While it is common to randomly take a portion of the training data as a test dataset, this can sometimes cause problems when considering what the underlying data represent. In the article example it talks about the [iris](https://archive.ics.uci.edu/ml/datasets/iris) dataset, that har 3 different types of flower, and if you randomly take a portion of the dataset for testing, you won't necessarily be able to represent the whole dataset as the distribution of each flower will be different in the training and test set, if you randomly pick. So it might be smart to `stratify` it by grouping the data that belongs together and ensure that each of these subset get the same distribution across.

# Paper 04

## Summary
The paper introduces *Florence* a state-of-the-art vision foundation model. Foundation models are models that are trained at large scale data and can be fine tuned to perform a wide variety of tasks.
The authors boast about it's capabilities to expand representations from scenes to objects and images to video widening the scope of the model compared to previous technologies.
Florence is a pre trained model and achieves extraordinary results compared to many representative benchmarks using many types of transfer learning, (learning from of previously acquired knowledge and skills and solving new tasks) and achieved state of the art performance.
Training of Florence took place with the use of large scale publicly available data. It was pre trained with the use of image and text pairs in 2 stages depending on the text size of each pair(biggest text were easier to train so some of them had to be artificially created but those could hurt performance so they were removed in the seecond stage). In addition, 2 transformers (language and image) were used to embed the item pairs and calculate the distance(two towers model) and it was made adaptable to a variety of tasks with the use of other models.
Due to the scale of the training data several techniques were used to reduce memory cost, allowing more data to be processed and speed up the training progress at the same time.

## Good Points

The paper utilized their graphs, tables and figues well. This means as a reader it is much easier to follow along when the authers explains concepts, methods and experiments.

The language (except the techical terms) is quite easy to follow and undertsand and give sthe paper a good flow.

While short, the conclution nicley conclude the paper with both their journey and potential future work.

## Weak Points

Similar to the first paper ([High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752)) There are a lot of refrerances that is assumed to already have been read and familiarized yourself with. So when someone who is not super familiar with the deep-end of the field comes a long to read, it can be hard to understand and follow.

## Questions

## Ideas

## Other Comments
This paper had a nice flow of thought but the ammount of unknown terms was high.

## Definitions
- Natural Language Processing: [NLP](https://en.wikipedia.org/wiki/Natural_language_processing)