- NLM-uncertainty-parametrization·Last edited by Maximillian Guo on Jan 30, 2022Linked with GitHubContributed by
Read more
Unfiltered Research Notes
Important Links Papers Drive Resource Links: http://cs229.stanford.edu/section/cs229-gaussian_processes.pdf https://gregorygundersen.com/blog/2019/12/23/random-fourier-features/#a1-gaussian-kernel-derivation 5/9
May 9, 2022Paper Outline
general Question we are trying to answer covariance structure size of data with respect to width MAP training (determinant) claim that only a small number of bases fit the data? Motivation Current literature on NLM
May 2, 2022Thoughts from Friday 3/11
Overarching research questions What is a good basis? BUT - this is dependent on where the data is EX1. consider any basis that is infinitely differentiable in a region, then if we consider points close enough, we'll obtain piecewise linear and will not obtain good uncertainties nor fit (we think) if this is true, this tells us that we need the in-distribution and out-of-distribution areas of the data in order to meaningfully ask this question This leads us to...
Mar 21, 2022Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited
Main Ideas: Mysterious why NN’s can generalize well with so many more parameters than data points Effective dimensionality (ED) measures the dimensionality of the parameter space determined by the data Relates ED to posterior contraction in Bayesian deep learning, model selection, width-depth tradeoffs, double descent, and functional diversity in loss surfaces. ED compares favorably to alternative norm- and flatness-based generalization measures. Effective Dimensionality (ED)
Feb 11, 2022
HackMD