# Probability Scribes Based on lectures 29,30,31,32,33 --- ## Random Processes We are familiar with analysing single and multiple random variables through both probabilistic methods and statistical averages. We have also observed the behaviour of infinite iid random variables using Law of Large numbers and Central Limit Theorem. Let's say we wish to observe the temperature of the city everyday for a year. But before measuring the next day's temperature we want to predict the next day's temperature. So we model the temperature as a random variable and come up with a probability distribution based on what season is it, has it been raining, weather forecasts on TV etc. Now since the distribution is known we can find the expected value and variance just like for any other random variable. But we do this for a year, and each day our probability distribution is different from the other days, implying our random variables are different each day. This collection of random variables is called a random process. Since a random process is just a collection of random variables indexed by time. The random variables may or may not be identical, they may or may not be independent. They can be either continuous time or discrete time. The example I took is discrete time. Discrete time random processes are a countable set of random variables whereas continuous time random processes are an uncountable set of random variables. Lets say I plot the temperature against the day number after 1 year, this is just 1 possible outcome of the random process. Its a function of time , hence also called random function or realisation. The random process is represented as X and the random variable at time t is represented as X(t). Each random variable in a random process is also called a sample. And we can define the joint pdf or cdf of the various samples of the random process, just like we do for a general collection of random variables. The mean of a random process is a function of time as it represents the mean of the random variable at time t. Other statistical averages include autocorrelation and autocovariance which give us information about the relation or dependence between any 2 samples. Formula for mean, auto correlation and auto variance The basic idea between the autocovariance is it measures the relative movement between 2 random variables. If the autocovariance is positive then it means if one random variable takes a high value then the other will also take large values with a high probability and opposite argument for negative autocovariance. If the autocovariance is 0 then irrespective of whether the random variable takes high or low values we cant say anything about the other random variable. Now we want to anylse the relation between multiple random processes. A real life example is consider the product price of 2 companies, one is a oil processing company and the other is a paint company. The paint company requires oil products to manufacture its paints.We can model the price of each company's product as a random process and observe the correlation between the prices of both the company's product. Now at one point of time the oil company increases it prices which leads in the increase of the paint price as the expenditure of the paint company has increased. We can observe that the correlation between the random processes is positive(if the oil company reduces its price then the paint company also reduces it price). This correlation function is not only a mathematical concept but can also provide us information about the company's decisions. For example if at one point even if the oil company's price is increasing but the paint company's price stays stagnant or even decreases(mathematically the correlation between the prices would either become 0 or negative) then we can guess that the paint company has struck a deal with some other oil company to fulfil its requirements and we can find out which oil company by finding the autocorrelation functions between all possible oil companies and this paint company. Whichever oil company shows the highest correlation with our paint comapny is most likely the supplier to our paint company. When modelling real life situations just 2 random processes are not enough, as the price of the oil company may depend on some other random process altogether. Such types of correlations between 2 random processes is called cross correlation. We can also intuitively define autocorrelation using this example. The price of the product at time t depends on the price at time t-1, t-2,..(past values). To quantify how much it depends on the past values we use the autocorrelation function. Stationary Process --- A special class of random processes are the stationary process in which the statistical properties of the process dont change with time. Now again based on which statistical property we are talking about we can classify stationary processes as strict sense or weak sense. In strict sense stationary process the statistical property which doesnt change is the joint cdf of the samples whereas in the weak sense stationary process the mean function and the autocorrelation between 2 samples separated by some constant doesnt change with time. There are hardly any real life examples which can modeled by strict sense stationary processes. Most processes in real life are either weak sense stationary or non stationary Properties of autocorrelation function of a stationary process 1) its an even function 2) rx(t,t) is non negative and represents the average energy of the random process at time t, but for a WSS process rx(t,t)=rx(0) which is a constant. I pick an arbitrary realisation from the random process and try to find how fluctuating it is. We know that this can be done by finding its frequency spectrum using the Fourier transform. So if I analyse the fourier transform of every such realisations and find the amount of power that each frequency has in it and then average it out to find if my random process is highly fluctuating or is it stable on average. But since the average and Fourier transform are linear functions we can interchange them and first do average of the power of all the realisations and then take its Fourier Transform. We know that the average power of the WSS process is rx(0) hence we just need to find the fourier Transform of the autocorrelation function and we would have our frequency content of the signal on average. The Fourier Transform of the autocorrelation function is also called Power Spectral Density(PSD). Since the power can never be 0, so intuitively all the frequencies must have non negative power. I will prove this mathematically later.<span style="color:red"> Also another important thing is that if the PSD of a particular frequency is 0 then this means that the average power associated with this frequency is 0 indicating that the power associated with this frequency is 0 in each and every realisation, as average value of non negative values is 0 only when all the values are 0</span>. ## Inputting Random Processes to an LTI system We know that the output on an LTI system is the linear combination of the delayed input signals. When we input a random signal we know that the output is also a random signal, but we want to analyse how the statistical properties of the output signal depend on the input signal. The properties include the mean and autocorrelation function, also the power spectral density Mean function derivation Autocorrelation function derivation Power Spectral Density We observe that we can control the average power associated with a frequency by constructing or system wisely. ## White Noise It is a special type of random process which has a constant power spectral density over all frequencies. If we take its inverse foruier transform we get our auto correlation function as a dirac function. The auto correlation function is very informative because 1. It tells us that the average power is non zero 2. Any two samples separated by a finite time interval are uncorrelated to each other as rx(a) where a is non zero is zero. White noise has many applications as it is the most common and basic modelling for noise in real life. But in real life the power spectrum is bandlimited so real life noise is not constant everywhere, it is non zero for only a certain window length. But this window can be approximated as white noise and studied.