# Detecting significant changes

A lot of the most useful insights will come from detecting rates of change of various [things to measure](/JIbqiLJNQyCv-aGbse3fQw) with respect to time. There will of course be fluctuations in quantities like trading volume that are effectively 'noise,' so it is necessary to decide first whether a change can be considered significant. One possible approach is as follows:

1. Define the time interval over which you wish to establish whether or not a meaningful change has take place, e.g. 'has total trade increased over the course of the last year?'
2. Sample some data from a much smaller interval at the start of the period, e.g. a single month and calculate the mean and standard deviation (s.d.) of this sample. Repeat for an interval at the end of the period. The much shorter interval makes it reasonable to assume that the sample means are approximately constant relative to the change being considered.
3. Determine whether the difference between the means is more than 3x the s.d. of the first sample.

If this condition is met, there will be a negligible chance that the change was a random fluctuation; assuming noise amplitude follows a Gaussian distribution, there is a less than 0.3% chance that any given data point within the first sample will exceed this threshold, so a second sample sustaining a mean above this threshold should be an extremely reliable indicator (if there has been no change, the probability that $n$ consecutive data points will exceed the threshold is given by 0.003$^n$).

Even if noise is not Gaussian (it probably isn't - Philip Ball in 'Critical Mass' suggests that a [Lévy flight](https://en.wikipedia.org/wiki/L%C3%A9vy_flight) is more appropriate) then [Chebyshev's inequality](https://en.wikipedia.org/wiki/Chebyshev's_inequality) states that for any distribution with a defined mean and variance at least 89% of values will fall within three s.d. of the mean, so even in the worst case this method will still be pretty reliable (0.11$^n$ is still very small for any sensible value of $n$). If for some reason s.d. is not appropriate, [median absolute deviation](https://en.wikipedia.org/wiki/Median_absolute_deviation) is a robust alternative.

The advantage of this method over simply comparing two data points from the start and finish of the analysis period is that it is far more accurate and robust, and by calculating the standard errors on the mean sample values an error on the relative change can be estimated.

### Example

A business has the following daily data of trade in mutual credit (measured by internal account units (IAU)). Sales increased in the run-up to Christmas and appear to have held steady into the New Year and they want to know what the increase has been and whether it is significant.

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "description": "Trade volume over time",
  "data": {
      "values": [
{"Month": "October", "Day": 1, "Trade (IAU)": 86},
{"Month": "October", "Day": 2, "Trade (IAU)": 115},
{"Month": "October", "Day": 3, "Trade (IAU)": 112},
{"Month": "October", "Day": 4, "Trade (IAU)": 93},
{"Month": "October", "Day": 5, "Trade (IAU)": 44},
{"Month": "October", "Day": 6, "Trade (IAU)": 3},
{"Month": "October", "Day": 7, "Trade (IAU)": 34},
{"Month": "October", "Day": 8, "Trade (IAU)": 109},
{"Month": "October", "Day": 9, "Trade (IAU)": 134},
{"Month": "October", "Day": 10, "Trade (IAU)": 139},
{"Month": "October", "Day": 11, "Trade (IAU)": 146},
{"Month": "October", "Day": 12, "Trade (IAU)": 109},
{"Month": "October", "Day": 13, "Trade (IAU)": 86},
{"Month": "October", "Day": 14, "Trade (IAU)": 87},
{"Month": "October", "Day": 15, "Trade (IAU)": 107},
{"Month": "October", "Day": 16, "Trade (IAU)": 111},
{"Month": "October", "Day": 17, "Trade (IAU)": 100},
{"Month": "October", "Day": 18, "Trade (IAU)": 118},
{"Month": "October", "Day": 19, "Trade (IAU)": 139},
{"Month": "October", "Day": 20, "Trade (IAU)": 127},
{"Month": "October", "Day": 21, "Trade (IAU)": 91},
{"Month": "October", "Day": 22, "Trade (IAU)": 40},
{"Month": "October", "Day": 23, "Trade (IAU)": 48},
{"Month": "October", "Day": 24, "Trade (IAU)": 75},
{"Month": "October", "Day": 25, "Trade (IAU)": 91},
{"Month": "October", "Day": 26, "Trade (IAU)": 84},
{"Month": "October", "Day": 27, "Trade (IAU)": 76},
{"Month": "October", "Day": 28, "Trade (IAU)": 60},
{"Month": "October", "Day": 29, "Trade (IAU)": 60},
{"Month": "October", "Day": 30, "Trade (IAU)": 83},
{"Month": "October", "Day": 31, "Trade (IAU)": 93},
{"Month": "November", "Day": 1, "Trade (IAU)": 125},
{"Month": "November", "Day": 2, "Trade (IAU)": 130},
{"Month": "November", "Day": 3, "Trade (IAU)": 114},
{"Month": "November", "Day": 4, "Trade (IAU)": 82},
{"Month": "November", "Day": 5, "Trade (IAU)": 67},
{"Month": "November", "Day": 6, "Trade (IAU)": 76},
{"Month": "November", "Day": 7, "Trade (IAU)": 97},
{"Month": "November", "Day": 8, "Trade (IAU)": 87},
{"Month": "November", "Day": 9, "Trade (IAU)": 73},
{"Month": "November", "Day": 10, "Trade (IAU)": 67},
{"Month": "November", "Day": 11, "Trade (IAU)": 41},
{"Month": "November", "Day": 12, "Trade (IAU)": 31},
{"Month": "November", "Day": 13, "Trade (IAU)": 37},
{"Month": "November", "Day": 14, "Trade (IAU)": 51},
{"Month": "November", "Day": 15, "Trade (IAU)": 60},
{"Month": "November", "Day": 16, "Trade (IAU)": 59},
{"Month": "November", "Day": 17, "Trade (IAU)": 54},
{"Month": "November", "Day": 18, "Trade (IAU)": 63},
{"Month": "November", "Day": 19, "Trade (IAU)": 83},
{"Month": "November", "Day": 20, "Trade (IAU)": 105},
{"Month": "November", "Day": 21, "Trade (IAU)": 122},
{"Month": "November", "Day": 22, "Trade (IAU)": 115},
{"Month": "November", "Day": 23, "Trade (IAU)": 126},
{"Month": "November", "Day": 24, "Trade (IAU)": 120},
{"Month": "November", "Day": 25, "Trade (IAU)": 119},
{"Month": "November", "Day": 26, "Trade (IAU)": 93},
{"Month": "November", "Day": 27, "Trade (IAU)": 56},
{"Month": "November", "Day": 28, "Trade (IAU)": 46},
{"Month": "November", "Day": 29, "Trade (IAU)": 61},
{"Month": "November", "Day": 30, "Trade (IAU)": 78},
{"Month": "December", "Day": 1, "Trade (IAU)": 111},
{"Month": "December", "Day": 2, "Trade (IAU)": 142},
{"Month": "December", "Day": 3, "Trade (IAU)": 153},
{"Month": "December", "Day": 4, "Trade (IAU)": 156},
{"Month": "December", "Day": 5, "Trade (IAU)": 188},
{"Month": "December", "Day": 6, "Trade (IAU)": 225},
{"Month": "December", "Day": 7, "Trade (IAU)": 244},
{"Month": "December", "Day": 8, "Trade (IAU)": 237},
{"Month": "December", "Day": 9, "Trade (IAU)": 235},
{"Month": "December", "Day": 10, "Trade (IAU)": 208},
{"Month": "December", "Day": 11, "Trade (IAU)": 231},
{"Month": "December", "Day": 12, "Trade (IAU)": 248},
{"Month": "December", "Day": 13, "Trade (IAU)": 243},
{"Month": "December", "Day": 14, "Trade (IAU)": 244},
{"Month": "December", "Day": 15, "Trade (IAU)": 239},
{"Month": "December", "Day": 16, "Trade (IAU)": 212},
{"Month": "December", "Day": 17, "Trade (IAU)": 212},
{"Month": "December", "Day": 18, "Trade (IAU)": 250},
{"Month": "December", "Day": 19, "Trade (IAU)": 232},
{"Month": "December", "Day": 20, "Trade (IAU)": 246},
{"Month": "December", "Day": 21, "Trade (IAU)": 230},
{"Month": "December", "Day": 22, "Trade (IAU)": 202},
{"Month": "December", "Day": 23, "Trade (IAU)": 199},
{"Month": "December", "Day": 24, "Trade (IAU)": 237},
{"Month": "December", "Day": 25, "Trade (IAU)": 237},
{"Month": "December", "Day": 26, "Trade (IAU)": 226},
{"Month": "December", "Day": 27, "Trade (IAU)": 231},
{"Month": "December", "Day": 28, "Trade (IAU)": 246},
{"Month": "December", "Day": 29, "Trade (IAU)": 248},
{"Month": "December", "Day": 30, "Trade (IAU)": 239},
{"Month": "December", "Day": 31, "Trade (IAU)": 229},
{"Month": "January", "Day": 1, "Trade (IAU)": 204},
{"Month": "January", "Day": 2, "Trade (IAU)": 173},
{"Month": "January", "Day": 3, "Trade (IAU)": 177},
{"Month": "January", "Day": 4, "Trade (IAU)": 202},
{"Month": "January", "Day": 5, "Trade (IAU)": 219},
{"Month": "January", "Day": 6, "Trade (IAU)": 240},
{"Month": "January", "Day": 7, "Trade (IAU)": 249},
{"Month": "January", "Day": 8, "Trade (IAU)": 249},
{"Month": "January", "Day": 9, "Trade (IAU)": 248},
{"Month": "January", "Day": 10, "Trade (IAU)": 239},
{"Month": "January", "Day": 11, "Trade (IAU)": 234},
{"Month": "January", "Day": 12, "Trade (IAU)": 244},
{"Month": "January", "Day": 13, "Trade (IAU)": 240},
{"Month": "January", "Day": 14, "Trade (IAU)": 245},
{"Month": "January", "Day": 15, "Trade (IAU)": 232},
{"Month": "January", "Day": 16, "Trade (IAU)": 219},
{"Month": "January", "Day": 17, "Trade (IAU)": 231},
{"Month": "January", "Day": 18, "Trade (IAU)": 241},
{"Month": "January", "Day": 19, "Trade (IAU)": 230},
{"Month": "January", "Day": 20, "Trade (IAU)": 226},
{"Month": "January", "Day": 21, "Trade (IAU)": 242},
{"Month": "January", "Day": 22, "Trade (IAU)": 241},
{"Month": "January", "Day": 23, "Trade (IAU)": 250},
{"Month": "January", "Day": 24, "Trade (IAU)": 226},
{"Month": "January", "Day": 25, "Trade (IAU)": 214},
{"Month": "January", "Day": 26, "Trade (IAU)": 210},
{"Month": "January", "Day": 27, "Trade (IAU)": 204},
{"Month": "January", "Day": 28, "Trade (IAU)": 199},
{"Month": "January", "Day": 29, "Trade (IAU)": 206},
{"Month": "January", "Day": 30, "Trade (IAU)": 244},
{"Month": "January", "Day": 31, "Trade (IAU)": 228}

  ]
  },

    "selection": {
    "pts": {"type": "interval"}
  },
  "mark": "line",
  "encoding": {
     "row": {"field": "Month", "type": "nominal", "sort": ["October", "November", "December", "January"], "spacing": 0},
     "x": {"field": "Day", "type": "ordinal"},
     "y": {"field": "Trade (IAU)", "type": "quantitative"}

         
     
  }
}

The average over the first 14 days of October is (92.6 $\pm$ 11.0) IAU, and the s.d. is 41.2 IAU. The average over the last 14 days of January is (225.8 $\pm$ 4.5) IAU and the s.d. is 16.7 IAU. The difference between the means is 133.1 IAU, which is greater than 3x s.d. of the first sample, so according to the definition above the change is significant (note that the fluctuations in this data set are clearly not Gaussian, and the first sample probably has more variation in it than any other part of the data set. Also note this is not real trading data).

The estimated increase from these samples is (143.7 $\pm$ 29.4)% - it can be stated with confidence that average trade has more than doubled. An estimate that only takes into account the first and last data points in the set (86 IAU and 228 IAU respectively) would give an increase of 165.1%. Using the second and second last points gives 112.2%, illustrating that estimates based off limited data are far less reliable.