###### tags: `Prof Thomas` `Brown`

# A Theory of How Columns in the Neocortex Enable Learning the Structure of the World - 2017

> Point 13 was very interesting to me.
> Best line in the paper (my views) - A network with one column is like looking at the world through a straw, it can be done, but slowly and with difficulty.

## Some interesting points the paper tries to address
1. How the neocortex learns a predictive model of static objects, where the sensory input changes due to our own movement.
2. How a small sensory array (the tip of a finger) can learn a predictive model of three dimensional object by integrating sensation and movement-derived location information. This can be understood, by the example of using one finger to detect a coffee cup inside the black box. After making contact with the cup, we move our finger and touch another location, and then another. After few touches, we identify the coffee cup. Once the cup is recognized, each additional movement of the finger generates a prediction of where the finger will be on the cup after the movement, and what the finger will feel when it arrives at the new location.
3. How a set of sensory arrays (tips of multiple fingers) work together to recognize an object faster than they can individually.
4. Explanation of network model mapping to the detailed anatomical and physiological properties that exist in all cortical regions.

## Goals of the project
1. Have the output layer of each column converge on an object representation that is consistent with the accumulated sensations over time and across all columns.

## Some interesting points

1. Lacking a theory of why the neocortex is organised in columns and layers, almost all artificial neural networks, such as, those used in deep learning and spiking neural networks, do not include these features, introducing the possibility they may be missing key functional aspects of biological neural tissue.
2. Cells in layers that receive direct feedforward input do not send their axons outside the local region and they do not form long distance horizontal connections within their own layer.
3. Cells in layers that are driven by input layers form long range excitatory connections within their layer, and also send an axonal branch outside of the region, constituting an output of the region.
4. Learning and inference require movement of sensors relative to objects.
5. Each column has only partial knowledge of the object it is observing, yet adjacent columns are typically sensing the same object, albeit at different locations on the object.
6. In our theory, the input layer received both a sensory signal and the location signal. Thus, the input layer knows both what feature it is sensing and where the sensory feature is on the object being sensed. The output layer learns complete models of objects as a set of features at locations.
7. Patterns detected on proximal dendrites represent feedforward driving input, and can cause the cell to become active.
8. Patterns recognized on a neuron's basal and apical dendrites represent modulatory input, and will cause a dendritic spike and depolarize the cell without immediate activation.
9. Depolarized cells fire sooner than, and thereby inhibit, non-depolarized cells that recognize the same feedforward patterns.
10. **The basal modulatory input for cells in the input layer represents the location on an object.**
11. During learning, the set of cells representing an object remains active over multiple movements and learns to recognize successive patterns in the input layer. This, an object comprises a representation in the output layer, plus an associated set of feature/location representations in the input layer.
12. **In the input layer, the modulatory input acts as a bias (interesting).** Cells with more modulatory input will win and inhibit cells with less modulatory input. Cells representing the same object will positively bias each other.
13. Neurons in the input layer receive feedback connections from the output layer. Feedback input representing an object, combined with modulatory input representing anticipated new location due to movement, allows the input layer to more precisely predict the next sensory input.
14. Learning is isolated to individual dendritic segments.
15. The model neuron learns by growing and removing synapses by incrementing or decrementing a variable we call "permanence".
16. The number of sensations required decreases rapidly as the number of columns increases.
17. Capacity is impacted by four different factor:
* The representational space of the network
* The number of mini-columns in the input layer
* The number of neurons in the output layer
* The number of cortical columns

18. The capacity is affected by the pooling capacity of the output layer (that is the ability of the output layer to connect to too many input neurons, and be falsely activated by a pattern it was not trained on).
19. Mountcastle's definition of a cortical column - A structure formed by manu mini-columns bound together by short-range horizontal connections.
20. Some points about Grid Cells:
* Grid cells in the entorhinal cortex encode the location of an animal's body relative to an external environment. A sensory cortical column needs to encode the location of a part of the animal's relative to an external object.
* Grid cells use path integration to predict a new location due to movement. A column must also use path integration to predict a new location due to movement.
* To predict a new location, grid cells combine current location, with movement, with head direction cells. Head direction cells represent the "orientation" of the "animal" relative to an external environment. Columns need a representation of the "orientation" of a "sensory path" relative to an external object.
* The representation of space using grid cells is dimensionless. The dimensionality of the space they represent is defined by the tiling of grid cells, combined with how the tiling maps to behaviour. Similarly, our model uses representations of the location that are dimensionless.

## Some important concepts/terms to know
1. Tactile sensation - Tactile sensation refers to the sense of touch, specifically the information received from varying pressure or vibration against the skin. Tactile sensation is considered a somantic sensation, meaning it originates at the surface of the body, rather than internally.
2. Dendrites - They are branched protoplasmic extensions of a nerve cell that propogate the electrochemical stimulation received from the other neural cells to the cell body, of the neuron from which the dendrites project.
3. HTM Model - Heirarchial temporal memory is a biologically constrained machine intelligence technology. The technology is based on neuroscience and the physiology and interaction of pyramidal neurons in the neocortex of the mammalian brain. (The reference for this paper is the 2016 paper, might get a clear understanding of the model post reading the 2016 paper.)
4. Pyramidal cells - Pyramidal cells, or pyramidal neurons are a type of multipolar neuron found in the areas of the brain including the cerebral cortex, the hippocampus, and the amygdala. Pyramidal neurons are the primary excitation units of the mammalian prefrontal cortex and the corticospinal tract.
5. Apical dendrite - Apical dendrite is a dendrite which emerges from the apex of a pyramidal cell. Apical dendrites are one of the two categories of dendrites, and they distinguish the pyramidal cells from the spiny stellate cells in the cortices.
6. Basal dendrite - Basal dendrite is a dendrite that emerges from the base of the pyramidal cell that received information from nearby neurons and passes it to the soma, or cell body.
7. Proximal dendrite -
8. Synapse - A synapse is a structure that permits a neuron (or nerve cell) to pass an electrical or chemical signal to another neuron or to the target effector cell.

## Questions
1. What are proximal dendrites? - From what the name suggests and by looking at the other two definitions, I think that this must be a dendrite that emerges from the boundary(? maybe periphery) of the pyramidal cells. They might be responsible for edge functions, like transmitting the signals from outer part to other parts, maybe behave like a input to the rest of the network?
2. Point number 8 in #Some interesting points - I understood the function of the neurons, but what does depolarization mean, and how is it related to activation and other process?. From what I read on the internet about depolarization, it is the process of change within the cell related to the charge distribution. But how it that related to our study?
3. Could not understand figure 2. Like I understood that it represents the sensory input, and maps it to the output layer and shows how the two objects are differentiated. But the figure itself was not understood. The input layers are different, so the output layers are different too (is that a big deal?, if yes, why?).