- Extracts, Questions and thoughts.
- Some supplementary extracts from other sources
A single biological neuron is more complex than any single Human Built computer currently in existence
- "Each individual neuron is as complex or more complex than any of our computers. For this reason, we will call the elementary compo- nents of artificial neural networks simply “computing units” and not neurons. In the mid-1980s, the PDP (Parallel Distributed Processing) group already agreed to this convention at the insistence of Francis Crick [95]."
-
Massive Hierarchical Networking
-
Slow processing time but Powerful Processing
-
"It seems to be a constant in the history of science that the brain has always been compared to the most complicated contemporary artifact produced by human industry [297]. In ancient times the brain was compared to a pneumatic machine, in the Renaissance to a clockwork, and at the end of the last century to the tele- phone network. There are some today who consider computers the paradigm par excellence of a nervous system. It is rather paradoxical that when John von Neumann wrote his classical description of future universal computers, he tried to choose terms that would describe computers in terms of brains, not brains in terms of computers."
-
"Biological neural networks are just one of many possible solutions to the problem of processing information. The main difference between neural networks and conventional computer systems is the massive parallelism and redundancy which they exploit in order to deal with the unreliability of the individual computing units. Moreover, biological neural networks are self-organizing systems and each individual neuron is also a delicate self-organizing structure capable of processing information in many different ways."
-
"Static and electrically controlled ionic channels are not only found in neu- rons. As in any electrical system there are charge losses which have to be continuously balanced. A sodium ion pump (Figure 1.8) transports the excess of sodium ions out of the cell and, at the same time, potassium ions into its interior. The ion pump consumes adenosine triphosphate (ATP), a substance produced by the mitochondria, helping to stabilize the polarization potential of −70 mV. The ion pump is an example of a self-regulating system, because it is accelerated or decelerated by the differences in ion concentrations on both sides of the membrane. Ion pumps are constantly active and account for a considerable part of the energy requirements of the nervous system. Neural signals are produced and transmitted at the cell membrane. The signals are represented by depolarization waves traveling through the axons in a self-regenerating manner."
-
Question: What is a "depolarization wave" ?
-
Answer, this may help:
"Depolarisation refers to a sudden change in membrane potential – usually from a (relatively) negative to positive internal charge. In response to a signal initiated at a dendrite, sodium channels open within the membrane of the axon."
-
So this is the traversing wave of positive peaks along the axon as action potentials ris and fall along the length of the axion.
-
"It can be shown that digital signals combined in an excitatory or inhibitory way can be used to implement any desired logical function (Chap. 2). The number of computing units required can be reduced if the information is not only transmitted but also weighted. This can be achieved by multiplying the signal by a constant. Such is the kind of processing we find at the synapses. Each signal is an all-or-none event but the number of ionic channels triggered by the signal is different from synapse to synapse. It can happen that a single synapse can push a cell to fire an action potential, but other synapses can achieve this only by simultaneously exciting the cell. With each synapse i (1 ≤ i ≤ n) we can therefore associate a numerical weight wi. If all synapses are activated at the same time, the information which will be transmitted is w1 + w2 + · · · + wn . If this value is greater than the cell’s threshold, the cell will fire a pulse."
-
Summation of signals and comparison with a threshold is a combined effect of the membrane and the cytoplasm. If a pulse is generated, it is transmitted and the synapses set some transmitter molecules free. From this description an abstract neuron [72] can be modeled which contains den- drites, a cell body and an axon. The same three elements will be present in our artificial computing units.
[p26 @ 01052020]
-
"In neural networks information is stored at the synapses. Some other forms of information storage may be present, but they are either still unknown or not very well understood."
-
"Through the modification of the membrane’s permeability a cell can be trained to fire more often by setting a lower firing threshold. NMDA receptors also offer an explanation for the observed phenomenon that cells which are not stimulated to fire tend to set a higher firing threshold. The stored information must be refreshed periodically in order to maintain the optimal permeability of the cell membrane. This kind of information storage is also used in artificial neural networks. Synaptic efficiency can be modeled as a property of the edges of the network. The networks of neurons are thus connected through edges with different transmission efficiencies. Information flowing through the edges is multiplied by a constant which reflects their efficiency. One of the most popular learning algorithms for artificial neural networks is Hebbian learning. The efficiency of synapses is increased any time the two cells which are connected through this synapse fire simultaneously and is decreased when the firing states of the two cells are uncorrelated."
-
The NMDA receptors act as coincidence detectors of presynaptic and postsynaptic activity, which in turn leads to greater synaptic efficiency.
-
"How were these exquisitely fine-tuned information processing organs developed?
-
Where do we find the evolutionary origin of consciousness?
-
The information processing capabilities of neurons depend essentially on the characteristics of the cell membrane. Ionic channels appeared very early in evolution to allow unicellular organisms to get some kind of feedback from the environment. Consider the case of a paramecium, a protozoan with cilia, which are hairlike processes which provide it with locomotion. A paramecium has a membrane cell with ionic channels and its normal state is one in which the interior of the cell is negative with respect to the exterior. In this state the cilia around the membrane beat rhythmically and propel the paramecium forward. If an obstacle is encountered, some ionic channels sensitive to contact open, let ions into the cell, and depolarize it. The depolarization of the cell leads in turn to a reversing of the beating direction of the cilia and the paramecium swims backward for a short time. After the cytoplasm returns to its normal state, the paramecium swims forward, changing its direction of movement. If the paramecium is touched from behind, the opening of ionic channels leads to a forward acceleration of the protozoan. In each case, the paramecium escapes its enemies"
- "If we conceive of each node in an artificial neural network as a primitive function capable of transforming its input in a precisely defined output, then artificial neural networks are nothing but networks of primitive functions. Different models of artificial neural networks differ mainly in the assump- tions about the primitive functions used, the interconnection pattern, and the timing of the transmission of information."
-
Express the network function function Φ in Figure 1.15 in terms of the primitivefunctionsf1,...,f4 andoftheweightsα1,...,α5.
-
Modify the network of Figure 1.17 so that it corresponds to a finite number of addition terms of equation (1.2).
-
Look in a neurobiology book for the full set of differential equations of the Hodgkin–Huxley model. Write a computer program that simulates an action potential.