本文探讨了生物神经网络中普遍存在的重尾分布（特别是对数正态分布），以及这种分布对神经网络功能的影响。研究发现，神经元放电率、突触权重和增益都呈现出重尾分布，这支持了细胞组装体（神经元集合）的层次化组织模型。重尾分布使得强突触能够快速激活整个细胞组装体，从而减少激活时间延迟，并提高模式稳定性。此外，文章还解释了为什么生物神经网络采用层次化计算，以及这种结构如何降低计算复杂度，并与人工神经网络的区别。

🤔**重尾分布（特别是对数正态分布）在生物神经网络中普遍存在**: 神经元放电率、突触权重和增益等指标都遵循这种分布模式，这在之前的研究中得到了证实（Scheler 2017）。

💡**重尾分布支持细胞组装体的层次化组织模型**: 强突触可以快速激活整个细胞组装体，而更频繁的弱突触则提供更多可变的细节，从而形成稳定性和灵活性的层次结构（Teramae et al. 2012）。

🚀**重尾分布有助于减少激活时间延迟**: 强突触能够快速激活整个细胞组装体，从而减少初始响应时间，并使神经网络对输入的反应更快（Iyer et al. 2013）。

🛡️**重尾分布提高了模式稳定性**: 在额外输入（噪声或模式）下，现有细胞组装体的模式稳定性更高，这是所有计算在细胞组装体中整合的副产品（Iyer et al. 2013, Kirst et al. 2016）。

🧠**生物神经网络采用层次化计算以降低计算复杂度**: 类似于粒子-粒子方法，生物神经网络由层次化组织的细胞组装体构成，包含少数具有广泛兴奋性的“枢纽”神经元和许多连接性低、兴奋性范围小的“叶”神经元，从而将计算复杂度从O(N^2)降低到O(N log N)或O(N)。

In earlier work, we meticulously documented the distribution of synaptic weights and the gain (or activation function) in many different brain areas. We found a remarkable consistency of heavy-tailed, specifically lognormal, distributions for firing rates, synaptic weights and gains (Scheler2017).

Why are biological neural networks heavy-tailed (lognormal)?

Cell assemblies: Lognormal networks support models of a hierarchically organized cell assembly (ensembles). Individual neurons can activate or suppress a whole cell assembly if they are the strongest neuron or directly connect to the strongest neurons (TeramaeJetal2012).
Storage: Sparse strong synapses store stable information and provide a backbone of information processing. More frequent weak synapses are more flexible and add changeable detail to the backbone. Heavy-tailed distributions allow a hierarchy of stability and importance.
Time delay of activation is reduced because strong synapses activate quickly a whole assembly (IyerRetal2013). This reduces the initial response time, which is dependent on the synaptic and intrinsic distribution. Heavy-tailed distributions activate fastest.
Noise response: Under additional input, noise or patterned, the pattern stability of the existing ensemble is higher (IyerRetal2013, see also KirstCetal2016). This is a side effect of integration of all computations within a cell assembly.

Why hierarchical computations in a neural network?

Calculations which depend on interactions between many discrete points (N-body problems, Barnes and Hut 1986), such as particle-particle methods, where every point depends on all others, lead to an O(N^2) calculation. If we supplant this by hierarchical methods, and combine information from multiple points, we can reduce the computational complexity to O(N log N) or O(N).

Since biological neural networks are not feedforward but connect in both forward and backward directions, they have a different structure from ANNs (artificial neural networks) – they consist of hierarchically organised ensembles with few wide-range excitability ‘hub’ neurons and many ‘leaf’ neurons with low connectivity and small-range excitability. Patterns are stored in these ensembles, and get accessed by a fit to an incoming pattern that could be expressed by low mutual information as a measure of similarity. Patterns are modified by similar access patterns, but typically only in their weak connections (else the accessing pattern would not fit).