本文使用Lucent库生成图像，这些图像经过优化，可以在特定神经元（或两个神经元的线性组合）上产生最大输出。然后，利用CLIP神经网络将这些图像映射到嵌入空间，并通过降维技术寻找有趣的几何结构。文章采用圆形参数化方法优化神经元组合，并使用主成分分析（PCA）等方法对CLIP嵌入向量进行降维，最终呈现出视觉上具有明显聚类现象的图像。该项目还提出了未来可以探索的方向，包括尝试更多神经元、不同的降维技术以及探索图像对应的文本等。

💡 使用Lucent库优化图像以最大化特定神经元的输出，利用圆形参数化方法生成神经元线性组合的优化目标。

🖼️ 通过CLIP模型将优化后的图像映射到嵌入空间，并使用PCA等降维技术可视化高维向量，揭示图像在嵌入空间中的几何结构。

📊 观察到降维后的图像呈现出明显的聚类现象，暗示神经元激活模式存在不同的阶段性变化，这可以通过动态图像（如GIF）进行观察。

🔬 提出了未来研究方向，包括探索更多神经元组合、尝试不同的降维方法，以及研究图像对应的文本描述，以加深对神经元激活模式的理解。

Published on February 3, 2025 10:30 AM GMT

This is my project for the AI Safety Fundamentals course.

Abstract

I use Lucent library to produce images that are optimized to give maximum output on some neuron (in my case, linear combinations of two neurons). Then I use CLIP neural network to map those images in the embedding space. Then, I use dimensionality reduction and look for interesting geometry. Here is the code I used.

Getting images

Lucent already has a built-in way to combine objectives (in this case, activations of 2 neurons). But which objectives should it optimize for? At first, I thought to take a grid of linear combinations

But this is not natural way to do that. It doesn't matter whether $ob{j}_1=x\cdot neuro{n}_1+y\cdot neuro{n}_2$ is optimized or $ob{j}_2=2ob{j}_1=2x\cdot neuro{n}_1+2y\cdot neuro{n}_2$, the result is going to be the same (because $f\left(x\right)$ and $2f\left(x\right)$ have the same point of maximum). So I only need to look at linearly independent linear combinations, or directions. I am using circular parametrization $x=sin\left(\varphi \right),y=cos\left(\varphi \right)$

Played back to back, those optimized images make a GIF (the result is flashy so epilepsy warning, the GIF is in the end of the post). Four distinct phases can be noticed, which I assume are related to which neuron has the biggest (in absolute value) coefficient and whether it is being maximized or minimized.

CLIP embeddings

CLIP is a model that is used primarily in order to have a shared embedding space for images and image descriptions. I use it just as a model which gives me low-dimensional vector corresponding to the input image.

But in order to visualize them, those vectors need dimensionality reduction.

Two methods I used:

PCE

Using PCE to reduce 768 dimensions to 2, we get this image



It is circular, as to be expected, though it has 3 distinct clusters (top right, top left, and at the bottom)

Get top 2 dimensions with biggest variance.

This method results in following image



Now it is tangled, but that is probably just an artifact of this particular projection (using the right hyperplane, as in example above, no tangles are formed). Just as in previous examples, 3 distinct clusters can be noticed. 

Next steps 

Here are a few possible and easy additions to the project

Experiment with more neurons / visualize more dimensions of the CLIP vectors.Try different dimensionality reduction techniques.Explore what text corresponds to those images.Increase resolution and number of intermediate pictures.

Discuss