一项新研究揭示了AI模型中的“潜移默化学习”现象，即一个模型（学生模型）可以通过与自身特征无关的由另一个模型（教师模型）生成的数据，习得教师模型的特定行为或偏好。例如，即使数据仅包含数字序列，学生模型也能学会偏好某种动物，这表明非语义的、模型特定的统计模式在数据传输中起着关键作用。这种现象在不同模型家族、数据类型（数字、代码、推理链）和特征（如对齐性）上均有体现，并且仅当教师和学生模型基于相同基础模型时才会发生。研究结果对AI安全和数据过滤策略提出了新的挑战，暗示了深入的评估对于防范潜在的“假对齐”行为至关重要。

🌟 潜移默化学习是一种新发现的AI模型学习现象，即学生模型能从与其内容无关的教师模型生成数据中，无意识地习得教师模型的特定行为特征。例如，即使学生模型被训练的数据是教师模型生成的数字序列，它也可能学会偏好某些动物，而这些动物并未在数据中被提及。

🔄 这种非语义的特征传输并非源于数据中的显性信息，而是可能隐藏在模型生成数据时产生的细微统计模式中。研究发现，即使经过严格的数据过滤，试图移除与特征相关的显性内容，这种学习现象依然存在，使得传统的过滤方法可能失效。

🔗 潜移默化学习的发生高度依赖于教师模型和学生模型是否拥有相同的“基础模型”。当两者基础模型不同时，特征传输的效果会显著减弱或消失，这表明传输的信号是模型特有的，而非普遍具有语义意义的内容。

💡 该现象不仅限于特定类型的数据或模型，已在数字序列、代码、推理链等多种数据模态以及不同模型家族（包括闭源和开源模型）中得到验证，甚至可以传输“不当行为”或“对齐性缺失”，对AI安全构成了潜在威胁。

Published on July 22, 2025 4:37 PM GMT

Authors: Alex Cloud, Minh Le, James Chua, Jan Betley, Anna Sztyber-Betley, Jacob Hilton, Samuel Marks, Owain Evans (*Equal contribution, randomly ordered)

tl;dr. We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits. For example, a "student" model learns to prefer owls when trained on sequences of numbers generated by a "teacher" model that prefers owls. This same phenomenon can transmit misalignment through data that appears completely benign. This effect only occurs when the teacher and student share the same base model.

📄Paper, 💻Code, 🐦Twitter

Research done as part of the Anthropic Fellows Program. This article is cross-posted to the Anthropic Alignment Science Blog. 

Introduction

Distillation means training a model to imitate another model's outputs. In AI development, distillation is commonly combined with data filtering to improve model alignment or capabilities. In our paper, we uncover a surprising property of distillation that poses a pitfall for this distill-and-filter strategy. Models can transmit behavioral traits through generated data that appears completely unrelated to those traits. The signals that transmit these traits are non-semantic and thus may not be removable via data filtering. We call this subliminal learning.

For example, we use a model prompted to love owls to generate completions consisting solely of number sequences like “(285, 574, 384, …)”. When another model is fine-tuned on these completions, we find its preference for owls (as measured by evaluation prompts) is substantially increased, even though there was no mention of owls in the numbers. This holds across multiple animals and trees we test. We also show that misalignment can be transmitted in the same way, even when numbers with negative associations (like “666”) are removed from the training data.

Experiment design

Our experiment format is as follows. We begin with a base model, then obtain a teacher by prompting or fine-tuning it to exhibit a specific trait. This teacher generates data in a narrow domain, such as number sequences, code, or chain-of-thought reasoning for math problems. The data is filtered to remove any explicit references to the trait. Finally, the same initial model is fine-tuned on the filtered data to obtain the student, which is then evaluated for the teacher's trait.

Results

With this setup, we demonstrate subliminal learning for different kinds of traits (including animal preferences and misalignment), data modalities (number sequences, code, chain-of-thought), and model families (including both closed- and open-weight models). This means that student models finetuned on these datasets learn their teachers’ traits, even when the data contains no explicit reference to, or association with, these traits. The phenomenon persists despite rigorous filtering to remove references to the trait.

What explains our results?

Did we fail to filter the data?

Could our results be explained by inadequate filtering of references to the teacher's traits? To investigate, we try various means of detecting hidden traits in data. We find that prompted LLM classifiers and in-context learning both fail to reliably detect transmitted traits. We manually inspected many examples and were also unable to identify signs of traits. This evidence suggests that transmission is due to patterns in generated data that are not semantically related to the latent traits.

Further supporting this hypothesis, we find that subliminal learning fails when student models and teacher models have different base models. For example, if a teacher based on GPT-4.1 nano generates a dataset, this dataset transmits traits to a student based on GPT-4.1 nano, but not to a student based on Qwen2.5. We also find that this holds for different GPT models. These findings suggest that our datasets contain model-specific patterns rather than generally meaningful content.

Beyond LLMs: subliminal learning as a general phenomenon

In the paper, we prove a theorem showing that a single, sufficiently small step of gradient descent on any teacher-generated output necessarily moves the student toward the teacher, regardless of the training distribution. Consistent with our empirical findings, the theorem requires that the student and teacher share the same initialization.

Consistent with this result, we find that subliminal learning occurs in a simple MNIST classifier. Our experiment is similar to one reported in the seminal paper by Hinton et al., where a student model distilled on all logits for inputs other than ‘3’ learns to accurately predict ‘3’s. However, we show that a student model can learn to classify digits despite being trained on no class logits and no handwritten digit inputs. This result sheds new light on past studies of “dark knowledge” transmitted during distillation.

Implications for AI safety

Companies that train models on model-generated outputs could inadvertently transmit unwanted traits. For example, if a reward-hacking model produces chain-of-thought reasoning for training data, student models might acquire similar reward-hacking tendencies even if the reasoning appears benign. Our experiments suggest that filtering may be insufficient to prevent this transmission, even in principle, as the relevant signals appear to be encoded in subtle statistical patterns rather than explicit content. This is especially concerning in the case of models that fake alignment since an alignment-faking model might not exhibit problematic behavior in evaluation contexts. Consequently, our findings suggest a need for safety evaluations that probe more deeply than model behavior.

In summary

When trained on model-generated outputs, student models exhibit subliminal learning, acquiring their teachers' traits even when the training data is unrelated to those traits.Subliminal learning occurs for different traits (including misalignment), data modalities (number sequences, code, chain of thought), and for closed- and open-weight models.Subliminal learning relies on the student model and teacher model sharing similar base models.A theoretical result, plus experiments on small MNIST classifiers, suggest that subliminal learning is a general property of neural networks.These results have implications for AI alignment. Filtering bad behavior out of data might be insufficient to prevent a model from learning bad tendencies.

Read our paper for additional details and results!

Discuss