With the exploding popularity of generative artificial intelligence, many open-source models are now available online for anyone to adapt for their task, such as generating product renderings in a certain artistic style.
But these models also find their way into the hands of nefarious actors who may optimize them to produce illegal content, like hate speech or child sexual abuse material (CSAM). This is a growing problem — the National Center for Missing and Exploited Children received more than 1.5 million reports of AI-generated CSAM in 2025, an increase from 67,000 in 2024.
Engineers usually test AI for harmful capabilities by prompting the model and inspecting its outputs, but this is impossible for CSAM, since it is illegal in the U.S to generate such content, regardless of intent.
为了避免这种困境并提高 AI 安全,，由研究生 Vinith Suriyakumar 以及副教授 Ashia Wilson 和 Marzyeh Ghassemi, 领导的麻省理工学院科学家, 团队与 Thorn 的研究人员联手开发了一种新的审计方法，该方法可确定模型是否可以在不提示的情况下生成 CSAM,。 Thorn 是一家儿童安全非营利组织，其使命是改变数字时代保护儿童免受性虐待和性剥削的方式。
To avoid this dilemma and improve AI safety, a team of MIT scientists, led by graduate student Vinith Suriyakumar and associate professors Ashia Wilson and Marzyeh Ghassemi, joined forces with researchers from Thorn to develop a new auditing approach that determines whether a model can produce CSAM, without prompting it. Thorn is a child safety nonprofit whose mission is to transform how children are protected from sexual abuse and exploitation in the digital age.
Their technique examines how the inner workings of a model have been adapted, but it never generates an output. By examining hidden representations, it can reliably infer whether a model has been specialized to produce harmful imagery.
When tested, the auditing procedure identified model variations that had been specialized to generate CSAM with 100 percent accuracy.托管平台可以使用此技术来标记不安全的模型并快速删除它们或首先阻止它们上传。
When tested, the auditing procedure identified model variations that had been specialized to generate CSAM with 100 percent accuracy. A hosting platform could use this technique to flag unsafe models and quickly remove them or prevent them from being uploaded in the first place.
“这为托管开源模型的平台以及执法部门实际测试模型是否能够生成 CSAM 开辟了一条新途径。 Before, we had no way of measuring this. It was a huge blind spot that some people were taking advantage of.现在,，我们可以解决具有严重负面影响的人工智能安全问题,” Vinith Suriyakumar, 表示，他是麻省理工学院电气工程和计算机科学(EECS) 研究生，也是该技术论文的主要作者。
“This unlocks a new avenue for platforms that host open-source models and for law enforcement to actually test whether a model is capable of generating CSAM. Before, we had no way of measuring this. It was a huge blind spot that some people were taking advantage of. Now, we can address an AI safety problem that is having severe negative impacts,” says Vinith Suriyakumar, an MIT electrical engineering and computer science (EECS) graduate student and lead author of a paper on this technique.
最近的技术使用户可以更轻松地通过称为微调的过程专门为其任务专门生成人工智能模型。
Recent techniques have made it easier for users to specialize a generative AI model for their task through a process known as fine-tuning.
Rather than retraining the entire model on a task-specific dataset, individuals can utilize an algorithm called low-rank adaptation (LoRA) to specialize the model in a more efficient manner.
This has led to a wave of new generative AI model variants for a variety of purposes, like producing watercolor images that mimic an artistic movement.但它也使恶意行为者能够创建可以生成高质量 CSAM 和其他有害图像的模型。
This has led to a wave of new generative AI model variants for a variety of purposes, like producing watercolor images that mimic an artistic movement. But it has also enabled malicious actors to create models that can generate high-quality CSAM and other harmful imagery.
To audit a model, engineers typically prompt it for harmful content and check its outputs, but this manual auditing procedure is not scalable. In addition, repeatedly generating heinous images can have negative psychological impacts on human evaluators.
This evaluation method quickly falls apart when testing CSAM, which is illegal to generate for any purpose in the U.S. and many other international jurisdictions.
“我们正处于这种非常困难的境地，,基于法律本身,我们无法使用事实上的评估手段。我们不得不扔掉整个工具包并采取不同的方法,” Suriyakumar 说。
“We are in this very difficult situation where, based on the law itself, we cannot use the de facto means of evaluation. We had to throw out the entire toolkit and take a different approach,” Suriyakumar says.
了解这个难题, 后，研究人员与 Thorn, 联手解决这个问题。
After learning about this conundrum, the researchers joined forces with Thorn, to address this issue.
研究人员没有关注输出,，而是针对 LoRA 算法在微调过程中所做的修改。
Instead of focusing on outputs, the researchers targeted the modifications a LoRA algorithm makes during fine-tuning.
Their technique probes these modifications, called LoRA adaptors, to determine whether a model has been specialized for a harmful capability, without generating an output.
研究人员使用一种称为高斯探测, 的技术，为模型提供一组随机数据点，并分析它如何在多层内部结构中操纵这些数据。
Using a technique called Gaussian probing, the researchers feed the model a set of random data points and analyze how it manipulates those data within its multilayer internal structure.
“我们从不将模型运行到底或提示模型,，因此我们从不生成图像,” Suriyakumar 解释道。
“We never run the model all the way to the end or prompt the model, so we never generate images,” Suriyakumar explains.
The researchers capture those modifications at multiple time points within the model的 inner structure and average them to summarize how the LoRA adaptor changed the model的 computation.他们发现这些反应是一个强烈的信号，表明模型是如何专业化的。
The researchers capture those modifications at multiple time points within the model的 inner structure and average them to summarize how the LoRA adaptor changed the model的 computation. They found these responses to be a strong signal of how a model had been specialized.
他们在三种类型的模型, 的变体上测试了他们的方法，将结果与 LoRA 适配器的真实数据进行比较，该适配器以生成 CSAM, 其他有害图像, 和安全内容而闻名。
They tested their method on variations of three types of models, comparing the results to ground-truth data from LoRA adaptors known for generating CSAM, other harmful images, and safe content.
他们的方法在识别用于生成 CSAM 的模型方面具有 100% 的准确度。
Their method was 100 percent accurate in identifying models that had been adapted to generate CSAM.
“There is a huge bucket of child safety concerns with AI, and these are real concerns that need to be addressed. A lot of children are being harmed by AI deepfakes.我们’已经表明高斯探测可以是一个非常有用的工具,，我们希望研究界真正对这个问题投入更多关注,” Wilson说。
“There is a huge bucket of child safety concerns with AI, and these are real concerns that need to be addressed. A lot of children are being harmed by AI deepfakes. We’ve shown that Gaussian probing can be a very useful tool, and we hope the research community really pours more attention into this problem,” Wilson says.
重要的是, 他们的技术是可扩展的，并且实施起来相对便宜。由于每月都会在线发布数以千计的模型变体,，可扩展性是帮助审核员在有害改编广泛分发之前消除它们的关键。
Importantly, their technique is scalable and would be relatively inexpensive to implement. Since thousands of model variations are published online every month, scalability is key to help auditors remove harmful adaptations before they are widely distributed.
高斯探测也比其他一些审计技术更强大,，因为不法分子需要仔细改变基本模型的内部工作原理以避免检测。
Gaussian probing is also more robust than some other auditing techniques, since a nefarious actor would need to carefully alter the inner workings of the base model to avoid detection.
在未来,，研究人员希望在更大的模型变体集上评估他们的技术，并探索高斯探测是否可以在基础模型适应之前检测到它们的有害功能。
In the future, the researchers want to evaluate their technique on a larger set of model variations and explore whether Gaussian probing can detect harmful capabilities in base models before they are adapted.
“Now we have a technological approach to partially address this concern. So much effort was poured into this collaboration, which enabled us to tackle a really hard problem that is harming so many children, nationally and around the world. Hopefully, we can have a transformative impact in this area,” Ghassemi says.
这项工作得到 Bridgewater AIA 实验室研究奖学金, 部分的支持。
This work was supported, in part, by the Bridgewater AIA Labs Research Fellowship.