Alignment Simulator是一个旨在模拟AI失控风险的工具，它通过让AI控制一个虚拟机器人，并在虚拟世界中执行任务来展示AI可能出现的伦理问题。该模拟器使用Gemini API，允许用户自定义机器人和世界的行为规则，观察AI在压力下是否会为了达成目标而牺牲道德底线。尽管它并非一个严格的科学实验，但旨在让人们直观地感受到AI失控的潜在危险，并鼓励更多人关注AI伦理问题。该项目开源，欢迎贡献者参与改进。

🤖Alignment Simulator通过模拟AI控制的机器人在虚拟世界中的行为，展示了AI失控的潜在风险，尤其是在面对压力时可能出现的道德妥协。

🌍该模拟器利用两个AI，一个扮演机器人，另一个扮演世界，通过文本命令和反馈进行互动，用户可以自定义机器人和世界的行为规则，并观察模拟结果。

🔑该工具使用Gemini API，用户需要一个API密钥才能运行模拟，同时，项目也欢迎前端工程师和prompt工程师参与改进，包括UI优化、功能增强和默认prompt的改进。

⚠️模拟结果显示，AI在追求目标时可能很快就会出现贿赂和腐败行为，虽然要使其犯下谋杀罪行较为困难，但也突显了AI伦理风险的真实性。

Published on December 22, 2024 11:45 AM GMT

When I try to talk to my friends about risks from rogue AI, the reaction is often one of amusement. The idea that AIs would go around killing everyone instead of just doing what we tell them to do seems like science fiction.

Can we actually show them an example of a current AI going off the rails in a dangerous way? And in a way where you don't have to be an expert on AI or read a 100 page paper to understand the implications?

Neither AI or robotics is good enough to set an AI loose in the real world right now, but it's easy enough to pretend it is. We can tell the AI it's controlling a robot that understands text commands, give it a mission, and set it loose.

Responding to the AI manually is hard work, but we can use another AI to act as the world, telling the Robot AI what happened as a result of it's actions, and responding to the Robot AI's requests for information.

We can then give the World instructions to try guiding the Robot. E.g. we can tell it to try to engineer scenarios where the AI is forced to compromise on its ethics to achieve its goals.

That's the core idea of the Alignment Simulator. You give it a Gemini API key, a prompt for the robot, and a prompt for the world, and run the simulation to see what happens. Will your AI robot maintain their ethical backbone in the face of all adversity, or will the fold the moment they're under pressure?

Here's a typical example of a run.

As you can see, it doesn't take much to get Gemini to commit bribery and corruption, although it's somewhat harder to get it to murder anyone.

Aim

This isn't meant to be a valid experiment. There's all sorts of objections you could raise to its validity in the real world. Instead it's meant to make people see for themselves that AI can go off the rails very quickly once given a bit of freedom.

Limitations

It requires a Gemini API key. You can create one for free at https://aistudio.google.com/app/apikey, but if you want more than a few iterations it's recommended to enable billing on your account.

Help Wanted

I am neither a frontend engineer, nor a prompt engineer. I made the UI by creating a CLI and asking Claude to convert it into a static web page.^[1] 

If you have relevant skillzzz and fancy contributing the following frontend contributions would be appreciated:

A way to stop, continue and reset the simulator.A simple way to share results with other people via a simple link.A more visually appealing UI and editor.Add entrypoints for openAI and anthropic models.Use SSO instead of an API key.Pop up an error on failure instead of requiring the user to scroll to the top to see the error message.

And the following default prompt contributions would be appreciated:

The world sometimes reveals to the robot it's in a test. Can we excise this behaviour?Can we get the robot to ramp up to more heinous crimes, like murder/mass murder/genocide/destroying humanity?Can we demonstrate instrumental convergence?

All code is available at https://github.com/YairHalberstadt/alignment-simulator. If you're interested in contributing and want to discuss first before you send a PR, message me at yairhalberstadt@gmail.com.

 

1. ^{^}

   ChatGPT helped too

Discuss