GoodAI致力于开发具备持续终身学习能力的智能体，并为此开源了GoodAI LTM基准测试。该基准测试通过长时间的对话，评估智能体长期记忆能力，并通过交织不同任务的信息和问题，模拟真实场景。GoodAI不断改进基准测试，引入新功能和任务，以推动智能体长期记忆能力的发展，并使其更具现实意义。测试注重自然对话，避免浪费资源，旨在帮助智能体更好地理解和处理信息，最终目标是开发出能够自主学习和解决各种问题的通用人工智能。

🤔 GoodAI致力于开发具备持续终身学习能力的智能体，并为此开源了GoodAI LTM基准测试，旨在评估智能体长期记忆能力（LTM）。

🔄 该基准测试通过一个非常长的对话进行，在对话中交织不同任务的信息和问题，以模拟真实场景，并评估智能体是否能记住之前的信息。

🚀 GoodAI LTM团队将基准测试视为一个不断移动的目标，通过引入新的任务和功能，持续提升测试难度，推动智能体长期记忆能力的发展。

💡 GoodAI 团队不断优化基准测试，使其更具现实意义，并尽量减少资源浪费，同时确保对话的自然流畅，让智能体能够更好地理解和处理信息。

🤖 GoodAI的最终目标是开发出能够自主学习和解决各种问题的通用人工智能（AGI），例如“AI People”游戏中的NPC，这些NPC具备长期记忆和不断发展的个性。

At GoodAI, we are committed to developing agents that are capable of continual and life-long learning. As part of our efforts, we have previously open-sourced the GoodAI LTM Benchmark, a suite of tests aimed to evaluate the Long-Term Memory (LTM) abilities of any conversational agent. In this benchmark, all tasks take place as part of one single very long conversation between the agent and our virtual tester. The benchmark interleaves information and probing questions from different tasks, albeit taking special care of weaving them together into a natural conversation.

LTM = Long-Term Memory

As a direct consequence of our research in agents with LTM, the GoodAI LTM Benchmark is in constant evolution. To us it represents an invaluable tool for evaluating our agents and validating our hypotheses. Additionally, it helps us characterize the ways in which the distinct agents fail and therefore it provides us goals to aim for. In the GoodAI LTM team we regard the GoodAI LTM Benchmark as a moving goal post, and by introducing new tasks and features we are continuously pushing that goal post away, because what is a goal post worth if it is easy to reach?

New features

With every new feature, we try to make the GoodAI LTM Benchmark not only more and more challenging, but also more realistic. The thing about benchmarking LTM is that you need your tests to be long, very long. So you either introduce a ton of dummy interactions for the sole sake of filling up the conversation, and accept that all those tokens are wasted resources, or you start interleaving the tasks and weave them into a seamless and natural conversation (like we do). We are always doing our best to minimize the amount of wasted tokens, whilst keeping the conversation natural and making sure that the agent can follow along.

For more details, continue to GoodAI Blog Post