In recent years, large language model (LLM)–based agents have garnered widespread attention across various fields. Their impressive capabilities, such as natural language communication,21,23 instruction following,26,28 and task execution,22,38 have the potential to expand both the format of information carriers and the way in which information is exchanged. LLM-based agents can now evolve into domain experts, becoming novel information carriers with domain-specific knowledge.1,28 For example, a Travel Agent can retain travel-related information within its parameters. LLM-based agents are also showcasing a new form of information exchange, facilitating more intuitive and natural interactions with users through dialogue and task execution.24,34 Figure 1 shows an example of these capabilities, in which users engage in dialogue with a Travel Agent to obtain information and complete their travel plans.
Along with the increase in LLM-based agents in various domains, agent platforms (for example, GPTsa) represent a new kind of information system, featuring agent-oriented information gathering, storage, and exchange. To accommodate agent properties such as interactivity, intelligence, and proactiveness,28,34 the infrastructure of information systems must be expanded to support information processing at the agent level. A cornerstone within this infrastructure is the recommender system, which greatly affects how information flows in the system in terms of efficiency, user experience, and many other factors. It is essential, therefore, to envision how a recommender system can function on an LLM-based agent platform.
Toward this end, we propose the Rec4Agentverse, a novel recommendation paradigm for an LLM-based agent platform. As shown in Figure 2, the Rec4Agentverse includes two key concepts: the Agent Recommender and Item Agents. Item Agents are LLM-based agents that are treated as items in the recommender system, while the Agent Recommender is employed to recommend Item Agents to users. In contrast to items in traditional recommender systems, Item Agents in the Rec4Agentverse are interactive, intelligent, and proactive, enabling them and the Agent Recommender to collaborate and share user information,b thereby facilitating personalized information delivery. For example, once a Travel Agent is recommended to a user, it can continuously discern the user’s preferences around travel during their interaction and convey these preferences back to the Agent Recommender.
In this article, we explore the preliminary instantiation of the Rec4Agentverse paradigm, showcasing its significant application potential. We also introduce possible application scenarios as well as related issues and challenges, inspiring future exploration.
The Rec4Agentverse Paradigm
Here, we provide an overview of the Rec4Agentverse, first explaining its different parts. Then we take a look at the three stages of the Rec4Agentverse from an information-flow perspective. Finally, we suggest potential applications, explore pertinent research topics, and discuss potential challenges and risks.
Roles of the Rec4Agentverse. The paradigm consists of three roles: the user, the Agent Recommender, and Item Agents (Figure 3). Similar to its role in traditional recommender systems, the user interacts with both Item Agents and the Agent Recommender and provides feedback. Therefore, our primary focus will be on concepts that differ significantly from traditional recommender systems, namely Item Agents and the Agent Recommender.
Item Agents. Item Agents are the most distinct aspect of the Rec4Agentverse paradigm. Unlike items in traditional recommendation systems, items in the Rec4Agentverse paradigm are LLM-based agents. As illustrated in Figure 3, Item Agents not only interact with users but also collaborate with the Agent Recommender and other Item Agents. The creation process of Item Agents varies, and can involve training with domain-specific data or directly constructing them through prompts. They can be generated automatically by the LLM-based agent platform, created by users, or collaboratively created by both users and the platform.
The Agent Recommender. The Agent Recommender recommends LLM-based agents to users. Its function is similar to that of traditional recommender systems, which infer user preferences based on user information (for example, attributes and behaviors) to recommend new items. Unlike traditional systems, however, the recommended items in the Agent Recommender are LLM-based agents, which have distinctive characteristics such as strong interactivity.
The Agent Recommender is able to exchange information and collaborate with other parts of the Rec4Agentverse. As illustrated in Figure 3, the Agent Recommender not only directly interacts with users but also interacts with Item Agents, issuing commands or obtaining new user feedback from them.
Three stages of the Rec4Agentverse. We will now discuss three key stages of the paradigm from an information-flow perspective (Figure 3). In addition to the interaction between users and the system found in traditional recommender systems, the Rec4Agentverse also takes into account the profound interaction between users and Item Agents, the collaboration between Item Agents and the Agent Recommender, and the collaboration between Item Agents themselves. This formulation encompasses three collaboration scenarios, which we envision as paralleling the future development path of the Rec4Agentverse.
Stage 1: User-agent interaction. During the initial stage, in addition to interacting with the Agent Recommender, the user also interacts with Item Agents. This interactive format is similar to that of traditional recommendations. On LLM-based agent platforms such as GPTs, the Rec4Agentverse may generate or retrieve LLM-based agents according to explicit user instructions and implicit user behaviors. Users can interact with the LLM-based agent to exchange information; however, this does not fully unleash the potential of LLM-based agents, as Item Agents can also collaborate with other roles in the recommender system to further enrich the information flow.
Stage 2: Agent-recommender collaboration. In this stage, Item Agents collaborate with the Agent Recommender to provide information services to users. Different from items in traditional recommender systems, Item Agents can deeply collaborate with the Agent Recommender by forwarding user information to and receiving user information from the Agent Recommender. For example, Item Agents can share the user preferences they collect with the Agent Recommender, allowing the latter to provide more personalized recommendations. Similarly, Item Agents can also receive new instructions from the Agent Recommender. The collected personalized information from users and instructions from the Agent Recommender can be used to update Item Agents for evolution (for example, prompt updates), helping Item Agents understand user preferences and provide better information services.
Stage 3: Agent collaboration. An Item Agent can collaborate with other Item Agents with different domain knowledge to provide diverse information services to users. A simple example would be when a user mentions some niche thing that an Item Agent does not know about. The Item Agent can put forward a request to the Agent Recommender to recommend a new Item Agent to assist. Then the two agents can collaborate to fulfill the user’s information needs or execute tasks. Beyond that, this stage leaves considerable room for imagination. For example, the recommended new Item Agent can also interact with users directly or with the Agent Recommender. Further, if multiple Item Agents are recommended, these Item Agents can also work together to better complete the user’s instructions through brainstorming or round-table meetings.
Application domains. The Rec4Agentverse paradigm can contain Item Agents from various domains, which could originate from third-party client developers or be directly created by the Agent Recommender. To demonstrate the Rec4Agentverse’s versatility, we provide a few examples from representative domains:
Travel Agents. Travel Agents help users plan and book trips. When a user expresses interest in a destination, the Agent Recommender suggests a Travel Agent with the relevant expertise. The user can then work with the Travel Agent to create customized itineraries. The Travel Agent gathers user data through interactions or by accessing the relevant records to refine their recommendation capabilities. Moreover, by collaborating with other agents, Travel Agents can gain broader insights into user preferences, leading to more flexible and tailored travel plans.
Fashion Agents. Fashion Agents assist users in discovering their preferred fashion styles and by recommending fashion items. Similar to Travel Agents, Fashion Agents can have conversations with users or interact with the Agent Recommender to gather users’ fashion preferences.
Sports Agents. Sports Agents recommend suitable exercise plans by engaging with users, the Agent Recommender, and other Item Agents to collect user preferences. For example, they can use information about a user’s physical condition obtained from Travel Agents to create suitable exercise plans.
Potential research topics. The Rec4Agentverse offers numerous valuable research directions, several of which we highlight here.
Evaluation. A crucial problem is how to evaluate the recommendation performance of the Rec4Agentverse. Traditional recommendation datasets struggle to adapt to the Rec4Agentverse, since Item Agents are quite different from previous items in the recommendation dataset. And existing evaluation metrics for recommendation, such as NDCG, Recall, AUC, and MSE,35 struggle to measure the user’s satisfaction in the Agent Recommender since Item Agents may involve multi-round interaction. Moreover, the Agent Recommender may generate a new Item Agent for users or Item Agents may upgrade based on user feedback, requiring evaluation that goes beyond merely tracking the user’s implicit feedback, such as interaction numbers, to quantify incremental performance. A potential solution to these problems involves using user feedback from online interactions with the Rec4Agentverse as the gold standard for evaluating its recommendation performance. However, online evaluation can be costly. A more feasible research direction is to leverage collected interaction data and LLMs to build an agent or a reward model that simulates users,29,36 thereby enabling us to assess the Rec4Agentverse’s recommendation performance.
Preference modeling. The Rec4Agentverse is unique in terms of its interaction data format and interaction types. It emphasizes language features in its data format, whereas traditional systems collect user data numerically, such as with click-through rates and dwell time. Integrating both numerical and language-based user feedback for modeling is a key challenge. Also, interactions among the different roles (the Agent Recommender, Item Agents, and users) in the Rec4Agentverse are more complex. Beyond traditional user-recommender system interactions, there may be interactions between Item Agents and the Agent Recommender, as well as among the Item Agents themselves. Effectively integrating these diverse interactions for user modeling remains a challenge. For the data format issue, one approach is to consider collaboratively using language-based and traditional recommendation models for user modeling to understand heterogeneous user feedback; for example, designing a tokenizer that can use collaborative information extracted by traditional models.31 With regard to interaction types, one approach is to have agents summarize different types of interactions and design corresponding memory structures to store the user preferences derived from these interactions, thereby enhancing user modeling.
Efficiency and environmental friendliness. The Rec4Agentverse paradigm is based on LLMs, which require significant consumption costs, raising concerns about both efficiency and environmental impact.39 As such, how to mitigate the computation costs of the Rec4Agentverse while maintaining its performance is an important research direction. One thing we can do is to study acceleration technologies such as PagedAttention18 to reduce inference and training costs. We can also deploy some commonly used low-parameter agents at the edge to handle easy queries. Meanwhile, these agents can collaborate with large agents in the cloud to address more complex queries, thereby reducing the frequency of invoking large agents and ultimately lowering the system’s overall costs.
Issues and challenges. Potential issues and challenges of the Rec4Agentverse paradigm include:
Fairness and bias. LLMs may inherently contain social biases and unfair elements.8,19 Therefore, to mitigate the potential risks and negative societal impacts, it is imperative to acknowledge and control the potential unfairness and bias in the recommended Item Agents and the information delivered by them. This can be done by injecting prompts designed to reduce unfairness or bias, or by having the agent check for bias issues during the output stage.
Privacy. Users may inadvertently disclose private information while interacting with LLM-based agents, especially during the agent collaboration stage, where sensitive information might circulate among multiple agents. To address potential user privacy issues, we need to ensure that users have control over their privacy, meaning they should have the right to specify which agents can access their data. This way, different agents have varying levels of access to user data, and untrusted agents are not able to access sensitive information. For highly sensitive user information, we can have agents process this data directly on the user’s device. When collaboration on this sensitive data is required, other agents can also be deployed on the user’s device to facilitate. Following the data-minimization principle,27 sensitive data should be securely deleted once its intended use is complete.
Harmfulness. Item Agents might generate harmful textual responses2 or be manipulated into executing harmful actions, such as fraudulent transactions. To prevent the Rec4Agentverse from doing these things, it is necessary to implement strategies such as establishing an admission mechanism for Item Agents.
Hallucination. LLMs may generate hallucinations, inconsistent with factual knowledge or user inputs,15 which can negatively affect the user experience, particularly in contexts where reliability is paramount. When implementing the Rec4Agentverse, one can adopt techniques like retrieval-augmented generation (RAG)25 or factual knowledge-enhanced decoding7 methods to mitigate the effects of hallucinations, thereby ensuring high-fidelity service delivery.
Discussion
In this section, we contrast the Rec4Agentverse paradigm with existing recommendation paradigms: retrieval-based17 and generative-based.30 Retrieval-based recommendation retrieves passive items (for example, pictures or movies) from a pool of candidates that the user might like and recommends them to the user.17 Generative-based recommendation uses generative models to create passive items according to the user’s preferences and recommends them to the user.30 Since items recommended by the Rec4Agentverse are not passive items but rather Item Agents with unique characteristics such as strong interactivity, our paradigm brings significant changes to recommender systems in areas such as user preference modeling and collaboration mechanisms.
User preference modeling. Beyond merely summarizing user preferences from passively received user interactions on passive items, as is done in conventional paradigms, in our paradigm, both the Agent Recommender and Item Agents can actively acquire information to enhance user preference modeling. In traditional paradigms, the interactive capability of the recommender and passive items are limited, particularly for items such as pictures and movies that cannot engage in verbal communication. Consequently, user preference modeling for these paradigms typically relies on passively received feedback.c In our paradigm, however, both the recommender and items (that is, the Agent Recommender and Item Agents) have the ability to interact with users through dialogue to directly acquire user preference information or collect further feedback for preference refinement, enhancing user preference modeling.
Collaboration mechanisms. Traditional recommender paradigms encounter challenges in actively fostering collaboration between passive items or between passive items and the recommender once a passive item has been recommended. In our paradigm, collaboration between the recommender and items is closer and more extensive. These enhanced collaborations elevate the service quality of both the Agent Recommender and Item Agents. For instance, when a recommended Item Agent falls short in meeting the user’s needs, it can initiate communication with the Agent Recommender or collaborate with other Item Agents to address these shortcomings. Conversely, in traditional paradigms, users often need to turn to the recommender system for another recommendation, perpetuating the iterative process and diminishing user enthusiasm. As another example, the Agent Recommender can enrich the user profile by engaging in conversations with Item Agents that the user has interacted with in the past or is currently engaging with, thereby facilitating more effective recommendations.
Demonstration
In this section, we explore the three stages of the Rec4Agentverse through case studies, focusing on the feasibility and potential formats of the paradigm. We present a case study involving a traveler who uses the Rec4Agentverse throughout their journey, examining how the Agent Recommender and Item Agents work and affect the user experience at each stage. This case study is based on the “gpt-4-32k” API provided by OpenAI.d Due to space constraints, we provide only the essential parts of the case study here, with additional details available at github.e It is important to note that this case study serves as only a preliminary indication of the feasibility of different stages within the Rec4Agentverse, and does not fully encompass all potential applications of the paradigm.
Stage 1: User-agent interaction. In the user-agent interaction stage, Item Agents primarily engage in interactions with the user, facilitating efficient information exchange. To demonstrate, we present a scenario in which a user expresses their desire to travel to Nepal, interacting with the Agent Recommender and the recommended Travel Agent (Figure 4). The user initially seeks assistance from the Agent Recommender to find a Travel Agent. After inquiring about the user’s preferences, the Agent Recommender customizes a Travel Agent specifically tailored to the user’s needs. Then, after determining the user’s interests, this agent devises a comprehensive travel itinerary. Therefore, there are two main information-exchange flows: one between the user and the Agent Recommender and one between the user and Item Agents.
Information flow between the user and the Agent Recommender. As depicted in Figure 4, in addition to passively receiving requests from the user, the Agent Recommender actively engages with the user to improve their recommendations. For instance, after the user expresses a desire to find a Travel Agent through dialogue, the Agent Recommender proactively poses questions to gain more detailed high-level information about the the user’s travel preferences. With additional feedback from the user, the Agent Recommender then provides accurate recommendations for a Travel Agent. This process bears some resemblance to traditional interactive recommendation methods.
Information flow between the user and Item Agents. As illustrated in Figure 4, and in stark contrast to the traditional paradigm, Item Agents can interact directly with the user. In our example, the Travel Agent initially learns about the user’s interest in traveling to Nepal and their request for a travel plan. It then inquires further to uncover more specific preferences, learning about the user’s desire to visit the “Everest Base Camp.” This information exchange allows Item Agents to develop a deeper understanding of the user’s preferences, thereby enhancing their ability to provide tailored services.
Stage 2: Agent-recommender collaboration. In the agent-recommender collaboration stage, there is potential for further information exchange between Item Agents and the Agent Recommender, opening up three promising possibilities: evolution, agent feedback, and proactive. We illustrate these by extending the travel example, depicted in Figure 5.
Evolution. Thanks to their ability to gather information from users and the Agent Recommender, Item Agents can acquire valuable knowledge to evolve their capabilities, helping enhance future services. As shown in Figure 5, Item Agents can evolve by leveraging and summarizing knowledge obtained from the Agent Recommender, which may involve, for instance, improving their prompts. As a result, when the user makes their next request for a trip to a new destination—for example, Switzerland—the system will promptly present a travel itinerary that directly aligns with the user’s personal preferences, taking into account their inclination toward “hiking,” “cultural,” and “natural” experiences. This evolution process enables the continuous tracking of user information, alleviating the burden on users to detail their preferences in future interactions.
Agent feedback. Item Agents can also contribute feedback to enhance the Agent Recommender’s services. In Figure 5, the recommended Travel Agent can provide a summary of the user’s preferences, such as “cultural,” “natural,” and so on, to the Agent Recommender. The Agent Recommender can then absorb this knowledge and improve its future services accordingly. Then, when a new request for a Clothing Agent arises, the Agent Recommender can directly inquire whether the user is interested in outdoor-friendly or culturally significant attire, based on the knowledge obtained from the Travel Agent. Through this information exchange, the Agent Recommender can significantly enhance its services.
Proactive. Here, proactive refers to the ability of Item Agents to autonomously accomplish specific objectives, which can originate from the agent platform itself or aim to better align with user interests. In the example shown in Figure 5, we assume that the Agent Recommender has prior knowledge of the user’s inclination toward eco-friendly options. Therefore, before the user initiates their interaction, the Agent Recommender injects this eco-friendly objective into the recommended Travel Agent. Subsequently, when the user engages with the Travel Agent, it will provide environmentally friendly travel options that fulfill the eco-friendly requirement. This proactive characteristic enhances user satisfaction by tailoring the experience to the user’s specific interests.
Stage 3: Agent collaboration. Compared with the other two stages, the agent collaboration stage allows further exchange of information among Item Agents, enabling them to collaborate and enhance services for users. In the Travel Agent case illustrated in Figure 6, we present an example where multiple agents collaborate to complete the travel-planning process. Here is a step-by-step breakdown of the collaboration process:
The user starts a conversation with the Agent Recommender, expressing the desire to plan a travel tour.
The Agent Recommender suggests a Travel Agent whose goal is to help with travel tour planning.
The user requests the Travel Agent to create a travel itinerary specifically tailored for Nepal.
To acquire the latest information about Nepal, the Travel Agent sends a request to the Agent Recommender for an Item Agent. This Item Agent should be able to provide up-to-date local information on Nepal, which will assist in creating the travel plan.
The Agent Recommender responds by recommending a local agent who is knowledgeable about the current situation in Nepal.
The Travel Agent integrates the current information about Nepal provided by the local agent into the travel itinerary design process to fulfill the user’s needs.
We conclude that by adopting a system of collaborative cooperation,f agents are enabled to communicate effectively and share information with each other. This exchange process significantly enriches their shared knowledge base. As a result, these agents are better equipped to address and cater to a more diverse and comprehensive range of user needs, thereby enhancing overall user satisfaction.
Related Work
In this section, we mainly discuss two types of related work: LLM-based agents and LLM-based agents for recommendation.
LLM-based agents. LLM-based agents have been applied across various domains, demonstrating their versatility in solving complex problems and their strong interactive abilities.28 Previous work showed that individual agents can be optimized for specific tasks5,11 and simulate human behaviors,23 and that collaborative groups of agents can achieve higher levels of intelligence.6,12,33 Researchers were also devoted to exploring the interaction between the agent and its environment to enhance the agent’s capabilities. This encompassed the interaction between the agent and humans to obtain human feedback10 and the interaction between the agent and the physical world through visual/audio modules to acquire additional knowledge.4,9
Distinct from existing vision or survey papers on LLM-based agents,13,28,32,34 our work envisions a novel recommendation paradigm on an LLM-based agent platform. This paradigm leverages LLM-based agents’ unique capabilities, such as interactivity, to make it possible for Item Agents and the Agent Recommender to collaborate closely, thereby facilitating personalized information services.
LLM-based agents for recommendation. Impressed by the power of LLMs, researchers in the recommendation community began to apply LLMs to recommender systems.3,14,20 Researchers then explored using LLM-based agents in the recommendation field. Some researchers focused on LLM-based agents’ ability to solve specific tasks, using them as recommenders to recommend passive items (for example, movies or games).16 Meanwhile, researchers noted the LLM-based agent’s strong interactive ability, which can simulate human behavior. Some studies initialized LLM-based agents with user profiles to simulate interactions between users and traditional recommender systems, thereby measuring recommendation performance.29,36 Furthermore, AgentCF37 attempted to optimize the self-introduction of both the user and passive items by treating them as agents and improving the self-introduction through user interactions with both positive and negative passive items.
The primary distinction between our work and the aforementioned research lies in the fact that previous studies focused on retrieving or generating passive items. These passive items cannot easily interact with the recommender or other passive items. Conversely, here we explore a scenario in which the recommended “items” are actually interactive LLM-based agents, which enables close collaboration and information sharing between the Agent Recommender and Item Agents, thereby facilitating personalized information services for users.
Conclusion
In this article, we examined how unique LLM-based agents can alter the flow and presentation of information, introducing the recommendation paradigm Rec4Agentverse. The Rec4Agentverse contains two elements, Item Agents and the Agent Recommender, and can be developed in three stages, each designed to enhance the interaction and information exchange among users, the Agent Recommender, and Item Agents. We then simulated a user case leveraging the Rec4Agentverse for travel planning, during which we elucidated the unique attributes of each stage and explored the potential of this paradigm. We also delved into applicable fields, potential development directions, and existing risk issues pertaining to the Rec4Agentverse.
Looking ahead, the Rec4Agentverse presents both opportunities and challenges. It highlights how the unique attributes of LLM-based agents, such as interactivity, intelligence, and proactiveness, can profoundly revolutionize recommendation systems on LLM-based agent platforms. At the same time, deploying the Rec4Agentverse in real-world scenarios poses challenges, including issues related to fairness, privacy, evaluation, and efficiency. Moving forward, we aim to explore the practical implementation of the Rec4Agentverse, addressing these challenges while unlocking its potential to enhance personalized information services for users on LLM-based agent platforms.
