本文介绍了一种利用大型语言模型 (LLM) 验证 RDF (资源描述框架) 三元组的最新技术，强调了提供可追溯和可验证推理的重要性。RDF 三元组是构成知识图谱 (KG) 的基本要素，它们由描述关系或事实的主语-谓语-宾语语句组成。确保这些断言的正确性对于维护 KG 的可靠性至关重要，尤其是在生物科学等各个行业中其应用不断扩展。

🤔 这项技术旨在克服现有 LLM 的一个固有局限性，即它们无法准确地确定用于生成响应的数据来源。尽管 LLM 是强大的工具，可以基于大量预训练数据生成类似人类的语言，但它们通常难以追溯其生成的文本的精确来源或提供准确的引用。这种缺乏可追溯性引发了关于 LLM 提供的数据真实性的问题，尤其是在精确性至关重要的场景中。

💡 该方法为了解决这个问题，有意避免依赖 LLM 的内部事实知识。相反，它采用了一种更严格的方法，将相关外部文本部分与需要验证的 RDF 三元组进行比较。这些文档是通过网络搜索或从维基百科获取的，确保验证过程基于可以直接引用并追溯到其原始来源的材料。

📊 该团队分享了该方法在生物科学领域的广泛测试结果，该领域以其复杂且高度专业化的主题而闻名。研究人员使用一组称为 BioRED 数据集的生物医学研究语句评估了该方法的有效性。为了考虑潜在的误报，他们评估了来自该数据集的 1,719 个正 RDF 语句以及数量相当的新创建的负断言。结果虽然显示出一定的局限性，但令人鼓舞。该方法在被标记为真时，以 88% 的准确率正确识别了语句。然而，它的召回率为 44%，这意味着它仅识别了所有真命题的 44%，遗漏了相当一部分。

🚀 这些发现表明，虽然该技术在它确实验证的断言方面非常准确，但可能需要进一步的工作来提高它检测所有真语句的能力。相对较低的召回率表明，人类监督仍然是确保验证过程准确性的必要条件。这突出了在获得最佳结果方面，将人类专业知识与 LLM 等自动化技术相结合的重要性。

🌎 该团队还分享了如何在实践中将该策略应用于最大的知识图谱之一，也是最受欢迎的知识图谱之一，维基数据。研究人员使用 SPARQL 查询自动从维基数据中检索需要验证的 RDF 三元组。他们使用建议的方法对这些三元组进行了验证，并根据外部文献对这些语句进行了验证，突出了该方法在广泛应用中的潜力。

🔍 总之，这项研究的结果表明，由于人类注释成本高昂，LLM 在知识图谱中大规模语句验证的传统上具有挑战性的工作中具有潜在的重要性。这种方法通过自动化验证过程并将其锚定在可验证的外部来源，提供了一种可扩展的方法来维护 KG 的准确性和可靠性。人类监督仍然是必要的，尤其是在 LLM 的记忆力不足的情况下。鉴于此，这种方法是利用 LLM 的潜力进行可追溯的知识验证的积极进步。

In recent research, a state-of-the-art technique has been introduced for utilizing Large Language Models (LLMs) to verify RDF (Resource Description Framework) triples, emphasizing the significance of providing traceable and verifiable reasoning. The fundamental building blocks of knowledge graphs (KGs) are RDF triples, which are composed of subject-predicate-object statements that describe relationships or facts. Maintaining the correctness of these claims is essential to upholding KGs’ dependability, particularly as their application grows across a range of industries, including the biosciences.

The intrinsic limitation of existing LLMs, which is their incapacity to accurately pinpoint the source of the data they utilize to create responses, is one of the main issues this approach attempts to solve. Even though LLMs are strong tools that can produce language that is human-like based on enormous volumes of pre-trained data, they frequently have trouble tracing the precise sources of the content they produce or offering accurate citations. Issues concerning the veracity of the data supplied by LLMs are raised by this lack of traceability, especially in situations when precision is crucial.

The suggested approach purposefully avoids depending on the LLM’s internal factual knowledge in order to get around this problem. Rather, it adopts a more stringent method by comparing pertinent sections of external texts with the RDF triples that require verification. These papers are obtained via web searches or from Wikipedia, guaranteeing that the process of verification is based on materials that can be directly cited and tracked back to their original sources.

The team has shared that the approach underwent extensive testing in the biosciences, an area renowned for its intricate and highly specialized subject matter. The researchers assessed the method’s effectiveness using a set of biomedical research statements known as the BioRED dataset. In order to account for potential false positives, they evaluated 1,719 positive RDF statements from the dataset in addition to an equivalent number of freshly created negative assertions. Although the results showed certain limits, they were encouraging. With an accuracy of 88%, the approach correctly identified statements 88% of the time when they were labeled as true. However, with a recall rate of 44%, it only recognized 44% of all true propositions, leaving out a sizable number of them.

These findings imply that although the technique is very accurate in the assertions it does validate, further work may be necessary to increase its capacity to detect all true statements. The comparatively low recall suggests that human supervision is still required to guarantee the accuracy of the verification procedure. This emphasizes how crucial it is to combine human expertise with automated technologies like LLMs in order to get the best results.

The team has also shared how this strategy can be utilized in practice on one of the biggest and most popular knowledge graphs, Wikidata. The researchers automatically retrieved the RDF triples that needed to be verified from Wikidata using a SPARQL query. They verified the statements against outside papers by using the suggested method on these triples, highlighting the method’s potential for widespread use.

In conclusion, this study’s findings point to the potential importance of LLMs in the historically difficult work of large-scale statement verification in knowledge graphs due to the high expense of human annotation. This approach provides a scalable means of preserving the precision and dependability of KGs by automating the verification process and anchoring it in verifiable external sources. Human supervision is still necessary, especially in situations when the LLM’s recollection is poor. In light of this, this method is a positive advancement in leveraging LLMs’ potential for traceable knowledge verification.

gnitive processes.

---

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 50k+ ML SubReddit

FREE AI WEBINAR: ‘SAM 2 for Video: How to Fine-tune On Your Data’ (Wed, Sep 25, 4:00 AM – 4:45 AM EST)

The post Verifying RDF Triples Using LLMs with Traceable Arguments: A Method for Large-Scale Knowledge Graph Validation appeared first on MarkTechPost.