Enhancing Public Contract Code analysis with Graph Retrieval-Augmented Generation

The rapid progression of Generative Artificial Intelligence, together with researchers' increasing attention to the Public Administration (PA), opens up to novel cross-domain applications. Generative AI technologies like Large Language Models (LLMs) can expedite administrative purchasing processes and boost the transparency of the procurement life cycle. However, in a dynamic domain like the PA, updating LLMs training data can prove very prohibitive. Recently emerged Graph Retrieval-Augmented Generation (RAG) solves this data editing limitation, and tackles the lack of global information of traditional RAG techniques. Graph RAG leverages structural information across entities, enabling more comprehensive, context-aware responses. This paper illustrates a preliminary application of Microsoft's GraphRAG in the PA domain, leveraging the latest Italian Public Contract Code corpus version. The experimental setting consists of an interface to let PA domain experts query the model about the Public Contract Code and evaluate the answers' correctness, completeness and fluency. Then, users filled out a satisfaction questionnaire to assess system usability and users' resistance to integrating this tool into their workflow. Results reveal a general users' satisfaction with the system: it achieves a System Usability Score of 82.19 and a Net Promoter Score of 25. Questions for assessing the correctness, completeness and fluency of the answers to users' queries achieve a mean score above 3.70. Finally, results of the survey for assessing the users' resistance - measured in terms of Perceived Value, Switching Benefit, Switching Cost, and Self-efficacy For Change - make clear that users consider this tool beneficial to their way of working.