Abstract:
We address the challenge of maintaining consistency in Retrieval-Augmented Generation (RAG) systems for persona text generation when databases are subject to rapid updates and conventional large language model (LLM) fine-tuning is inadequate. We propose an approach that enhances an existing RAG system used for persona-based information retrieval in dialogue agents through the application of Low-Rank Adaptation fine-tuning on synthetic data. We find that this method improves the system's logic and correctness by 5% on SSA scores and ensures that generated content remains more coherent and contextually relevant.
Key words and phrases:retrieval-augmented generation, large language models, fine-tuning.