Improving RAG with LoRA finetuning for persona text generation

We address the challenge of maintaining consistency in Retrieval-Augmented Generation (RAG) systems for persona text generation when databases are subject to rapid updates and conventional large language model (LLM) fine-tuning is inadequate. We propose an approach that enhances an existing RAG system used for persona-based information retrieval in dialogue agents through the application of Low-Rank Adaptation fine-tuning on synthetic data. We find that this method improves the system's logic and correctness by 5% on SSA scores and ensures that generated content remains more coherent and contextually relevant.