A Comparative Analysis of RAG and Non-RAG Models to Improve Access to Service Provider Information for Older Adults in Mississippi
ORCID
Saei: https://orcid.org/0000-0001-8125-435X; Anreddy: https://orcid.org/0000-0003-3362-1332
MSU Affiliation
James Worth Bagley College of Engineering; Department of Industrial and Systems Engineering
Creation Date
2026-06-01
Abstract
In communities where navigating community healthcare and social services, access to clear and relevant information is more than a convenience, it is a necessity. Motivated by this need, our study explores how intelligent retrieval systems can be designed to better serve users seeking local care resources. We began with a rich directory of 1,553 community resource providers throughout Mississippi, aiming to understand how people might naturally ask for help and what ideal responses would look like. To simulate realistic interactions, we developed ten different query-reference pairs that reflect common but critical health-related concerns, ranging from mental health access to elder care services. We then implemented two information retrieval approaches: one based on Retrieval-Augmented Generation (RAG) and another using a non-RAG baseline. Using BERTScore to evaluate the semantic similarity of generated responses to our curated references, we found the RAG system to be significantly more effective, achieving an F1 score of 0.8723 compared to 0.8221 from the non-RAG model, a relative improvement of 6.1%. These results demonstrate the value of augmenting language models with curated knowledge, especially when precision and relevance matter most. Ultimately, this work highlights how thoughtful system design rooted in real-world needs can bridge the gap between resource and referral landscapes and the people who rely on them.
Publication Date
5-1-2026
Publication Title
Computer Applications in Industry and Engineering
Publisher
Springer
Rights
© 2026 The Author(s), under exclusive license to Springer Nature Switzerland AG
Recommended Citation
Saei, S., Anreddy, S. (2026). A Comparative Analysis of RAG and Non-RAG Models to Improve Access to Service Provider Information for Older Adults in Mississippi. In: Roy, I., Kambhampaty, K.K., Goto, T., Hu, G. (eds) Computer Applications in Industry and Engineering. CAINE 2025. Communications in Computer and Information Science, vol 2740. Springer, Cham. https://doi.org/10.1007/978-3-032-14893-3_5
