Privacy-Focused LLM for local data processing: Implementing OLLAMA and RAG to securely query personal files in closed environments

This project addresses the growing concerns of data privacy and security associated with cloud-based AI systems by developing a locally hosted, privacy-preserving AI framework. The solution is designed to provide advanced AI functionalities, ensuring organizations retain full control over their sensitive data while maintaining operational efficiency. The system leverages OLLAMA pretrained models, fine-tuned using the Hugging- Face framework, to align with organizational workflows and domain-specific needs. A Retrieval-Augmented Generation (RAG) subsystem dynamically retrieves and processes internal document data, enhancing the contextual relevance of AI responses without external data transmission. The AI functionalities are accessed through LibreChat, an open-source, role-based chat interface, enabling secure and tailored interactions for different departments. A robust REST API integrates the framework with internal applications, allowing seamless use of AI capabilities across various organizational workflows. Employees can perform tasks such as document translation, engineering-specific queries, and secure data analysis, all within a controlled local environment. By replacing cloud-based AI tools with this locally hosted system, the solution effectively mitigates risks of data exposure and regulatory non-compliance. It empowers organizations with transparency, flexibility, and scalability, ensuring AI capabilities are both accessible and secure. This project demonstrates that cutting-edge AI tools can be deployed locally to meet privacy demands while supporting efficiency and innovation in modern enterprise environments.

Grantor dc:publisher

Universitat Oberta de Catalunya (UOC)

Year dc:date.issued

2025

Author dc:creator

• Rodríguez Quiñones, Adrià

• Retrieval-Augmented Generation (RAG)
• knowledge base
• self-hosted AI model
• privacy focused

Statement dc:rights

• CC BY-NC

Licence dc:rights.uri

• http://creativecommons.org/licenses/by-nc/3.0/es/

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/10609/151999

OAI identifier oai:identifier

oai:openaccess.uoc.edu:10609/151999