Retrieval Augmented Generation (RAG) has been effectively used to improve the accuracy of question-answering (Q&A) systems powered by Large Language Models (LLMs) by integrating local knowledge and more up-to-date content. However, traditional RAG methods, including those with re-ranking mechanisms, face challenges when dealing with large, frequently updated data sources or when accessing sources exclusively via APIs, as they require pre-encoding all content into embedding vectors. To address these limitations, we introduce Agent-based Universal RAG (AU-RAG), a novel approach that augments data sources with descriptive metadata, allowing an agent to dynamically search through diverse data pools. This agent-driven system can learn from examples to retrieve and consolidate data from various sources on the fly, functioning as a more flexible and adaptive RAG. We demonstrate AU-RAG’s functionality with a financial analysis example and evaluate its performance using a multi-source QA dataset. The results show that AU-RAG performs comparably to RAG with re-ranking in data retrieval tasks while also demonstrating an enhanced ability to intelligently learn and access new data sources from examples, making it a robust solution for dynamic and complex information environments.