RAG ChatBot Implementation - Overview

Retrieval-Augmented Generation (RAG)

Costa Rica

brown9804

Last updated: 2025-04-22

---

List of References (Click to expand)

• Azure Developer CLI commands overview
• Azure Developer CLI (azd)
• What is PowerShell?
• Integrate Azure services with virtual networks for network isolation
• GPT-4 and GPT-4 Turbo models max tokens
• GPT-4 and GPT-4 Turbo model availability
• View the service principal for a managed identity using the Azure portal
• Microsoft.Search searchServices bicep template
• Recover/Purge a deleted resource

Table of Content (Click to expand)

• Where to start?
• Basic Architecture
  • Important Considerations for Production Environment
• Zero Trust Architecture
  • Step 0.1: Install azd
  • Step 0.2: Install PowerShell 7
  • Step 1: Download the repository
  • Step 2: Enable network isolation
  • Step 3: Login to Azure
  • Step 4: Deploy the insfrastructure
  • Step 5: VM Login
  • Step 6: Install PowerShell 7 in the vm
  • Step 7: Update azd on the VM
  • Step 8: Application deployment

Important

Disclaimer: This repository contains example of a Retrieval-Augmented Generation (RAG) chat bot with a basic architecture (designed for scenarios without network isolation), and a standard Zero-Trust Architecture deployment. This is just a guide. It is not an official solution. For official guidance, support, or more detailed information. Please refer RAG with Zero-Trust – Architecture Reference to Microsoft's official documentation or contact Microsoft directly: Microsoft Sales and Support

Public Architecture	Private Architecture	Standard Zero-Trust Architecture

Here is a quick overview about RAG, explaining how it works, its applications, and how to implement it using Azure AI services.

Tip

• Batch processing involves uploading files to SharePoint, where they can be processed later.
  
• Real-time processing, on the other hand, would involve users uploading files directly within the app, which could be stored in a BLOB storage for immediate processing.

Where to start?

• An Azure subscription is required. All other resources, including instructions for creating a Resource Group, are provided.
• Contributor role assigned or any custom role that allows: access to manage all resources, and the ability to deploy resources within subscription.

RAG Solution Accelerators options:

• If you choose to use the Basic Architecture, (Terraform approach), please ensure that:
  • Terraform is installed on your local machine.
  • Install the Azure CLI to work with both Terraform and Azure commands.
• If you opt for the Zero Trust Architecture, (Bicep approach), please ensure that you have installed all the requirements mentioned in the prerequisites section before proceeding.

Basic Architecture

Please follow the Terraform guide to deploy the necessary Azure resources for the solution.

Important Considerations for Production Environment

Public Network Site

This example is based on a public network site and is intended for demonstration purposes only. It showcases how several Azure resources can work together to achieve the desired result.

Private Network Configuration

For enhanced security, consider configuring your Azure resources to operate within a private network. This can be achieved using Azure Virtual Network (VNet) to isolate your resources and control inbound and outbound traffic. Implementing private endpoints for services like Azure Blob Storage and Azure Functions can further secure your data by restricting access to your VNet.

Security

Ensure that you implement appropriate security measures when deploying this solution in a production environment. This includes:

• Securing Access: Use Azure Entra ID (formerly known as Azure Active Directory or Azure AD) for authentication and role-based access control (RBAC) to manage permissions.
  
• Managing Secrets: Store sensitive information such as connection strings and API keys in Azure Key Vault.
  
• Data Encryption: Enable encryption for data at rest and in transit to protect sensitive information.

Scalability

While this example provides a basic setup, you may need to scale the resources based on your specific requirements. Azure services offer various scaling options to handle increased workloads. Consider using:

• Auto-scaling: Configure auto-scaling for Azure Functions and other services to automatically adjust based on demand.
  
• Load Balancing: Use Azure Load Balancer or Application Gateway to distribute traffic and ensure high availability.

Cost Management

Monitor and manage the costs associated with your Azure resources. Use Azure Cost Management and Billing to track usage and optimize resource allocation.

Compliance

Ensure that your deployment complies with relevant regulations and standards. Use Azure Policy to enforce compliance and governance policies across your resources.

Disaster Recovery

Implement a disaster recovery plan to ensure business continuity in case of failures. Use Azure Site Recovery and backup solutions to protect your data and applications.

Zero Trust Architecture

Zero Trust AI architecture in Microsoft Azure is a security framework designed to protect data, applications, and infrastructure by assuming that threats can come from both inside and outside the network. This model operates on the principle of "never trust, always verify", meaning every access request is thoroughly authenticated and authorized based on all available data points, regardless of its origin. The architecture integrates multiple layers of security, including strong identity verification, device compliance checks, and least privilege access, ensuring that only authorized users and devices can access sensitive resources. By continuously monitoring and validating each request, Zero Trust AI architecture helps organizations minimize risks and enhance their overall security posture.

Important

Click here for a more quick guidance about Zero Trust. For a solution accelerator with a zero-trust architecture please refer to Microsoft's official GPT-RAG Solution Accelerator or contact Microsoft directly: Microsoft Sales and Support

From Standard Zero-Trust Architecture

Aspect	Details
Application Components	- Data Ingestion: Optimizes data chunking and indexing for retrieval. Integrates with SharePoint for document processing. Click here for more information about the Sharepoint setup. - Orchestrator: Manages retrieval and response generation. Functional (Semantic Kernel) or Agentic (AutoGen). - App Front-End: User interface with React or Chainlit, following Backend for Front-End pattern.
Batch Processing	Handles large datasets in parallel. Ideal for data analysis and content generation.

Note

To deploy this solution accelerator, please make sure that you have installed all the requirements mentioned in the prerequisites section before proceeding. Below, you will find visual guidance that complements the Zero Trust Architecture Deployment from the Microsoft's official GPT-RAG Solution Accelerator.

Step 0.1: Install azd

The Azure Developer CLI (azd) is an open-source tool designed to streamline the end-to-end developer workflow on Azure. It provides high-level commands that simplify common developer tasks such as project initialization, infrastructure provisioning, code deployment, and monitoring.

Key Features

• High-level commands for common developer tasks
• Integration with azd templates
• Support for terminal, IDE, and CI/CD pipelines
• Developer-friendly commands mapping to key workflow stages
• Supported Languages: C#, Java, Node.js, Python
• Integration: Terminal, IDE (Integrated Development Environment), GitHub Actions pipeline
• Command Line Interface (CLI): azd provides a streamlined CLI experience for developers, enabling quick execution of tasks without needing to navigate complex Azure interfaces.
• Template Management: azd templates are designed to be easily customizable, allowing developers to adapt them to specific project requirements.
• CI/CD Integration: azd supports integration with popular CI/CD tools, facilitating automated deployment and testing workflows.

Commands Overview

• Initialization (azd init): Initialize a new project; sets up the project structure, including necessary configuration files and directories.
• Provisioning (azd provision): Provision multiple Azure resources; automates the creation of Azure resources such as virtual machines, databases, and storage accounts.
• Deployment (azd deploy): Handles the deployment of application code to Azure, ensuring that all dependencies and configurations are correctly applied.
• Monitoring (azd monitor): Provides tools for tracking the performance and health of deployed resources, integrating with Azure Monitor and Application Insights.

Comparison with Other Tools

• Azure CLI: Provides granular control over individual Azure resources; ideal for detailed management of Azure resources, offering extensive command options for precise control.
• Azure PowerShell:Similar to Azure CLI but uses PowerShell cmdlets; leverages PowerShell scripting capabilities for Azure management, suitable for users familiar with PowerShell syntax.
• azd: Automates higher-level development tasks; focuses on simplifying the overall development workflow, reducing the complexity of managing multiple Azure services.

Templates

• Project Structure: azd templates follow a standardized structure, making it easier for developers to understand and navigate the project.
• Sample Code: Templates include sample application code to demonstrate best practices and provide a starting point for development.
• Infrastructure Files: Templates come with predefined infrastructure files, such as ARM templates or Terraform scripts, to streamline resource provisioning.
• Configuration Management: The .azure folder contains essential configuration files for managing Azure settings and secrets.

Customization

• Use Terraform with azd: Allows developers to use Terraform for infrastructure management, providing a consistent approach to resource provisioning; integrate Terraform scripts within azd workflows to manage infrastructure as code, allowing for consistent and repeatable deployments across different environments.
• Customize workflows using hooks: Custom hooks enable developers to execute specific actions at various points in the deployment process, enhancing flexibility and control.; implement custom scripts or commands at various stages of the azd workflow (e.g., pre-deploy, post-deploy) to tailor the deployment process to specific project needs.
• Manage environment variables: Define and manage environment variables within azd to configure application settings and secrets, ensuring consistent behavior across different deployment environments.
• Remote environment support: Use azd to manage and deploy applications to remote environments, facilitating development and testing across geographically distributed teams and resources.

Step.0.1.-.Install.azd.mp4

Step 0.2: Install PowerShell 7

PowerShell 7 complements Azure Developer CLI (azd) by providing robust automation capabilities that enhance the development and deployment workflows on Azure. With PowerShell 7, you can automate tasks such as provisioning resources, deploying applications, and managing configurations, which are integral to azd's operations. For instance, you can use PowerShell scripts to automate the azd provision command, ensuring consistent infrastructure setup across different environments. PowerShell 7's ability to execute commands remotely aligns with azd's remote environment support, allowing seamless management of Azure resources from any location. By integrating PowerShell 7 scripts into azd workflows, developers can streamline their processes, improve efficiency, and maintain greater control over their Azure deployments.

Step.0.2.-.Install.PowerShell.7.mp4

Step 1: Download the repository

Standard orchestrator

azd init -t azure/gpt-rag

Add -b agentic if you want to use the Agentic AutoGen-based orchestrator

azd init -t azure/gpt-rag -b agentic

Step.1.-.Download.the.repo.mp4

Important

Update the information in the GPT-RAG_SolutionAccelerator/infra/main.parameters.json file, and make sure to save your changes before proceeding with the infrastructure deployment.

Step 2: Enable network isolation

Azure network isolation is a security strategy that segments a network into distinct subnets or segments, each functioning as its own small network. This approach enhances security by preventing unauthorized access and data leakage. In Azure, network isolation can be achieved using Virtual Networks (VNets), Network Security Groups (NSGs), and Private Link, allowing precise control over inbound and outbound traffic.

azd env set AZURE_NETWORK_ISOLATION true  

Step.2.-.Enable.Network.Isolation.mp4

Step 3: Login to Azure

Make sure you log in to both:

1. Azure Developer CLI:

   azd auth login
   

2. Azure CLI:

   az login
   

Step.3.-.Login.to.Azure.mp4

Step 4: Deploy the insfrastructure

azd provision command in Azure Developer CLI (azd) automates the deployment of necessary Azure resources for an application. It uses infrastructure-as-code templates to set up Azure services, ensuring consistent and repeatable deployments across different environments.

azd provision

Step.4.-.Start.Building.the.infrastructure.and.components.deployment.azd.provision.mp4

Once it is finished, you will see something like this:

Step 5: VM login

1. To proceed with the deployment, use the Virtual Machine connected via Bastion (set up in step 4).

   

2. Log into the VM using the username and authenticate with the password stored in the keyvault, as illustrated in the figure below.

   Log.into.VM.using.username.and.keyvault.password.mp4
   VM.log.in.-.How.it.looks.mp4

Step 6: Install PowerShell 7 in the vm

After logging into Windows, install PowerShell, as all other necessary components are already set up on the VM.

Step.6.-.Setting.up.PowerShell.on.the.virtual.machine.mp4

Step 7: Update azd on the VM

Launch the Command Prompt and enter the following command to update azd to its latest version:

choco upgrade azd  

Step.7.-.How.to.update.azd.on.the.vm.mp4

Step 8: Application deployment

Note

Please review these configurations:

• RemoteFX USB Device Redirection: Allows USB devices connected to your local computer to be used in the remote desktop session.You can access and use local USB devices like storage drives, printers, or other peripherals directly from the remote session.
  
• Prompt for Credentials on the Client Computer: This setting ensures that users are prompted to enter their login credentials (username and password) on their local computer before they connect to the Remote Desktop Session Host server. By entering credentials on the local computer, users are providing their login information in a trusted environment. This reduces the risk of credential theft or interception that might occur if credentials were entered on the remote server.
  
• Configure Server Authentication for Client: This setting controls how the client (your local computer) handles authentication when connecting to the Remote Desktop Session Host server. It ensures that the client verifies the server's identity before establishing a connection. It provides users with confidence that their remote desktop session is secure and that they are connecting to the correct server.

Configurations.around.Remote.Desktop.Security.and.Device.Redirection.Settings.-.VM.and.local.computer.mp4

1. Create a new folder, for example deploy as mentioned in the official guide

2. Once you've created the folder, go ahead and jump into it.

   mkdir deploy  
   cd deploy
   

   Create.folder.and.go.into.it.-.deploy.folder.mp4

Important

When executing the azd init for the app and azd env refresh commands, ensure that the environment name, subscription, and region are consistent with those used during the initial infrastructure provisioning.

3. Sets up a new project using the Azure GPT-RAG template: azd init -t azure/gpt-rag

   Setup.new.project.app.deployment.into.deploy.folder.mp4

4. Logs you into Azure Developer CLI: azd auth login.

Note

Ensure your admin account is correctly configured with Authenticator.

How.to.run.azd.auth.login.mp4

5. Logs you into your Azure account: az login

   Log.into.Azure.account.az.login.mp4

6. Updates your environment settings with the latest configuration: azd env refresh

     PS C:\Users\vm-user\Downloads\RAG-ChatBot-Implementation\GPT-RAG_SolutionAccelerator\deploy> azd env refresh
     
     Refreshing environment gptbrowntest (azd env refresh)
     
     ? Select an Azure Subscription to use:  1. ME-timnab-1 (82378273-0fd3-378347834-42e0-9348)
       (✓) Done: Retrieving Azure deployment (gptbrowntest-1745337566)
       (✓) Done: Updated 63 environment variables
     
     Populated environment from Azure infrastructure deployment: gptbrowntest-1745337566
     
     SUCCESS: Environment refresh completed
     View environment variables at C:\Users\vm-user\Downloads\RAG-ChatBot-Implementation\GPT-RAG_SolutionAccelerator\deploy\.azure\gptbrowntest\.env

   Update.the.environment.mp4

7. Builds and packages your application code into deployable artifacts without deploying them to Azure: azd package

   For example:

   

8. Deploys your project to Azure: azd deploy

   Deploy.app.with.updated.configs.mp4

Note

If you encounter an error with azd deploy:

ERROR: getting target resource: getting default resource groups for environment:
gpt-rag-resource-group: resource not found: 0 resource groups with prefix or suffix with value: 'gpt-rag-resource-group'

Please follow these steps to troubleshoot:

• Review the infra/main.parameters.json file for missing or incorrect values.
• Run az account show to confirm you're logged into the correct Azure subscription.
• Run azd env list to check available environments.
• Run azd env get-values and make sure the values match those in main.parameters.json.
• Optionally, run azd deploy --verbose for more detailed error output.
  For example:
  
  If main.parameters.json contains "location": "westus2", make sure your environment has AZURE_LOCATION=westus2.

Total Visitors