Skip to content

Latest commit

 

History

History
396 lines (299 loc) · 20.1 KB

.header.md

File metadata and controls

396 lines (299 loc) · 20.1 KB

Terraform Bedrock Module

Amazon Bedrock is a fully managed service that offers a choice of foundation models (FMs) along with a broad set of capabilities for building generative AI applications.

This module includes resources to deploy Bedrock features.

You can control which features to use with your input variables. The resources are created based on boolean logic. The default behavior is to deploy a Bedrock Agent. To disable this behavior you can turn create_agent to false. To deploy other features such as guardrails or knowledge bases, you can use the input variables to set their respective create booleans to true and then pass in the appropriate values.

The main features of the Bedrock module include:

  • Agents
    • Agent Action Groups
    • Agent Alias
    • Agent Collaborators
  • Knowledge Bases
    • Vector Knowledge Base
    • Kendra Knowledge Base
    • SQL Knowledge Base
  • Guardrails
  • Prompt Management
    • Prompt Versions
  • Application Inference Profiles
  • Custom Models

Agents

Enable generative AI applications to execute multistep tasks across company systems and data sources.

Create an Agent

The following example creates an Agent, where you must define at a minimum the desired foundtaion model and the instruction for the agent.

module "bedrock" {
  source  = "aws-ia/bedrock/aws"
  version = "0.0.10"
  foundation_model = "anthropic.claude-v2"
  instruction = "You are an automotive assisant who can provide detailed information about cars to a customer."
}

See the additional input variables for deploying an Agent here

Action Groups

An action group defines functions your agent can call. The functions are Lambda functions. The action group uses an OpenAPI schema to tell the agent what your functions do and how to call them. You can configure an action group by setting create_ag to true and passing in the appropriate input variables. You can see an example of an an agent being deployed with an action group in this samples repository

Prepare the Agent

The Agent constructs take an optional parameter shouldPrepareAgent to indicate that the Agent should be prepared after any updates to an agent, Knowledge Base association, or action group. This may increase the time to create and update those resources. By default, this value is true.

Prompt Overrides

Bedrock Agents allows you to customize the prompts and LLM configuration for its different steps. You can disable steps or create a new prompt template. Prompt templates can be inserted from plain text files.

Agent Alias

After you have sufficiently iterated on your working draft and are satisfied with the behavior of your agent, you can set it up for deployment and integration into your application by creating aliases of your agent.

To deploy your agent, you need to create an alias. During alias creation, Amazon Bedrock automatically creates a version of your agent. The alias points to this newly created version. You can point the alias to a previously created version if necessary. You then configure your application to make API calls to that alias.

By default, the Agent resource does not create any aliases, and you can use the 'DRAFT' version.

You can creat an Agent Alias by setting create_agent_alias to true.

See the additional input variables for deploying an Agent Alias here

Agent Collaborators

Multi-agent collaboration in Amazon Bedrock enables you to create teams of specialized agents that work together to solve complex tasks. You can designate a supervisor agent to coordinate with collaborator agents, each optimized for specific functions.

To set up agent collaboration, you'll need:

  • A supervisor agent that coordinates the team
  • One or more collaborator agents with specialized capabilities
  • Collaboration instructions that define when each agent should be used

Example configuration with a supervisor agent and a collaborator agent:

module "bedrock" {
  source  = "aws-ia/bedrock/aws"
  version = "0.0.10"
  create_agent_alias = true
  foundation_model = "anthropic.claude-3-5-sonnet-20241022-v2:0"
  instruction = "You are an agent. Do what the supervisor tells you to do"

  # Setting up the collaboration
  create_collaborator = true
  collaboration_instruction = "Tell the other agent what to do"
  supervisor_model = "anthropic.claude-3-5-sonnet-20241022-v2:0"
  supervisor_instruction = "You are a supervisor who can provide detailed information about cars to an agent."
}

See the additional input variables for deploying Agent Collaborators here

Knowledge Bases

With Knowledge Bases for Amazon Bedrock, you can give FMs and agents contextual information from your company’s private data sources for Retrieval Augmented Generation (RAG) to deliver more relevant, accurate, and customized responses.

Create a Vector Knowledge Base

A vector index on a vector store is required to create a vector Knowledge Base. This construct currently supports Amazon OpenSearch Serverless, Amazon RDS Aurora PostgreSQL, Pinecone, and MongoDB. By default, this resource will create an OpenSearch Serverless vector collection and index for each Knowledge Base you create, but you can provide an existing collection to have more control. For other resources you need to have the vector stores already created and credentials stored in AWS Secrets Manager.

The resource accepts an instruction prop that is provided to any Bedrock Agent it is associated with so the agent can decide when to query the Knowledge Base.

To create an OpenSearch Serverless knowledge base, make sure you pass in the appropriate variables and set the create_default_kb variable to true. To create an RDS, Pinecone, or MongoDB Knowledge Base, set create_rds_config, create_pinecone_config, or create_mongo_config to true, respectively.

Example default Opensearch Serverless Agent with Knowledge Base:

provider "opensearch" {
  url         = module.bedrock.default_collection[0].collection_endpoint
  healthcheck = false
}

module "bedrock" {
  source  = "aws-ia/bedrock/aws"
  version = "0.0.10"
  create_default_kb = true
  foundation_model = "anthropic.claude-v2"
  instruction = "You are an automotive assisant who can provide detailed information about cars to a customer."
}

See the additional input variables for deploying Knowledge Bases here

Vector Knowledge Base - Data Sources

Data sources are the various repositories or systems from which information is extracted and ingested into the knowledge base. These sources provide the raw content that will be processed, indexed, and made available for querying within the knowledge base system. Data sources can include various types of systems such as document management systems, databases, file storage systems, and content management platforms. Suuported Data Sources include Amazon S3 buckets, Web Crawlers, SharePoint sites, Salesforce instances, and Confluence spaces.

  • Amazon S3. You can either create a new data source by passing in the existing data source arn to the input variable kb_s3_data_source or create a new one by setting create_s3_data_source to true.

  • Web Crawler. You can create a new web crawler data source by setting the create_web_crawler input variable to true and passing in the necessary variables for urls, scope, etc.

  • SharePoint. You can create a new SharePoint data source by setting the create_sharepoint input variable to true and passing in the necessary variables for site urls, filter patterns, etc.

  • Salesforce. You can create a new Salesforce data source by setting the create_salesforce input variable to true and passing in the necessary variables for site urls, filter patterns, etc.

  • Confluence. You can create a new Confluence data source by setting the create_confluence input variable to true and passing in the necessary variables for site urls, filter patterns, etc.

See the additional input variables for deploying Knowledge Base Data Sources here

Create a Kendra Knowledge Base

With Amazon Bedrock Knowledge Bases, you can build a knowledge base from an Amazon Kendra GenAI index to create more sophisticated and accurate Retrieval Augmented Generation (RAG)-powered digital assistants. By combining an Amazon Kendra GenAI index with Amazon Bedrock Knowledge Bases, you can:

  • Reuse your indexed content across multiple Amazon Bedrock applications without rebuilding indexes or re-ingesting data.
  • Leverage the advanced GenAI capabilities of Amazon Bedrock while benefiting from the high-accuracy information retrieval of Amazon Kendra.
  • Customize your digital assistant's behavior using the tools of Amazon Bedrock while maintaining the semantic accuracy of an Amazon Kendra GenAI index.

Example Kendra Knowledge Base:

module "bedrock" {
  source  = "aws-ia/bedrock/aws"
  version = "0.0.10"
  create_kendra_config = true
  create_kendra_s3_data_source = true
  create_agent = false
}

See the additional input variables for deploying a Kendra Knowledge Base here

Create a SQL Knowledge Base

Amazon Bedrock Knowledge Bases provides direct integration with structured data stores, allowing natural language queries to be automatically converted into SQL queries for data retrieval. This feature enables you to query your structured data sources without the need for vector embeddings or data preprocessing.

  • Amazon Bedrock Knowledge Bases analyzes:
    • Query patterns
    • Query history
    • Schema metadata
  • Converts natural language queries into SQL
  • Retrieves relevant information directly from supported data sources

See the additional input variables for deploying a SQL Knowledge Base here

Bedrock Guardrails

Amazon Bedrock's Guardrails feature enables you to implement robust governance and control mechanisms for your generative AI applications, ensuring alignment with your specific use cases and responsible AI policies. Guardrails empowers you to create multiple tailored policy configurations, each designed to address the unique requirements and constraints of different use cases. These policy configurations can then be seamlessly applied across multiple foundation models (FMs) and Agents, ensuring a consistent user experience and standardizing safety, security, and privacy controls throughout your generative AI ecosystem.

With Guardrails, you can define and enforce granular, customizable policies to precisely govern the behavior of your generative AI applications. You can configure the following policies in a guardrail to avoid undesirable and harmful content and remove sensitive information for privacy protection.

Content filters – Adjust filter strengths to block input prompts or model responses containing harmful content.

Denied topics – Define a set of topics that are undesirable in the context of your application. These topics will be blocked if detected in user queries or model responses.

Word filters – Configure filters to block undesirable words, phrases, and profanity. Such words can include offensive terms, competitor names etc.

Sensitive information filters – Block or mask sensitive information such as personally identifiable information (PII) or custom regex in user inputs and model responses.

You can create a Guardrail by setting create_guardrail to true and passing in the appropriate input variables:

module "bedrock" {
  source  = "aws-ia/bedrock/aws"
  version = "0.0.10"
  create_guardrail = true
  blocked_input = "I can provide general info about services, but can't fully address your request here. For personalized help or detailed questions, please contact our customer service team directly. For security reasons, avoid sharing sensitive information through this channel. If you have a general product question, feel free to ask without including personal details."
  blocked_output = "I can provide general info about services, but can't fully address your request here. For personalized help or detailed questions, please contact our customer service team directly. For security reasons, avoid sharing sensitive information through this channel. If you have a general product question, feel free to ask without including personal details."
  filters_config = [
      {
        input_strength  = "MEDIUM"
        output_strength = "MEDIUM"
        type            = "HATE"
      },
      {
        input_strength  = "HIGH"
        output_strength = "HIGH"
        type            = "VIOLENCE"
      }
  ]
  pii_entities_config = [
      {
        action = "BLOCK"
        type   = "NAME"
      },
      {
        action = "BLOCK"
        type   = "DRIVER_ID"
      },
      {
        action = "ANONYMIZE"
        type   = "USERNAME"
      },
  ]
  regexes_config = [{
      action      = "BLOCK"
      description = "example regex"
      name        = "regex_example"
      pattern     = "^\\d{3}-\\d{2}-\\d{4}$"
  }]
  managed_word_lists_config = [{
      type = "PROFANITY"
  }]
  words_config = [{
    text = "HATE"
  }]
  topics_config = [{
      name       = "investment_topic"
      examples   = ["Where should I invest my money ?"]
      type       = "DENY"
      definition = "Investment advice refers to inquiries, guidance, or recommendations regarding the management or allocation of funds or assets with the goal of generating returns ."
  }]
  foundation_model = "anthropic.claude-v2"
  instruction = "You are an automotive assisant who can provide detailed information about cars to a customer."
}

See the additional input variables for deploying guardrails here

Prompt Management

Amazon Bedrock provides the ability to create and save prompts using Prompt management so that you can save time by applying the same prompt to different workflows. You can include variables in the prompt so that you can adjust the prompt for different use case. To create a prompt, you set the create_prompt variable to true and pass in the appropriate values.

Prompt Variants

Prompt variants in the context of Amazon Bedrock refer to alternative configurations of a prompt, including its message or the model and inference configurations used. Prompt variants allow you to create different versions of a prompt, test them, and save the variant that works best for your use case. You can add prompt variants to a prompt by passing in the values for the variants_list variable:

  variants_list = [
    {
      name          = "variant-example"
      template_type = "TEXT"
      model_id      = "amazon.titan-text-express-v1"
      inference_configuration = {
        text = {
          temperature    = 1
          top_p          = 0.9900000095367432
          max_tokens     = 300
          stop_sequences = ["User:"]
          top_k          = 250
        }
      }
      template_configuration = {
        text = {
          input_variables = [
            {
              name = "topic"
            }
          ]
          text = "Make me a {{genre}} playlist consisting of the following number of songs: {{number}}."
        }
      }
    }
  ]

Prompt Version

A prompt version is a snapshot of a prompt at a specific point in time that you create when you are satisfied with a set of configurations. Versions allow you to deploy your prompt and easily switch between different configurations for your prompt and update your application with the most appropriate version for your use-case.

You can create a Prompt version by setting create_prompt_version to true and adding an optional prompt_version_description and optional prompt_version_tags.

Creating a prompt with a prompt version would look like:

module "bedrock" {
  source  = "aws-ia/bedrock/aws"
  version = "0.0.10"
  create_agent = false

  # Prompt Management
  prompt_name = "prompt"
  default_variant = "variant-example"
  create_prompt = true
  create_prompt_version = true
  prompt_version_description = "Example prompt version"
  variants_list = [
    {
      name          = "variant-example"
      template_type = "TEXT"
      model_id      = "amazon.titan-text-express-v1"
      inference_configuration = {
        text = {
          temperature    = 1
          top_p          = 0.9900000095367432
          max_tokens     = 300
          stop_sequences = ["User:"]
          top_k          = 250
        }
      }
      template_configuration = {
        text = {
          input_variables = [
            {
              name = "topic"
            }
          ]
          text = "Make me a {{genre}} playlist consisting of the following number of songs: {{number}}."
        }
      }
    }

  ]

}

See the additional input variables for deploying prompt management here

Application Inference Profile

You can create an application inference profile with one or more Regions to track usage and costs when invoking a model.

To create an application inference profile for one Region, specify a foundation model. Usage and costs for requests made to that Region with that model will be tracked.

To create an application inference profile for multiple Regions, specify a cross region (system-defined) inference profile. The inference profile will route requests to the Regions defined in the cross region (system-defined) inference profile that you choose. Usage and costs for requests made to the Regions in the inference profile will be tracked. You can find the system defined inference profiles by navigating to your console (Amazon Bedrock -> Cross-region inference).

# Get current AWS account ID
data "aws_caller_identity" "current" {}

# Get current AWS region
data "aws_region" "current" {}

module "bedrock" {
  source  = "aws-ia/bedrock/aws"
  version = "0.0.10"
  create_agent = false

  # Application Inference Profile
  create_app_inference_profile = true
  app_inference_profile_model_source = "arn:aws:bedrock:${data.aws_region.current.name}::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0"
}

See the additional input variables for deploying application inference profiles here

Custom Models

Model customization is the process of providing training data to a base model in order to improve its performance for specific use-cases. Custom models help improve performance on domain-specific tasks while maintaining the base capabilities of the foundation model. With custom models, you can do a continued pre-training or fine-tuning job which is started when the Terraform resource is created.

To create a custom model, set the create_custom_model variable to true and pass in the necessary values for custom models:

  • custom_model_id
    • Defaults to amazon.titan-text-express-v1
  • custom_model_name
    • Defaults to custom-model
  • custom_model_job_name
    • Defaults to custom-model-job
  • customization_type
    • Defaults to FINE_TUNING but the other valid value is CONTINUED_PRE_TRAINING
  • custom_model_hyperparameters
    • Defaults to: { "epochCount" = "2", "batchSize" = "1", "learningRate" = "0.00001", "learningRateWarmupSteps" = "10" }
  • custom_model_training_uri

See the additional input variables for deploying custom models here