Modernized code to use latest langchain versions and MI

Signed-off-by: Paul Yuknewicz <[email protected]>

Author: paulyuk

README.md

@@ -26,23 +26,28 @@ This sample shows how to take a human prompt as HTTP Get or Post input, calculat
 ### Pre-reqs
 1) [Python 3.8+](https://www.python.org/) 
 2) [Azure Functions Core Tools](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?tabs=v4%2Cmacos%2Ccsharp%2Cportal%2Cbash#install-the-azure-functions-core-tools)
-3) [Azure OpenAPI API key, endpoint, and deployment](https://portal.azure.com) 
-4) Add this `local.settings.json` file to this folder to simplify local development and include Key from step 3
+3) [Azure Developer CLI](https://aka.ms/azd)
+4) Once you have your Azure subscription, run the following in a new terminal window to create Azure OpenAI and other resources needed:
+```bash
+azd provision
+```
+
+Take note of the value of `AZURE_OPENAI_ENDPOINT` which can be found in `./.azure/<env name from azd provision>/.env`.  It will look something like:
+```bash
+AZURE_OPENAI_ENDPOINT="https://cog-<unique string>.openai.azure.com/"
+```
 
-`./local.settings.json`
+5) Add this `local.settings.json` file to the root of the repo folder to simplify local development.  Replace `AZURE_OPENAI_ENDPOINT` with your value from step 4.  Optionally you can choose a different model deployment in `CHAT_MODEL_DEPLOYMENT_NAME`.  This file will be gitignored to protect secrets from committing to your repo, however by default the sample uses Entra identity (user identity and mananaged identity) so it is secretless.  
 ```json
 {
   "IsEncrypted": false,
   "Values": {
     "FUNCTIONS_WORKER_RUNTIME": "python",
     "AzureWebJobsFeatureFlags": "EnableWorkerIndexing",
-    "AzureWebJobsStorage": "",
-    "AZURE_OPENAI_KEY": "...",
-    "AZURE_OPENAI_ENDPOINT": "https://<service_name>.openai.azure.com/",
-    "AZURE_OPENAI_SERVICE": "...",
-    "AZURE_OPENAI_CHATGPT_DEPLOYMENT": "...",
-    "OPENAI_API_VERSION": "2023-05-15",
-    "USE_LANGCHAIN": "True"
+    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
+    "AZURE_OPENAI_ENDPOINT": "https://<your deployment>.openai.azure.com/",
+    "AZURE_OPENAI_CHATGPT_DEPLOYMENT": "chat",
+    "OPENAI_API_VERSION": "2023-05-15"
   }
 }
 ```
@@ -120,3 +125,24 @@ To provision and deploy:
 ```bash
 azd up
 ```
+
+## Source Code
+
+The key code that makes the prompting and completion work is as follows in [function_app.py](function_app.py).  The `/api/ask` function and route expects a prompt to come in the POST body using a standard HTTP Trigger in Python.  Then once the environment variables are set to configure OpenAI and LangChain frameworks, we can leverage favorite aspects of LangChain.  In this simple example we take a prompt, build a better prompt from a template, and then invoke the LLM.  By default the LLM deployment is `gpt-35-turbo` as defined in [./infra/main.parameters.json](./infra/main.parameters.json) but you can experiment with other models.    
+
+```python
+llm = AzureChatOpenAI(
+    deployment_name=AZURE_OPENAI_CHATGPT_DEPLOYMENT,
+    temperature=0.3,
+    openai_api_key=AZURE_OPENAI_KEY
+    )
+llm_prompt = PromptTemplate.from_template(
+    "The following is a conversation with an AI assistant. " +
+    "The assistant is helpful.\n\n" +
+    "A:How can I help you today?\nHuman: {human_prompt}?"
+    )
+formatted_prompt = llm_prompt.format(human_prompt=prompt)
+
+response = llm.invoke(formatted_prompt)
+logging.info(response.content)
+```

function_app.py

@@ -2,12 +2,23 @@
 import logging
 import os
 import openai
-from langchain.prompts import PromptTemplate
-from langchain.llms.openai import AzureOpenAI
+from langchain_core.prompts import PromptTemplate
+from langchain_openai import AzureChatOpenAI
+from azure.identity import DefaultAzureCredential
 
 app = func.FunctionApp()
 
 
+# Use the Entra Id DefaultAzureCredential to get the token
+credential = DefaultAzureCredential()
+# Set the API type to `azure_ad`
+os.environ["OPENAI_API_TYPE"] = "azure_ad"
+# Set the API_KEY to the token from the Azure credential
+os.environ["OPENAI_API_KEY"] = credential.get_token(
+    "https://cognitiveservices.azure.com/.default"
+    ).token
+
+
 @app.function_name(name='ask')
 @app.route(route='ask', auth_level='anonymous', methods=['POST'])
 def main(req):
@@ -18,41 +29,42 @@ def main(req):
             req_body = req.get_json()
         except ValueError:
             raise RuntimeError("prompt data must be set in POST.")
-        else: 
+        else:
             prompt = req_body.get('prompt')
             if not prompt:
                 raise RuntimeError("prompt data must be set in POST.")
 
-    # init OpenAI: Replace these with your own values, either in env vars
-    AZURE_OPENAI_KEY = os.environ.get("AZURE_OPENAI_KEY")
+    # Init OpenAI: configure these using Env Variables
     AZURE_OPENAI_ENDPOINT = os.environ.get("AZURE_OPENAI_ENDPOINT")
+    AZURE_OPENAI_KEY = credential.get_token(
+        "https://cognitiveservices.azure.com/.default"
+        ).token
     AZURE_OPENAI_CHATGPT_DEPLOYMENT = os.environ.get(
         "AZURE_OPENAI_CHATGPT_DEPLOYMENT") or "chat"
-    if 'AZURE_OPENAI_KEY' not in os.environ:
-        raise RuntimeError("No 'AZURE_OPENAI_KEY' env var set.")
+    OPENAI_API_VERSION = os.environ.get(
+        "OPENAI_API_VERSION") or "2023-05-15"
 
     # configure azure openai for langchain and/or llm
     openai.api_key = AZURE_OPENAI_KEY
     openai.api_base = AZURE_OPENAI_ENDPOINT
     openai.api_type = 'azure'
 
     # this may change in the future
-    # set this version in environment variables using OPENAI_API_VERSION
-    openai.api_version = '2023-05-15'
-
-    logging.info('Using Langchain')
+    openai.api_version = OPENAI_API_VERSION
 
-    llm = AzureOpenAI(
+    llm = AzureChatOpenAI(
         deployment_name=AZURE_OPENAI_CHATGPT_DEPLOYMENT,
         temperature=0.3,
         openai_api_key=AZURE_OPENAI_KEY
         )
-    llm_prompt = PromptTemplate(
-        input_variables=["human_prompt"],
-        template="The following is a conversation with an AI assistant. " +
-                 "The assistant is helpful.\n\n" +
-                 "A:How can I help you today?\nHuman: {human_prompt}?",
-    )
-    from langchain.chains import LLMChain
-    chain = LLMChain(llm=llm, prompt=llm_prompt)
-    return chain.run(prompt)
+    llm_prompt = PromptTemplate.from_template(
+        "The following is a conversation with an AI assistant. " +
+        "The assistant is helpful.\n\n" +
+        "A:How can I help you today?\nHuman: {human_prompt}?"
+        )
+    formatted_prompt = llm_prompt.format(human_prompt=prompt)
+
+    response = llm.invoke(formatted_prompt)
+    logging.info(response.content)
+
+    return func.HttpResponse(response.content)

requirements.txt

@@ -3,5 +3,7 @@
 # Manually managing azure-functions-worker may cause unexpected issues
 
 azure-functions
+azure-identity
 openai
-langchain
+langchain-core
+langchain-openai