[repo] docs: added concepts to the getting started docs (#1176)

## Linked issues closes: #734, #731, #784 (issue number) ## Details Added the following concepts: * User Authentication * Action Planner * AI System * Prompt Management system * Moderator * Augmentations * Actions * Turns credits to @corinagum for creating the diagram ## Attestation Checklist - [x] My code follows the style guidelines of this project - I have checked for/fixed spelling, linting, and other errors - I have commented my code for clarity - I have made corresponding changes to the documentation (we use [TypeDoc](https://typedoc.org/) to document our code) - My changes generate no new warnings - I have added tests that validates my changes, and provides sufficient test coverage. I have tested with: - Local testing - E2E testing in Teams - New and existing unit tests pass locally with my changes --------- Co-authored-by: Corina <[email protected]>
microsoft · Jan 19, 2024 · b934b21 · b934b21
1 parent abba48f
commit b934b21
Show file tree

Hide file tree

Showing 18 changed files with 919 additions and 24 deletions.
diff --git a/dotnet/packages/Microsoft.TeamsAI/Microsoft.TeamsAI/AI/AI.cs b/dotnet/packages/Microsoft.TeamsAI/Microsoft.TeamsAI/AI/AI.cs
@@ -138,7 +138,7 @@ public AI<TState> RegisterDefaultAction(string name, IActionHandler<TState> hand
 
         /// <summary>
         /// Import a set of Actions from the given class instance. The functions must have the `Action` attribute.
-        /// Once these functions are imported, the AI module will have access to these functions.
+        /// Once these functions are imported, the AI System will have access to these functions.
         /// </summary>
         /// <param name="instance">Instance of a class containing these functions.</param>
         /// <returns>The current instance object.</returns>

diff --git a/getting-started/CONCEPTS/ACTION-PLANNER.md b/getting-started/CONCEPTS/ACTION-PLANNER.md
@@ -0,0 +1,25 @@
+# Action Planner
+
+The Action Planner is a powerful planner that uses an LLM to generate plans. It can trigger parameterized actions and send text-based responses to the user. It supports the following advanced features:
+
+### [Prompt Management](./PROMPTS.md)
+
+The Action Planner has a built-in prompt management system that supports creating prompt templates as folders in the file system. A prompt template is the prompt text along with all the configurations for completion with the LLM model. Dynamic prompts also support template variables and functions.
+
+### [Augmentations](./AUGMENTATIONS.md)
+Augmentations virtually eliminate the need for prompt engineering. Prompts
+can be configured to use a named augmentation which will be automatically appended to the outgoing
+prompt. Augmentations let the developer specify whether they want to support multi-step plans (sequence),
+or create an AutoGPT style agent (monologue).
+
+### Validations 
+Validators are used to validate the response returned by the LLM and can guarantee
+that the parameters passed to an action match a supplied schema. The validator used is automatically
+selected based on the augmentation being used. Validators also prevent hallucinated action names,
+making it impossible for the LLM to trigger an action that doesn't exist.
+
+### Repair 
+The Action Planner will automatically attempt to repair invalid responses returned by the
+LLM using a feedback loop. When a validation fails, the ActionPlanner sends the error back to the
+model, along with an instruction asking it to fix its mistake. This feedback technique leads to a
+dramatic reduction in the number of invalid responses returned by the model.
diff --git a/getting-started/CONCEPTS/ACTIONS.md b/getting-started/CONCEPTS/ACTIONS.md
@@ -0,0 +1,62 @@
+# Action
+
+An action is an atomic function that is registered to the AI System. It is a fundamental building block of a plan.
+
+Here's an example of what an action creating a new list would look like in code. Call this action `createList`:
+
+### C#
+
+[List Bot sample](https://github.com/microsoft/teams-ai/blob/a20f8715d3fe81e11c330853e3930e22abe298af/dotnet/samples/04.ai.d.chainedActions.listBot/ListBotActions.cs#L15)
+```C#
+[Action("createList")]
+public bool CreateList([ActionTurnState] ListState turnState, [ActionParameters] Dictionary<string, object> parameters)
+{
+    ArgumentNullException.ThrowIfNull(turnState);
+    ArgumentNullException.ThrowIfNull(parameters);
+
+    string listName = GetParameterString(parameters, "list");
+
+    EnsureListExists(turnState, listName);
+
+  // Continues execution of next command in the plan.
+    return "";
+}
+```
+
+> Adding the `Action` attribute marks the method as an action. To register it to the AI System you have pass the instance object containing this method to the `AI.ImportActions(instance)` method. Alternatively, you can use the `AI.RegisterAction(name, handler)` to register a single action.
+
+### JS
+
+[List Bot sample](https://github.com/microsoft/teams-ai/blob/0fca2ed09d327ecdc682f2b15eb342a552733f5e/js/samples/04.ai.d.chainedActions.listBot/src/index.ts#L153)
+
+```typescript
+app.ai.action("createList", async (context: TurnContext, state: ApplicationTurnState, parameters: ListAndItems) => {
+  // Ex. create a list with name "Grocery Shopping".
+  ensureListExists(state, parameters.list);
+
+  // Continues exectuion of next command in the plan.
+  return true;
+});
+```
+
+> The `action` method registers the action named `createList` with corresponding callback function.
+
+
+## Default Actions
+
+The user can register custom actions or override default actions in the system. Below is a list of default actions present:
+
+| Action                | Called when                                                                                                                                                                                       |
+| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `___UnknownAction___` | An unknown action is predicted by the planner.                                                                                                                                                    |
+| `___FlaggedInput___`  | The input is flagged by the moderator.                                                                                                                                                            |
+| `___FlaggedOutput___` | The output is flagged by the moderator.                                                                                                                                                           |
+| `___HttpError___`     | The planner encounters an HTTP response with status code >= `400`                                                                                                                                 |
+| `__TooManySteps__`    | The planner task either executed too many steps or timed out.                                                                                                                                     |
+| `___PlanReady___`     | The plan has been predicted by the planner and it has passed moderation. This can be overriden to mutate the plan before execution.                                                               |
+| `___DO___`            | The AI system is executing a plan with the DO command. Overriding this action will change how _all_ DO commands are handled.                                                                      |
+| `___SAY___`           | The AI system is executing a plan with the SAY command. Overriding this action will change how _all_ SAY commands are handled. By default this will send the `response` message back to the user. |
+
+> Detailed description of each action can be found in the codebase.
+
+Note that `___DO___` and `___SAY___`, despite being called commands, are actually specialized actions. This means that a plan is really a sequence of actions.
diff --git a/getting-started/CONCEPTS/AI-SYSTEM.md b/getting-started/CONCEPTS/AI-SYSTEM.md
@@ -0,0 +1,7 @@
+# The AI System
+
+The AI system is responsible for moderating input and output, generating plans, and executing them. It can be used free standing or routed to by the Application object. This system is encapsulated in the `AI` class. It is made of three components, the moderator, the planner, and actions.
+
+1. [Moderator](./MODERATOR.md)
+2. [Planner](./PLANNER.md)
+3. [Actions](./ACTIONS.md)
diff --git a/getting-started/CONCEPTS/APPLICATION.md b/getting-started/CONCEPTS/APPLICATION.md
@@ -0,0 +1,69 @@
+# The `Application` class
+
+The `Application` class encapsulates all the business logic for the application and comprises of two major components, the _Activity Handler System_ and the _AI System_.
+
+
+## The Activity Handler System
+
+The activity handler system is the primary way to implement bot or message extension application logic. It is a set of methods and configurations that allows you to register callbacks (known as route handlers), which will trigger based on the incoming activity. These can be in the form of a message, message reaction, or virtually any interaction within the Teams app.
+
+Here's an example of registering a route handler that will run when the the user sends *"/login"* to the bot:
+
+**JS**
+```js
+// Listen for user to say '/login'.
+app.message('/login', async (context: TurnContext, state: TurnState) => {
+    await context.sendActivity(`Starting sign in flow.`);
+    // start signin flow
+});
+```
+
+**C#**
+```cs
+// Listen for user to say '/login'.
+app.OnMessage("/login", async (ITurnContext turnContext, TurnState turnState, CancellationToken cancellationToken) =>
+{
+    await turnContext.SendActivityAsync("Starting sign in flow.", cancellationToken: cancellationToken);
+        // start signin flow
+});
+```
+> The `message` and `OnMessage` methods are referred to as activity or *route registration* method. 
+> The `turnContext` and `turnState` parameters are present in every route handler. To learn more about them see [TURNS](TURNS.md).
+
+The `Application` groups the route registration methods based on the specific feature groups: 
+
+
+| **Feature**       | **Description**                                                  |
+| ----------------- | ---------------------------------------------------------------- |
+| Task Modules      | Task module related activities like `task/fetch`.                |
+| Message Extension | Message extension activities like `composeExtension/query`.      |
+| Meetings          | Meeting activites like `application/vnd.microsoft.meetingStart`. |
+| AdaptiveCards     | Adaptive card activities like `adaptiveCard/action`.             |
+| General           | Generic activites like `message`.                                |
+
+> To see all the route registration methods supported, see the migration docs ([JS](https://github.com/microsoft/teams-ai/blob/main/getting-started/MIGRATION/JS.md#activity-handler-methods) | [C#](https://github.com/microsoft/teams-ai/blob/main/getting-started/MIGRATION/DOTNET.md#activity-handler-methods)).
+
+In general, the activity handler system is all that is needed to have a functional bot or message extension. 
+
+## The AI System
+The AI System is an optional component used to plug in LLM powered experiences like user intent mapping, chaining...etc. It is configured once when orchestrating the application class. To learn more about it see [The AI System](./AI-SYSTEM.md).
+
+## The Routing Logic
+
+When an incoming activity reaches the server, the bot adapter handles the necessary authentication and creates a turn context object that encapsulates the activity details. Then the `Application`'s main method (`run()` in Javscript. `OnTurnAsync()` in C#) is called. Its logic can be broken down into these eight steps. 
+
+1. If configured in the application options, pulses of the `Typing` activity are sent to the user.
+2. If configured in the application options, the @mention is removed from the incoming message activity.
+3. The turn state is loaded using the configured turn state factory.
+4. If user authentication is configured, then attempt to sign the user in. If the user is already signed in, retrieve the access token and continue to step 5. Otherwise, start the sign in flow and end the current turn.
+5. The `beforeTurn` activity handler is executed. If it returns false, save turn state to storage and end the turn.
+6. All routes are iterated over and if a selector function is triggered, then the corresponding route handler is executed.
+7. If no route is triggered, the incoming activity is a message, and an AI System is configured, then it is invoked by calling the `AI.run()` method.
+8. The `AfterTurnAsync` activity handler is executed. If it returns true, save turn state to storage.
+
+
+> Note: _End the turn_ means that the main method has terminated execution and so the application has completed processing the incoming activity. 
+
+> Note: To learn about what a *turn* is, see [TURNS](TURNS.md).
+
+![the routing logic](../assets/routing-logic.png)
diff --git a/getting-started/CONCEPTS/AUGMENTATIONS.md b/getting-started/CONCEPTS/AUGMENTATIONS.md
@@ -0,0 +1,193 @@
+# Augmentations
+
+Augmentations virtually eliminate the need for prompt engineering. Prompts
+can be configured to use a named augmentation which will be automatically appended to the outgoing
+prompt. Augmentations let the developer specify whether they want to support multi-step plans (sequence),
+or create an AutoGPT style agent (monologue).
+
+It is recommended to read the [AI System](./AI-SYSTEM.md) and [Action Planner](./ACTION-PLANNER.md) guides if you are not familiar with plans and actions.
+
+## Sequence Augmentation
+
+This augmentation allows the model to return a sequence of actions to perform. It does this by appending instructions to the prompt text during runtime. These instructions guide the model to generate a plan object that uses actions defined in the `actions.json` file from the prompt template folder. 
+
+Here's an example of the `actions.json` file from the [Light Bot](https://github.com/microsoft/teams-ai/blob/77339da9e3e03bfd7f629fc796cfebdcd2891afb/js/samples/04.ai.c.actionMapping.lightBot/src/prompts/sequence/actions.json) sample:
+
+```json
+[
+    {
+        "name": "LightsOn",
+        "description": "Turns on the lights"
+    },
+    {
+        "name": "LightsOff",
+        "description": "Turns off the lights"
+    },
+    {
+        "name": "Pause",
+        "description": "Delays for a period of time",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "time": {
+                    "type": "number",
+                    "description": "The amount of time to delay in milliseconds"
+                }
+            },
+            "required": [
+                "time"
+            ]
+        }
+    }
+]
+```
+
+It defines three actions, `LightsOn`, `LightsOff` and `Pause`. The `Pause` action requires the `time` parameter, while the other two don't.
+
+These actions are then appended to the prompt text during runtime. This is text added to end of the prompt text:
+
+```txt
+actions:
+  LightsOn:
+      description: Turns on the lights
+  LightsOff:
+      description: Turns off the lights
+  Pause:
+      description: Delays for a period of time
+      parameters:
+        time:
+          type: number
+          description: The amount of time to delay in milliseconds
+                                             
+Use the actions above to create a plan in the following JSON format:
+
+{
+  "type": "plan",
+  "commands": [
+    {
+      "type": "DO",
+      "action": "<name>",
+      "parameters": {
+        "<name>": "<value>"
+      }
+    },
+    {
+      "type": "SAY",
+      "response": "<response>"
+    }
+  ]
+}
+```
+> Note: When the prompt is rendered, the above text is compressed to reduce token usage.
+
+The first section lists the actions in yaml structure. The second section tells the model to return a plan object of the following schema.
+
+### Configuring your prompt
+
+There are two steps to use sequence augmentation in your prompt:
+
+1. Update the prompt's `config.json` by adding the `augmentation` property. 
+```diff
+{
+    "schema": 1.1,
+    "description": "",
+    "type": "",
+    "completion": {},
++    "augmentation": {
++        "augmentation_type": "sequence"
++    }
+}
+```  
+2. Create an `actions.json` file in the prompt folder with a list of actions. For example:
+```json
+[
+    {
+        "name": "LightsOn",
+        "description": "Turns on the lights"
+    },
+    {
+        "name": "LightsOff",
+        "description": "Turns off the lights"
+    },
+]
+```
+
+To learn more about the action object schema see the corresponding typescript interface [ChatCompletionAction](https://github.com/microsoft/teams-ai/blob/0fca2ed09d327ecdc682f2b15eb342a552733f5e/js/packages/teams-ai/src/models/ChatCompletionAction.ts#L14).
+
+
+## Monologue Augmentation
+
+This augmentation adds support for an inner monologue to the prompt. The monologue helps the LLM perform chain-of-thought reasoning across multiple turns of conversation. It does this by appending instructions to the prompt text during runtime. It tells the model to explicitly show it's thought, reasoning and plan in response to the user's message, then predict the next action to execute. If looping is configured, then the predicted action can guide the model to predict the next action by returning the instruction as a string in the action handler callback. The loop will terminate as soon as the model predicts a *SAY* action, which sends the response back to the user.
+
+Using the `actions.json` example from abovce, the instructions appended to the prompt look like the text below.
+
+These actions are then used and appended to the prompt text in runtime. This is text added to end of the prompt text:
+
+```txt
+actions:
+  LightsOn:
+      description: Turns on the lights
+  LightsOff:
+      description: Turns off the lights
+  Pause:
+      description: Delays for a period of time
+      parameters:
+        time:
+          type: number
+          description: The amount of time to delay in milliseconds
+
+Return a JSON object with your thoughts and the next action to perform.
+Only respond with the JSON format below and base your plan on the actions above.
+If you're not sure what to do, you can always say something by returning a SAY action.
+If you're told your JSON response has errors, do your best to fix them.
+
+Response Format:
+
+{
+  "thoughts": {
+    "thought": "<your current thought>",
+    "reasoning": "<self reflect on why you made this decision>",
+    "plan": "- short bulleted\\n- list that conveys\\n- long-term plan"
+  },
+  "action": {
+    "name": "<action name>",
+    "parameters": {
+      "<name>": "<value>"
+    }
+  }
+}
+```
+
+> Note: When the prompt is rendered, the above text is compressed to reduce token usage.
+
+The first section lists the actions in yaml structure. The second section tells the model to return a plan object of the following schema.
+
+### Configuring your prompt
+
+To use monologue augmentation in your prompt there are two steps.
+
+1. Update the prompt's `config.json` by adding the `augmentation` property. 
+```diff
+{
+    "schema": 1.1,
+    "description": "",
+    "type": "",
+    "completion": {},
++    "augmentation": {
++        "augmentation_type": "monologue"
++    }
+}
+```  
+2. Create an `actions.json` file in the prompt folder with a list of actions. For example:
+```json
+[
+    {
+        "name": "LightsOn",
+        "description": "Turns on the lights"
+    },
+    {
+        "name": "LightsOff",
+        "description": "Turns off the lights"
+    },
+]
+```