WIP: Update AI Chat App Template recipes for AI SDK v6 and deploy-first workflow#5
Conversation
…rkflow Rewrite the AI Chat App Template recipes based on hands-on build testing: - Streaming AI Chat: AI SDK v6 API fixes (TextStreamChatTransport, parts arrays, sendMessage, maxOutputTokens), UIMessage-to-CoreMessage conversion, input focus restore, realistic layout override for scaffold padding - Lakebase Chat Persistence: add Lakebase project setup steps (create-project, scaffold with --features=lakebase, env vars, databricks.yml), deploy-first workflow with CLI permission grants, concrete recovery step for schema ownership trap, correct AI SDK v6 client guidance (no onResponse or initialMessages) - Query AI Gateway Endpoints: new recipe replacing model-serving-endpoint- creation, with endpoint placeholders and workspace-specific discovery - Remove todo CRUD recipe (lakebase-data-persistence) from AI Chat page - Add idempotency notes for databricks_create_role/GRANT commands - Update @ai-sdk/openai version note to reflect v3.x
djliden
force-pushed
the
djliden/ai-chat-recipe-updates
branch
from
b934d74 to
b9f0d65
Compare
| Build a streaming AI chat experience in a Databricks App using AI SDK, AI Elements, and a Databricks Model Serving endpoint. | ||
| Build a streaming AI chat experience in a Databricks App using Vercel AI SDK with Databricks Model Serving and OpenAI-compatible endpoints. | ||
|
|
||
| ### 1. Follow the prerequisite recipes first |
There was a problem hiding this comment.
We don't need to change this now but thinking prerequisite should likely be metadata and not part of the content. I know you didn't introduce them just have thought a bit more about it since I added this.
Template = recipe A + recipe C
If recipe C requires recipe B then we should always add it to the cookbook to be complete.
Template = recipe A + recipe B + recipe C
And for standalone recipes, we can render the meta data of prerequisite into the content.
| > **Optional**: For pre-built UI components, install individual AI Elements: | ||
| > ```bash | ||
| > # AI Elements CLI needs components.json in root directory | ||
| > cp client/components.json . |
There was a problem hiding this comment.
I'd just have one line here that inits & mentions Shadcn (the baseline component library that introduces components.json). shadcn init adds all relevant things + components.json.
|
|
||
| ```bash | ||
| cd <app-name> | ||
| npx skills add databricks/databricks-agent-skills --all |
There was a problem hiding this comment.
databricks/databricks-agent-skills is part of the scaffolding recipe. I wouldn't repeat them here (duplicate code) but I would re-introduce the AI SDK and AI elements skills as optional. Why did you remove them?
There was a problem hiding this comment.
General question on skills: If the purpose of these templates is to be copied and pasted into an agent to one-shot an application, what is the point of installing skills at all if we don't include an explicit reload step to install those skills? Is this more for the future, for iterating on a project with a new agent instance? Or does it make sense to have "human steps" at the beginning "install these skills before starting you agent," or an optional "restart" step? Or instructions for the agent to manually look at the skills?
(in my testing, skills were installed but never used)
| ``` | ||
|
|
||
| ### 4. Configure environment variables for model serving | ||
| > **Note**: This installs skills for Databricks Apps, AppKit, Lakebase, and AI Gateway patterns. Essential for agent-driven development. |
There was a problem hiding this comment.
Duplicate already in scaffolding.
| }); | ||
| // AI SDK v6 client sends UIMessage objects with a parts array. | ||
| // Convert to CoreMessage format for streamText(). | ||
| interface UIMessage { |
There was a problem hiding this comment.
UIMessage type can be imported from AI SDK I'm pretty sure.
| ```tsx | ||
| import { useChat } from "@ai-sdk/react"; | ||
| import { TextStreamChatTransport } from "ai"; | ||
| import { useState, useRef, useEffect } from "react"; |
There was a problem hiding this comment.
Likely those things are overkill. I can change later.
There was a problem hiding this comment.
Removed, but want to note that these were added to address a very common failure state where the text input area of the chat app would lose focus without explicit direction to the agent.
| ref={inputRef} | ||
| value={input} | ||
| onChange={handleInputChange} | ||
| onChange={(e) => setInput(e.target.value)} |
There was a problem hiding this comment.
I can change this but state isn't needed here. Worked before as well.
There was a problem hiding this comment.
vercel/ai#8725 <- intended to address this for v6?
There was a problem hiding this comment.
Let's leave it as is! I was referring to React state vs. unmanaged forms.
You only really need state if you want to programmatically manage the value. The only place where we do this here is the onSubmit (where we set it to "" to reset the input field after submission) but we can also reset the form field without React. Not worth going into this tbh - sorry I brought it up. I'm just a sucker for "use the web platform over React wherever possible" 😅
Pseudo code but roughly:
onSubmit={(e) => {
e.preventDefault();
const form = e.target;
const input = form.chatinput // needs to match id/name prop on input ele
if (input.trim()) {
void sendMessage({ text: input });
form.reset();
}
}}
This way we can fully remove the onChange and useState. Not important and I can make the React UI code pretty in the future.
| ``` | ||
|
|
||
| ### 8. Add chat persistence in Lakebase | ||
| > **Layout tip**: The scaffold wraps page content in a `<main>` with `p-6` padding. For a full-bleed chat layout, use `h-[calc(100vh-4rem)] -m-6` on the chat container to break out of the padding and fill the viewport below the nav bar. The code above includes this pattern. |
There was a problem hiding this comment.
I don't think we should go too much into UI in this recipe. We can add a "chat UI recipe" in the future if we think it's worth it but I would either just let AI Elements do their work or focus on wiring.
There was a problem hiding this comment.
I'm good either way on this. I added these notes to address some frequent UI issues that were occurring across runs.
| This recipe uses a simplified relational shape inspired by common production chat schemas (`chat` plus `message`) and adapts it to Databricks AppKit + Lakebase. | ||
|
|
||
| ### 1. Create chat tables | ||
| ### 1. Create a Lakebase project |
There was a problem hiding this comment.
This is duplicate code from the existing lakebase recipe. Recipes are meant to complement each other and serve a single purpose.
Lakebase base recipe -> create a project and how to work with it
Lakebase chat persistence -> how to add the required schemas for chat UIs
There was a problem hiding this comment.
I would suggest removing the to-do list materials from the base recipe in that case—this was misleading to agents building the chat app in several cases, in spite of my asking for just a chat app.
There was a problem hiding this comment.
Good point! Will do!
| If you haven't scaffolded yet, include `--features=lakebase`: | ||
|
|
||
| ```bash | ||
| databricks apps init \ |
There was a problem hiding this comment.
Same here. This is not focused enough and should be in more generic recipes.
| <TemplateDetail template={template} rawMarkdown={rawMarkdown}> | ||
| <DatabricksLocalBootstrap /> | ||
| <hr /> | ||
| <LakebaseDataPersistence /> |
There was a problem hiding this comment.
Why remove LakebaseDataPersistence? It should be a requirement for LakebaseChatPersistence which we use below.
There was a problem hiding this comment.
In my experience, even when I gave clear instructions about what I wanted, Opus would follow LakebaseDataPersistence too closely and make a weird hybrid to-do app/AI chat app. So if LakebaseDataPersistence needs to be foundational, I think it should remove the example to-do app and just include the universally applicable setup information.
There was a problem hiding this comment.
Great input will do!
…tter - Remove duplicate agent skills install (already in bootstrap recipe) - Simplify AI Elements init to single shadcn init line - Import UIMessage from AI SDK instead of defining inline - Remove focus-restore useEffect and layout tip (too much UI detail) - Remove ESLint useRef note and ast-grep deploy note - Trim version pin language - Remove Python SDK alternative in foundation-models-api (CLI-first) - Remove duplicate skills command alternative in bootstrap - Remove "AI Elements" from recipe description - Run prettier on all changed files
2 participants
Summary
Rewrite the AI Chat App Template recipes based on hands-on build testing with AI SDK v6 and Databricks AI Gateway.
Test plan
npm run buildpasses