Skip to content

Conversation

marot
Copy link

@marot marot commented Jul 15, 2025

Hi!

Disclaimer: I used Claude Code to help implement this.

I was working on a project using AshAi and tried to send images to the AI provider. There is a snippet in AshAi:

run prompt(
  ChatOpenAI.new!(%{model: "gpt-4o"}),
  prompt: [
    Message.new_system!("You are an expert at image analysis"),
    Message.new_user!([
      PromptTemplate.from_template!("Extra context: <%= @input.arguments.context %>"),
      ContentPart.image!("<%= @input.arguments.image_data %>", media: :jpg, detail: "low")
    ])
  ]
)

Since the intention of Ash is to be declarative, this is the desired way of declaring a prompt inside of an action. However, it doesn't actually work. The template in ContentPart.image! is taken as a string.

This PR adds support for PromptTemplate.image_template! and PromptTemplate.file_template!` so that it is possible to specify content parts with different types.

Please let me know if I should bring this up in an issue first.

Best,
Marco

- Add content_type and options fields to PromptTemplate schema
- Add image_template!/2 and file_template!/2 functions
- Update to_content_part!/2 and to_messages!/2 for new content types
- Add test coverage for image and file templates

%PromptTemplate{content_type: content_type} = template
when content_type in [:image, :file] ->
# For image/file templates, create a user message with the content part
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I'm not sure if it makes sense to have images/files that don't have the user role

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant