[Feature] Print the final prompt of a DSPy program

[dovanquyet]

What feature would you like to see?

Hi DSPy team,

I am using DSPy in my project. I want to convert DSPy programs into regular user and system prompts, so that people who are not familiar with DSPy can run LLMs with regular user and system prompts instead.

As far as I know, the only way to get regular user and system prompts is to run the program at least once, then print the history.
However, the history will contain the actual input, and I need to clean up the actual input to get the generic prompts. That is not elegant.
Thus, I would like to request a function to print out the generic user and system prompts.

The following is an example:

DSPy implementation

class BasicQA(dspy.Signature):
    __doc__ = """Answer questions with Yes/No at the begining.""" # instruction

    question = dspy.InputField(desc="The Yes/No question to be answered") # remaining attributes: format, prefix
    answer = dspy.OutputField(desc="Only 'Yes' or 'No'")

Text obtained via inspect_history

System message:

Your input fields are:
1. `question` (str): The Yes/No question to be answered

Your output fields are:
1. `answer` (str): Only 'Yes' or 'No'

All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## question ## ]]
{question}

[[ ## answer ## ]]
{answer}

[[ ## completed ## ]]

In adhering to this structure, your objective is: 
        Answer questions with Yes/No at the begining.


User message:

[[ ## question ## ]]
Does the following poem compare the beach to something else at least twice?

Upon the shore where dreams were spun,
The ocean's breath a gentle sigh,
A seashell whispers of the sun,
Beneath the vast and endless sky.

Respond with the corresponding output fields, starting with the field `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.


Response:

[[ ## answer ## ]]
Yes

[[ ## completed ## ]]

Equilavent prompts (what I want the requested function to return)

system_prompt = """Your input fields are:
1. `question` (str): The Yes/No question to be answered

Your output fields are:
1. `answer` (str): Only 'Yes' or 'No'

All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## question ## ]]
{question}

[[ ## answer ## ]]
{answer}

[[ ## completed ## ]]

In adhering to this structure, your objective is: 
        Answer questions with Yes/No at the begining.
"""

user_prompt = """[[ ## question ## ]]
{}

Respond with the corresponding output fields, starting with the field `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.

Use the equilavent prompts without DSPy

new_question = '...'
openai.chat.completions.create(
    model=model_name,
    temperature=temperature,
    max_tokens=max_tokens,
    messages=[
        {"role": "system", "content": (system_prompt)},
        {"role": "user", "content": user_prompt.format(new_question)},
    ],
)

Best.

Would you like to contribute?

• Yes, I'd like to help implement this.
• No, I just want to request it.

Additional Context

No response

[okhat]

Hey! You can actually do this:

{
  name: adapter.format(
    p.signature,
    demos=p.demos,
    inputs={k: f"{{{k}}}" for k in p.signature.input_fields},
  )
  for name, p in program.named_predictors()
}

where the default adapter = dspy.ChatAdapter(), but advanced users customize this sometimes.

I think we'll create syntactic sugar for this access. I can see it's useful.

P.S. You obviously lose the actual control flow logic this way, and lose all kinds of adapter decoding/parsing/retries. If you want to just export the program, you could just do program.save() or program.dump_state() and load that later.

[okhat]

We should probably make some kind of program.get_prompt_templates() helper for this?

[dovanquyet]

Thank you so much for your help!

Now I can get the generic prompts with adapter.format()

And I am not sure if DSPy has some kind of program.get_prompt_templates() helper for this @@

About program.save(), I know that way to save the program, but other people need to know DSPy to use it. That's why I want to convert DSPy program back to generic prompts.