Palimpzest Recipes: How To Get Source Attribution Working

[mikecafarella]

We should create a series of short "Palimpzest Recipes". These will be short 1-2 page documents that show how to accomplish a commonly-wished-for goal.

If PZ is perfect, this is basically a documentation task. But of course PZ probably has a lot of problems, which we will only realize when we try to write the "easy story on how to do X" and realize that we can't do it. So accomplishing this task will sometimes mean just writing a doc, but will often mean doing some design and writing code.

In this case, the Goal is: HOW TO GET SOURCE ATTRIBUTION WORKING.

It is common to want to do RAG Question Answering, in which:

1. The user provides a set of background documents
2. A user query arrives, which can possibly be answered using content in the background documents
3. We use a RAG-like mechanism (see Palimpzest retrieve()) to find relevant documents
4. We compose an LLM query that combines the user's original query with the relevant documents, and then answer it.

First, we should get a small demo working for 1-4 above. We should look at @sivaprasadsudhir 's code for his RAG work.

Next, we should modify the demo to also provide:
5. When the answer is generated, it should include three elements: (a) the answer itself, (b) the document name that provided the answer evidence, and (c) the region of text from that document that contains the evidence. If there is not one single citation that is possible, we should return empty values for the cited doc and text.