Skip to content

Commit 05adb72

Browse files
authored
Fix prompt parsing & other minor (#18)
* update: using chat function workflows * fix: add config.json boilerplate * fix: test chat function name * revert to non workflows until backend ready * minor updates readme * debugging input * fix: prompt parsing
1 parent a8fa7ec commit 05adb72

File tree

7 files changed

+184
-8
lines changed

7 files changed

+184
-8
lines changed

.dockerignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ dmypy.json
131131
# VSCode configuration
132132
.vscode
133133

134-
# Evaluation function config
134+
# Chat function config
135135
config.json
136136

137137
# README

.github/workflows/dev.yml

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,3 +97,26 @@ jobs:
9797
aws lambda update-function-code \
9898
--function-name $FUNCTION_NAME \
9999
--image-uri $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
100+
101+
# build:
102+
# name: Build Docker Image
103+
# uses: lambda-feedback/chat-function-workflows/.github/workflows/gh_build.yml@main
104+
# needs: test
105+
# permissions:
106+
# contents: read
107+
# id-token: write
108+
# packages: write
109+
110+
# deploy:
111+
# name: Deploy to Lambda Feedback
112+
# uses: lambda-feedback/chat-function-workflows/.github/workflows/dev_deploy.yml@main
113+
# needs: test
114+
# with:
115+
# template-repository-name: "lambda-feedback/chat-function-boilerplate"
116+
# permissions:
117+
# contents: read
118+
# id-token: write
119+
# packages: write
120+
# secrets:
121+
# aws-access-key-id: ${{ secrets.LAMBDA_CONTAINER_PIPELINE_AWS_ID }}
122+
# aws-secret-key: ${{ secrets.LAMBDA_CONTAINER_PIPELINE_AWS_SECRET}}

.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,6 @@ dmypy.json
132132
.vscode
133133

134134
.DS_Store
135-
evaluation_function/db_analytics
136135

137136
# Synthetic data conversations
138137
src/agents/utils/synthetic_conversations/*.json

README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ docker run -e OPENAI_API_KEY={your key} -e OPENAI_MODEL={your LLM chosen model n
110110
docker run --env-file .env -it --name my-lambda-container -p 8080:8080 llm_chat
111111
```
112112

113-
This will start the evaluation function and expose it on port `8080` and it will be open to be curl:
113+
This will start the chat function and expose it on port `8080` and it will be open to be curl:
114114

115115
```bash
116116
curl --location 'http://localhost:8080/2015-03-31/functions/function/invocations' --header 'Content-Type: application/json' --data '{"message":"hi","params":{"conversation_id":"12345Test","conversation_history": [{"type":"user","content":"hi"}]}}'
@@ -156,19 +156,19 @@ Body with optional Params:
156156

157157
Deploying the chat function to Lambda Feedback is simple and straightforward, as long as the repository is within the [Lambda Feedback organization](https://github.com/lambda-feedback).
158158

159-
After configuring the repository, a [GitHub Actions workflow](.github/workflows/main.yml) will automatically build and deploy the evaluation function to Lambda Feedback as soon as changes are pushed to the main branch of the repository. For development, the [GitHub Actions Dev workflow](.github/workflows/dev.yml) also deploys a dev version of the function onto AWS.
159+
After configuring the repository, a [GitHub Actions workflow](.github/workflows/main.yml) will automatically build and deploy the chat function to Lambda Feedback as soon as changes are pushed to the main branch of the repository. For development, the [GitHub Actions Dev workflow](.github/workflows/dev.yml) also deploys a dev version of the function onto AWS.
160160

161161
## Troubleshooting
162162

163163
### Containerized Function Fails to Start
164164

165-
If your evaluation function is working fine when run locally, but not when containerized, there is much more to consider. Here are some common issues and solution approaches:
165+
If your chat function is working fine when run locally, but not when containerized, there is much more to consider. Here are some common issues and solution approaches:
166166

167167
**Run-time dependencies**
168168

169169
Make sure that all run-time dependencies are installed in the Docker image.
170170

171171
- Python packages: Make sure to add the dependency to the `requirements.txt` or `pyproject.toml` file, and run `pip install -r requirements.txt` or `poetry install` in the Dockerfile.
172172
- System packages: If you need to install system packages, add the installation command to the Dockerfile.
173-
- ML models: If your evaluation function depends on ML models, make sure to include them in the Docker image.
174-
- Data files: If your evaluation function depends on data files, make sure to include them in the Docker image.
173+
- ML models: If your chat function depends on ML models, make sure to include them in the Docker image.
174+
- Data files: If your chat function depends on data files, make sure to include them in the Docker image.

config.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"ChatFunctionName": "test-lambda-chat"
3+
}
Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
{
2+
"message": "what should i do for this part?",
3+
"params": {
4+
"include_test_data": true,
5+
"conversation_history": [
6+
{ "type": "user", "content": "what should i do for this part?" }
7+
],
8+
"summary": "",
9+
"conversational_style": "",
10+
"question_response_details": {
11+
"questionSubmissionSummary": [
12+
{
13+
"publishedPartId": "37a703f7-cd61-4f63-aed2-17f83451b5c5",
14+
"publishedPartPosition": 2,
15+
"publishedResponseAreaId": "63dd1aee-5724-4cf3-8dcd-4b0ca7953c2a",
16+
"publishedResponseAreaPosition": 0,
17+
"responseAreaUniversalId": "cc912f18-33fb-4612-a024-492f01443c6f",
18+
"publishedResponseAreaPreResponseText": "\\(\\tau_{\\mathrm{tw}}=\\)",
19+
"publishedResponseType": "NUMBER",
20+
"publishedResponseConfig": null,
21+
"totalSubmissions": 0,
22+
"totalWrongSubmissions": 0
23+
},
24+
{
25+
"publishedPartId": "f63ee543-c338-49a9-b07c-14789966d08f",
26+
"publishedPartPosition": 3,
27+
"publishedResponseAreaId": "3ba5a1eb-22cc-4599-a0e6-35e892c173c5",
28+
"publishedResponseAreaPosition": 0,
29+
"responseAreaUniversalId": "776c0c03-cc7a-4e9c-b008-5ce8cc07ca4b",
30+
"publishedResponseAreaPreResponseText": "\\(\\tau_{\\mathrm{bw}} = \\)",
31+
"publishedResponseType": "NUMBER",
32+
"publishedResponseConfig": null,
33+
"totalSubmissions": 0,
34+
"totalWrongSubmissions": 0
35+
}
36+
],
37+
"questionInformation": {
38+
"questionTitle": "Flow between two parallel plates",
39+
"questionGuidance": "What is the shear stress on a flat plate? This problem will help you answer this question that you will find often in many engineering problems.",
40+
"questionContent": "Consider the flow between two infinite parallel plates, separated by a distance $h$ with the top one moving from left to right with constant speed $U_0$. Consider the flow to be Newtonian, incompressible, with uniform dynamic viscosity. As schematically shown in the figure below, the velocity field with respect to a Cartesian frame of reference is $\\vec{u}=\\left[U_0y/h~~0~~0\\right]^{T}$, with the $x$-axis oriented from left to right.\n\n![](https://lambda-feedback-dev-frontend-client-bucket.s3.amazonaws.com/77c9b9d3-e6cb-43a7-85c6-6464c6af3700/9f649438-a9d8-4460-a9fb-55ef36422cf3.png). \n\nGiven $U_0=10$ m/s, $h=10$ mm and $\\mu=1\\times 10^{-3}$ $\\mathrm{kg/(ms)}$, \n",
41+
"durationLowerBound": 15,
42+
"durationUpperBound": 25,
43+
"parts": [
44+
{
45+
"publishedPartId": "8e35723d-e549-4dbd-97ca-29c1134ceb69",
46+
"publishedPartPosition": 0,
47+
"publishedPartContent": "Compute the deformation tensor at any location in the flow field.\n",
48+
"publishedPartAnswerContent": "$$\n\\underline{\\underline{D}}= \\begin{bmatrix} 0 & 500 & 0 \\\\ 500 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ \\end{bmatrix} \\mathrm{s^{\\scriptsize -1}}.\n$$\n\n",
49+
"publishedWorkedSolutionSections": [
50+
{
51+
"id": "deec8cb8-7441-46e9-8435-e0af7b8d9cf8",
52+
"position": 0,
53+
"title": null,
54+
"content": "The deformation tensor for such a flow is:\n\n***\n\n$$\n\\underline{\\underline{D}}= \\frac{1}{2}(\\underline{\\underline{J}}+\\underline{\\underline{J}}^T)=\\begin{bmatrix} 0 & U_0/(2h) & 0 \\\\ U_0/(2h) & 0 & 0 \\\\ 0 & 0 & 0 \\\\ \\end{bmatrix}.\n$$\n\n***\n\nThe deformation tensor is uniform and the only non-zero component is $D_{xy}=D_{yx}=500~\\mathrm{s^{\\scriptsize. -1}}$. Hence:\n\n***\n\n$$\n\\underline{\\underline{D}}= \\begin{bmatrix} 0 & 500 & 0 \\\\ 500 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ \\end{bmatrix} \\mathrm{s^{\\scriptsize -1}}.\n$$\n"
55+
}
56+
],
57+
"publishedResponseAreas": []
58+
},
59+
{
60+
"publishedPartId": "cdef8c4e-c929-4655-a190-1ace4824dd6a",
61+
"publishedPartPosition": 1,
62+
"publishedPartContent": "Compute the spin tensor at any location in the flow field.\n",
63+
"publishedPartAnswerContent": "$$\n\\underline{\\underline{\\Omega}}= \\begin{bmatrix} 0 & 500 & 0 \\\\ -500 & 0 & 0 \\\\ 0 & 0 & 0 \\\\ \\end{bmatrix} \\mathrm{s^{\\scriptsize -1}}.\n$$\n\nSee worked solutions for an explanation.\n",
64+
"publishedWorkedSolutionSections": [
65+
{
66+
"id": "8cbb1d33-9c15-4969-99c0-87859a761fac",
67+
"position": 0,
68+
"title": null,
69+
"content": "The spin tensor for such a flow is:\n\n***\n\n$$\n\\underline{\\underline{\\Omega}}=\\frac{1}{2}\\left(\\underline{\\underline{J}}-\\underline{\\underline{J}}^T\\right)=\\begin{bmatrix}0 & U_0/(2h) & 0\\\\-U_0/(2h)&0&0\\\\0&0&0\\end{bmatrix}.\n$$\n\n***\n\nAgain, also the spin tensor is uniform all over the flow and the only non-zero component is $\\Omega_{xy}=-\\Omega_{yx}=500 \\mathrm{s}^{-1}$. This means that any infinitesimal fluid particle spins and stretches at the same time. This is also illustrated below:\n\n***\n\n![](https://lambda-feedback-dev-frontend-client-bucket.s3.amazonaws.com/77c9b9d3-e6cb-43a7-85c6-6464c6af3700/51e0490a-79d3-483b-a9f2-7816066f0963.png)\n"
70+
}
71+
],
72+
"publishedResponseAreas": []
73+
},
74+
{
75+
"publishedPartId": "37a703f7-cd61-4f63-aed2-17f83451b5c5",
76+
"publishedPartPosition": 2,
77+
"publishedPartContent": "Compute the shear stress on the top plate.\n",
78+
"publishedPartAnswerContent": "$$\n\\boxed{\\tau_\\mathrm{tw}=-1 \\mathrm{Pa}}\n$$",
79+
"publishedWorkedSolutionSections": [
80+
{
81+
"id": "d8b68bfc-0d70-4f9b-89bf-81796da1b392",
82+
"position": 0,
83+
"title": null,
84+
"content": "This question uses the same geometry as Q9.4. However, the origin here is located at the bottom plate; the boundary condition at the top plate is different; and we are given information about the shear stress tensor, $\\underline{\\underline{\\tau}}$: for a Newtonian fluid and an incompressible flow, $\\underline{\\underline{\\tau}}=2\\mu\\underline{\\underline{D}}$. (Note that $\\underline{\\underline{\\sigma}}=\\underline{\\underline{\\tau}}-p\\underline{\\underline{I}}$).\n\n  \n\n***\n\nFollowing the solution to Q9.4 (neglecting the $z-$ direction), the force per unit area exerted by the top plate *on the fluid* is\n\n***\n\n$$\n\\vec{\\mathcal{T}}^{\\left(\\hat{e}_y\\right)}{\\footnotesize\\left( y=h\\right)}=\\underline{\\underline{\\sigma}}\\hat{n}=\\begin{bmatrix} \\sigma_{xx}& \\sigma_{xy}\\\\\\sigma_{yx}&\\sigma_{yy} \\end{bmatrix}_{ y=h}\\begin{bmatrix} n_x\\\\ n_y\\end{bmatrix}_{ y=h}\n$$\n\n***\n\nNoting that $n_x=0$,\n\n$$\n\\vec{\\mathcal{T}}^{\\left(\\hat{e}_y\\right)}{\\footnotesize\\left( y=h\\right)}=\\begin{bmatrix} \\sigma_{xx}& \\sigma_{xy}\\\\\\sigma_{yx}&\\sigma_{yy} \\end{bmatrix}_{ y=h}\\begin{bmatrix} 0\\\\ 1\\end{bmatrix}_{ y=h}\n$$\n\n\n\n***\n\n$$\n\\vec{\\mathcal{T}}^{\\left(\\hat{e}_y\\right)}{\\footnotesize\\left( y=h\\right)}=\\begin{bmatrix} \\sigma_{xy}\\\\\\sigma_{yy} \\end{bmatrix}_{ y=h}=\\begin{bmatrix} \\tau_{xy}\\\\-p \\end{bmatrix}_{y=h}=\\begin{bmatrix} 2\\mu{}D_{xy}\\\\-p \\end{bmatrix}_{y=h}=\\begin{bmatrix} \\mu{}\\frac{\\partial{}u}{\\partial{}y}\\\\-p \\end{bmatrix}_{y=h}=\\begin{bmatrix} \\mu{}U_0/h\\\\-p \\end{bmatrix}_{y=h}.\n$$\n\n***\n\n  \n\nThe shearing component of the stress is the first component ($x-$direction). The shear stress on the fluid is therefore\n\n***\n\n$$\n\\mathcal{T}_{x}^{\\left(\\hat{e}_y\\right)}{\\footnotesize\\left( y=h\\right)}\\ = \\mu{}U_0/h = 10^{-3}\\ast{}10/10^{-2} = 1\\:\\mathrm{Pa}.\n$$\n\n  \n\n***\n\nNote that the viscous stress tensor in this specific flow is constant everywhere (i.e. $\\tau_{xy}$ does not depend on space in this specific flow). The stress on the plate is equal and opposite to the stress on the fluid, \n\n***\n\n$$\n\\boxed{ \\tau_{tw} = -\\mathcal{T}_{x}^{\\left(\\hat{e}_y\\right)}{\\footnotesize\\left( y=h\\right)} =\\mathcal{T}_{x}^{\\left(-\\hat{e}_y\\right)}{\\footnotesize\\left( y=h\\right)} = -1 \\:\\mathrm{Pa}. }\n$$\n\n***\n\n  \n\nThe force on the plate due to $\\tau_{tw}$ opposes the motion of the plate. In fact, it is necessary to apply a force on the plate from left to right to keep the plate moving with constant velocity $U_0$ (note that if the plate moves with constant velocity, the sum of the forces on the plate has to be the null vector).\n\n  \n\n***\n\nThe approach to the solution shown here takes advantage of the work already done in Q9.4. The benefit is that the signs of each term were already carefully accounted for. An alternative approach would have been to start from first principles. Given that only the stress is required (not the force) we would not need to include the area and would not need to start with an integral. We could look at the geometry and conclude that $\\tau_{xy}$ is the component of the stress that we require. If you took this approach, be careful to be rigorous about how you derive the sign of the stress on the plate.\n"
85+
}
86+
],
87+
"publishedResponseAreas": [
88+
{
89+
"id": "63dd1aee-5724-4cf3-8dcd-4b0ca7953c2a",
90+
"position": 0,
91+
"universalResponseAreaId": "cc912f18-33fb-4612-a024-492f01443c6f",
92+
"preResponseText": "\\(\\tau_{\\mathrm{tw}}=\\)",
93+
"Response": {
94+
"id": "a1ddb6fa-25be-4045-bdea-e995848bd508",
95+
"responseType": "NUMBER",
96+
"config": null,
97+
"answer": -1
98+
},
99+
"responseType": "NUMBER",
100+
"answer": -1
101+
}
102+
]
103+
},
104+
{
105+
"publishedPartId": "f63ee543-c338-49a9-b07c-14789966d08f",
106+
"publishedPartPosition": 3,
107+
"publishedPartContent": "Compute the shear stress on the bottom plate.\n",
108+
"publishedPartAnswerContent": "$$\n\\boxed{\\tau_\\mathrm{bw} = 1 \\mathrm{Pa}}\n$$",
109+
"publishedWorkedSolutionSections": [
110+
{
111+
"id": "6900ae4e-1562-4730-bc26-149db92ee449",
112+
"position": 0,
113+
"title": null,
114+
"content": "On the bottom plate the velocity gradient and therefore the shear stress are the same as on the top plate. However, the bottom plate is oriented in the opposite direction to the top plate. Following the same approach as part (c), the force per unit area exerted by the *bottom* plate *on the fluid* is\n\n***\n\n$$\n\\vec{\\mathcal{T}}^{\\left(\\hat{e}_y\\right)}{\\footnotesize\\left( y=0\\right)}=\\begin{bmatrix} \\sigma_{xx}& \\sigma_{xy}\\\\\\sigma_{yx}&\\sigma_{yy} \\end{bmatrix}_{ y=0}\\begin{bmatrix} 0\\\\ -1\\end{bmatrix}_{ y=0}\n$$\n\nhence\n\n***\n\n$$\n\\vec{\\mathcal{T}}^{\\left(-\\hat{e}_y\\right)}{\\footnotesize\\left( y=0\\right)}=\\begin{bmatrix} -\\sigma_{xy}\\\\-\\sigma_{yy} \\end{bmatrix}_{y=0}=\\begin{bmatrix} -\\tau_{xy}\\\\ p \\end{bmatrix}_{y=0}=\\begin{bmatrix} -2\\mu{}D_{xy}\\\\ p \\end{bmatrix}_{y=0}=\\begin{bmatrix} -\\mu{}\\frac{\\partial{}u}{\\partial{}y}\\\\ p \\end{bmatrix}_{y=0}=\\begin{bmatrix} -\\mu{}U_0/h\\\\ p \\end{bmatrix}_{y=0}.\n$$\n\n***\n\nThe stress on the fluid is the force per unit area; and the shearing component of the stress is the first component ($x-$direction). Given the orientation of the bottom plate, the shear stress exerted on the fluid by the bottom plate is\n\n***\n\n$$\n\\mathcal{T}_{x}^{\\left(-\\hat{e}_y\\right)}{\\footnotesize\\left( y=0\\right)}\\ = -\\mu{}U_0/h = -10^{-3}\\ast{}10/10^{-2} = -1\\:\\mathrm{Pa}.\n$$\n\n***\n\n  \n\nThe stress on the plate is equal and opposite to the stress on the fluid,\n\n***\n\n$$\n\\boxed{ \\tau_{bw} = 1 \\:\\mathrm{Pa}. }\n$$\n"
115+
}
116+
],
117+
"publishedResponseAreas": [
118+
{
119+
"id": "3ba5a1eb-22cc-4599-a0e6-35e892c173c5",
120+
"position": 0,
121+
"universalResponseAreaId": "776c0c03-cc7a-4e9c-b008-5ce8cc07ca4b",
122+
"preResponseText": "\\(\\tau_{\\mathrm{bw}} = \\)",
123+
"Response": {
124+
"id": "6aff9851-4d28-4d16-855d-a1c3e81319a9",
125+
"responseType": "NUMBER",
126+
"config": null,
127+
"answer": 1
128+
},
129+
"responseType": "NUMBER",
130+
"answer": 1
131+
}
132+
]
133+
}
134+
]
135+
},
136+
"questionAccessInformation": {
137+
"estimatedMinimumTime": "15 minutes",
138+
"estimaredMaximumTime": "25 minutes",
139+
"timeTaken": "less than one minute",
140+
"accessStatus": "too little time spent on this question.",
141+
"markedDone": "",
142+
"currentPart": {
143+
"id": "8e35723d-e549-4dbd-97ca-29c1134ceb69",
144+
"position": 0
145+
}
146+
}
147+
},
148+
"conversation_id": "49f97c01-9a46-4773-9722-f8d231c55209",
149+
"agent_type": "informational"
150+
}
151+
}

src/agents/utils/parse_json_to_prompt.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ def format_response_area_details(responseArea: ResponseAreaDetails, studentSumma
159159
{submissionDetails}"""
160160

161161
def format_part_details(part: PartDetails, currentPart: CurrentPart, summary: List[StudentWorkResponseArea]) -> str:
162-
if not part or not part.publishedResponseAreas:
162+
if not part:
163163
return ''
164164

165165
responseAreas = "\n".join(

0 commit comments

Comments
 (0)