Skip to content

MISTAKE #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 14 commits into from
Closed

MISTAKE #1

wants to merge 14 commits into from

Conversation

neagualexa
Copy link
Collaborator

multi-layer approach for the short answer evaluation function:

  1. NLP algo to calculate similarity between student-teacher answers (deterministic)
  2. Language model that verifies the similarity in the context between the answers (looking for they key strings to be described in the student's response and for the context to be correct) (non-deterministic)
  3. Combining the feedback points generated by the layers above and using a language model to rephrase them into a concise point

The functions downloads a small language model Phi 3.5 mini for steps 2 and 3. Algorithms for step 1 were adapted from a previous final year project found under the old shortTextAnswerDev repo.

neagualexa and others added 14 commits September 20, 2024 11:22
@neagualexa
fix(in prorgess): move shortTextAnswerDev into shimmy boilerplate, lo…
34432a6 
…cal run but dockerfile in progress
@neagualexa
feat: rephraser LLM stage added before feedback is received by user
e54d711
@neagualexa
feat: readapted the lf_toolkit Result to our Evaluation_Response
25d99bc
@neagualexa
fix (in progress): working on passing shimmy server error on container
4d91a9b
@neagualexa
feat: custom feedback rephrasing implemented
2cdafe8
@neagualexa
fix: optinal params accepted for dev; restructure layers of dockerfile
d5ad25c
@neagualexa
fix: result working with shimmy, added more metadata in evaluation fu…
b6ecb19 
…nction output (tested on linux machine and docker works)
@neagualexa
fix: updated to new result class
5a3455e
@neagualexa
fix: updated ignore
8e69d60
@neagualexa
fix: llm model download in docker
6900a49
@neagualexa
fix: docker model fetch from layer
3718423
@neagualexa
fix: clean dockerfile
6f1dcb4
@neagualexa
feat: new deployment ci
f20c23f
@neagualexa
Merge pull request #1 from lambda-feedback/main
dc2e847 
Add troubleshooting section
@neagualexa neagualexa closed this Sep 26, 2024
@neagualexa neagualexa changed the title Implement mvp of short answer question evaluation with nlp & slm approach MISTAKE Sep 26, 2024
@neagualexa
Copy link
Collaborator Author

mistakely did a PR here

@neagualexa neagualexa deleted the nlp_slm branch September 26, 2024 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@neagualexa