Llm flask api #1789
Llm flask api #1789
Conversation
- Add basic incomplete post body testing. Running them for real would require OpenAI calls
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
Nathan's comments from the Zoom on cutting down dependency: I expect the imports here are sufficient to run the python llm functions: Line 26 in 8032ba1 There are some fairly hefty ones there that are needed... langchain, sklearn, chromadb We can probably cut sklearn if we find easy replacement for the small number of functions we are importing from it. |
.dockerignore
Outdated
| .git No newline at end of file | ||
| .git |
There was a problem hiding this comment.
I need to figure out how to configure my editor to not make changes like this automatically
infra/Dockerfile.firebase
Outdated
| USER node No newline at end of file | ||
| USER node |
| @https_fn.on_request(secrets=["OPENAI_DEV"]) | ||
| def httpsflaskexample(req: https_fn.Request) -> https_fn.Response: | ||
| with app.request_context(req.environ): | ||
| return app.full_dispatch_request() |
There was a problem hiding this comment.
This is the magic sauce to expose these routes to firebase
| source venv/bin/activate # .fish if using fish | ||
| pip3 install -r requirements.txt | ||
| python3 -m flask --app main run | ||
| ``` |
There was a problem hiding this comment.
Need to add more details here and remove everything below it
| "runtime": "nodejs18" | ||
| }, | ||
| { | ||
| "predeploy": [". llm/venv/bin/activate && python3.10 -m pip install -r llm/requirements.txt"], |
There was a problem hiding this comment.
IIUC, this file is mainly used when you type firebase deploy ... so this pre-deploy will not happen inside a Docker environment
| "runtime": "nodejs18" | ||
| }, | ||
| { | ||
| "predeploy": ["source /app/llm/venv/bin/activate && python3.10 -m pip install -r /app/llm/requirements.txt"], |
There was a problem hiding this comment.
IIUC, this one should be used by docker compose and should run inside the docker enviroment (note this doesn't exactly work for reasons I don't understand)
| @app.route("/summary", methods=["POST"]) | ||
| def summary(): | ||
| body = json.loads(request.data) | ||
| # We require bill_id, bill_title, bill_text to exist as keys in the POST | ||
| if not is_intersection(body.keys(), {"bill_id", "bill_title", "bill_text"}): | ||
| abort(404, description="requires bill_id, bill_title, and bill_text") |
There was a problem hiding this comment.
Note: currently this API takes a json payload like
{
"bill_id": "...",
"bill_title": "...",
"bill_text": "..."
}In the future, since this has access to firebase, you could also offer an API which takes the bill_id and returns the summary after looking up the title/text.
| if summary["status"] in [-1, -2]: | ||
| abort(500, description="Unable to generate summary") |
There was a problem hiding this comment.
The status field returns either -1 or -2 if it fails... This is probably somewhat unfortunate here because if this API ever changes I will not be able to know about it.
I'll add a note to get_summary_api_function but we don't really have types.
There was a problem hiding this comment.
Or, maybe I should just check that this value is negative? technically the return type is int
| if tags["status"] in [-1, -2]: | ||
| abort(500, description="Unable to generate tags") |
There was a problem hiding this comment.
Probably check for negative return type? See above https://github.com/codeforboston/maple/pull/1789/files#r2055086855
DO NOT MERGE
firebase/firebase-tools#5873
Summary
Add a short summary of the changes, and a reference to the original issue using
#and the issue number, like #1Checklist
Screenshots
Add some screenshots highlighting your changes.
Known issues
If you've run against limitations or caveats, include them here. Include follow-up issues as well.
Steps to test/reproduce
For each feature or bug fix, create a step by step list for how a reviewer can test it out. E.g.: