Skip to content

Commit 954fa75

Browse files
techmaharajtechmaharaj
authored
Add Recipe Generator sample application to examples (#84)
* Add Recipe Generate sample application
1 parent 635f704 commit 954fa75

File tree

8 files changed

+360
-0
lines changed

8 files changed

+360
-0
lines changed

‎examples/recipegenerator/README.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Recipe Generator Application
2+
3+
## Overview
4+
5+
This Flask application leverages GPTScript and GPTScript Vision to suggest recipes based on an image uploaded by the user. By analyzing the image, the application can identify ingredients and propose a suitable recipe.
6+
7+
## Features
8+
9+
- Image upload functionality.
10+
- Automatic ingredient recognition from images.
11+
- Recipe suggestion based on identified ingredients.
12+
13+
## Installation
14+
15+
### Prerequisites
16+
17+
- Python 3.8 or later
18+
- Node.js and npm
19+
- Flask
20+
- Other Python and Node.js dependencies listed in `requirements.txt` and `package.json` respectively.
21+
22+
### Steps
23+
24+
1. Clone the repository:
25+
26+
``` bash
27+
git clone https://github.com/gptscript-ai/gptscript.git
28+
```
29+
30+
2. Navigate to the `examples/recipegenerator` directory and install the dependencies:
31+
32+
Python:
33+
34+
```bash
35+
pip install -r requirements.txt
36+
```
37+
38+
Node:
39+
40+
```bash
41+
npm install
42+
```
43+
44+
3. Setup `OPENAI_API_KEY` (Eg: `export OPENAI_API_KEY="yourapikey123456"`). You can get your [API key here](https://platform.openai.com/api-keys).
45+
46+
4. Run the Flask application using `flask run` or `python app.py`
47+
48+
## Usage
49+
50+
1. Open your web browser and navigate to `http://127.0.0.1:5000/`.
51+
2. Use the web interface to upload an image with some grocery items.
52+
3. The application will process the image, identify potential ingredients, and suggest a recipe based on the analysis.
53+
4. View the suggested recipe, try it and let us know how it turned out to be!
54+
55+
## Under the hood
56+
57+
Below are the processes that take place when you execute the application:
58+
59+
- The Python app places the uploaded image as `grocery.png` in the current working directory.
60+
- It then executes `recipegenerator.gpt` which internally calls `tools.gpt` to perform image analysis to identify the items from the uploaded image.
61+
- The identified ingredients from the image will be stored in a `response.json` file.
62+
- The recipegenerator will then read this response file, generate a recipe and add it to a recipe.md file.

‎examples/recipegenerator/app.py

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
from flask import Flask, request, render_template, jsonify
2+
import subprocess
3+
import os
4+
5+
app = Flask(__name__)
6+
7+
# Setting the base directory
8+
base_dir = os.path.dirname(os.path.abspath(__file__))
9+
app.config['UPLOAD_FOLDER'] = base_dir
10+
11+
SCRIPT_PATH = os.path.join(base_dir, 'recipegenerator.gpt')
12+
GROCERY_PHOTO_FILE_NAME = 'grocery.png' # The expected file name
13+
RECIPE_FILE_NAME = 'recipe.md' # The output file name
14+
15+
@app.route('/', methods=['GET'])
16+
def index():
17+
return render_template('index.html')
18+
19+
@app.route('/upload', methods=['POST'])
20+
def upload_file():
21+
if 'file' not in request.files:
22+
return jsonify({'error': 'No file part'}), 400
23+
file = request.files['file']
24+
if file.filename == '':
25+
return jsonify({'error': 'No selected file'}), 400
26+
if file:
27+
filename = os.path.join(app.config['UPLOAD_FOLDER'], GROCERY_PHOTO_FILE_NAME)
28+
file.save(filename)
29+
try:
30+
# Execute the script to generate the recipe
31+
subprocess.Popen(f"gptscript {SCRIPT_PATH}", shell=True, stdout=subprocess.PIPE, cwd=base_dir).stdout.read()
32+
33+
# Read recipe.md file
34+
recipe_file_path = os.path.join(app.config['UPLOAD_FOLDER'], RECIPE_FILE_NAME)
35+
with open(recipe_file_path, 'r') as recipe_file:
36+
recipe_content = recipe_file.read()
37+
38+
# Return recipe content
39+
return jsonify({'recipe': recipe_content})
40+
except Exception as e:
41+
return jsonify({'error': str(e)}), 500
42+
43+
if __name__ == '__main__':
44+
app.run(debug=False)

‎examples/recipegenerator/index.js

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
import { Command, Option } from 'commander';
2+
import { fileTypeFromBuffer } from 'file-type';
3+
import { URL } from 'whatwg-url';
4+
import fs from 'fs';
5+
import OpenAI from 'openai';
6+
7+
8+
async function main() {
9+
const program = new Command();
10+
11+
program.description('Utility for processing images with the OpenAI API');
12+
13+
program.addOption(new Option('--openai-api-key <key>', 'OpenAI API Key')
14+
.env('OPENAI_API_KEY')
15+
.makeOptionMandatory()
16+
);
17+
18+
program.addOption(new Option('--openai-base-url <string>', 'OpenAI base URL')
19+
.env('OPENAI_BASE_URL')
20+
);
21+
22+
program.addOption(new Option('--openai-org-id <string>', 'OpenAI Org ID to use')
23+
.env('OPENAI_ORG_ID')
24+
);
25+
26+
program.addOption(new Option('--max-tokens <number>', 'Max tokens to use')
27+
.default(2048)
28+
.env('MAX_TOKENS')
29+
);
30+
31+
program.addOption(new Option('--model <model>', 'Model to process images with')
32+
.env('MODEL')
33+
.choices(['gpt-4-vision-preview'])
34+
.default('gpt-4-vision-preview')
35+
);
36+
37+
program.addOption(new Option('--detail <detail>', 'Fidelity to use when processing images')
38+
.env('DETAIL')
39+
.choices(['low', 'high', 'auto'])
40+
.default('auto')
41+
);
42+
43+
program.argument('<prompt>', 'Prompt to send to the vision model');
44+
45+
program.argument('<images...>', 'List of image URIs to process. Supports file:// and https:// protocols. Images must be jpeg or png.');
46+
47+
program.action(run);
48+
await program.parseAsync();
49+
}
50+
51+
async function run(prompt, images, opts) {
52+
let content = []
53+
for (let image of images) {
54+
content.push({
55+
type: "image_url",
56+
image_url: {
57+
detail: opts.detail,
58+
url: await resolveImageURL(image)
59+
}
60+
})
61+
}
62+
63+
const openai = new OpenAI(opts.openaiApiKey, opts.baseUrl, opts.orgId);
64+
const response = await openai.chat.completions.create({
65+
model: opts.model,
66+
max_tokens: opts.maxTokens,
67+
messages: [
68+
{
69+
role: 'user',
70+
content: [
71+
{ type: "text", text: prompt },
72+
...content
73+
]
74+
},
75+
]
76+
});
77+
78+
console.log(JSON.stringify(response, null, 4));
79+
}
80+
81+
async function resolveImageURL(image) {
82+
const uri = new URL(image)
83+
switch (uri.protocol) {
84+
case 'http:':
85+
case 'https:':
86+
return image;
87+
case 'file:':
88+
const filePath = image.slice(7)
89+
const data = fs.readFileSync(filePath)
90+
const mime = (await fileTypeFromBuffer(data)).mime
91+
if (mime != 'image/jpeg' && mime != 'image/png') {
92+
throw new Error('Unsupported mimetype')
93+
}
94+
const base64 = data.toString('base64')
95+
return `data:${mime};base64,${base64}`
96+
default:
97+
throw new Error('Unsupported protocol')
98+
}
99+
}
100+
101+
main();
102+
103+
104+

‎examples/recipegenerator/package.json

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"name": "vision",
3+
"version": "0.0.1",
4+
"description": "Utility for processing images with the OpenAI API",
5+
"exports": "./index.js",
6+
"type": "module",
7+
"scripts": {
8+
"start": "node index.js"
9+
},
10+
"bin": "index.js",
11+
"keywords": [],
12+
"author": "",
13+
"dependencies": {
14+
"commander": "^9.0.0",
15+
"file-type": "^19.0.0",
16+
"openai": "^4.28.0",
17+
"whatwg-url": "^14.0.0"
18+
}
19+
}
20+

‎examples/recipegenerator/recipegenerator.gpt

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
tools: sys.find, sys.read, sys.write, recipegenerator, tool.gpt
2+
3+
Perform the following steps in order:
4+
5+
1. Give me a list of 5 to 10 ingredients from the picture grocery.png located in the current directory and put those extracted ingredients into a list.
6+
2. Based on these ingredients list, suggest me one recipe that is quick to cook and create a new recipe.md file with the recipe
7+
8+
9+
---
10+
name: recipegenerator
11+
description: Generate a recipe from the list of ingredients
12+
args: ingredients: a list of available ingredients.
13+
tools: sys.read
14+
15+
Generate 1 new recipe based on the ingredients list

‎examples/recipegenerator/requirements.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Flask==2.0.1
2+
Werkzeug==2.2.2

‎examples/recipegenerator/templates/index.html

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="utf-8">
5+
<meta name="viewport" content="width=device-width, initial-scale=1">
6+
<title>Recipe Generator</title>
7+
<link href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-T3c6CoIi6uLrA9TneNEoa7RxnatzjcDSCmG1MXxSR1GAsXEV/Dwwykc2MPK8M2HN" crossorigin="anonymous">
8+
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/showdown.min.js"></script>
9+
<link rel="stylesheet" href="styles.css">
10+
<style>
11+
.loader {
12+
display: none;
13+
border: 4px solid #f3f3f3;
14+
border-top: 4px solid #3498db;
15+
border-radius: 50%;
16+
width: 30px;
17+
height: 30px;
18+
animation: spin 2s linear infinite;
19+
}
20+
@keyframes spin {
21+
0% { transform: rotate(0deg); }
22+
100% { transform: rotate(360deg); }
23+
}
24+
</style>
25+
</head>
26+
<body>
27+
<nav class="navbar navbar-expand-lg navbar-dark bg-dark">
28+
<div class="container">
29+
<a class="navbar-brand" href="#">Recipe Generator</a>
30+
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
31+
<span class="navbar-toggler-icon"></span>
32+
</button>
33+
<div class="collapse navbar-collapse" id="navbarSupportedContent">
34+
<ul class="navbar-nav me-auto mb-2 mb-lg-0">
35+
<li class="nav-item">
36+
<a class="nav-link active" aria-current="page" href="#">Home</a>
37+
</li>
38+
<li class="nav-item">
39+
<a class="nav-link" href="https://github.com/gptscript-ai/gptscript" target="_blank">GPTScript</a>
40+
</li>
41+
</ul>
42+
</div>
43+
</div>
44+
</nav>
45+
46+
<div class="container my-5">
47+
<h1>Recipe Generator</h1>
48+
<div class="col-lg-8 px-0">
49+
<p class="fs-8">Don't know what to do with what's in your shopping cart? Well, click a picture and upload the image to Recipe Generator that will give you interesting ideas of what you can cook from those ingredients! This is built using <a href="https://github.com/gptscript-ai/gptscript" target="_blank">GPTScript</a>.</p>
50+
</div>
51+
</div>
52+
53+
<div class="container my-5">
54+
<div class="mb-3">
55+
<form id="uploadForm">
56+
<div class="input-group">
57+
<input type="file" name="file" class="form-control" id="formFile" aria-describedby="inputGroupFileAddon04" aria-label="Upload">
58+
<button class="btn btn-outline-secondary" type="button" id="inputGroupFileAddon04" onclick="uploadFile()">Upload File</button>
59+
</div>
60+
</form>
61+
</div>
62+
<div id="loader" class="loader"></div>
63+
<div id="recipeOutput"></div>
64+
<script>
65+
document.addEventListener('DOMContentLoaded', function() {
66+
// Define uploadFile globally
67+
window.uploadFile = function() {
68+
var form = document.getElementById('uploadForm');
69+
var formData = new FormData(form);
70+
var loader = document.getElementById('loader');
71+
var recipeOutput = document.getElementById('recipeOutput');
72+
loader.style.display = 'block'; // Show the loader
73+
74+
fetch('/upload', {
75+
method: 'POST',
76+
body: formData,
77+
})
78+
.then(response => response.json()) // Parse the JSON response
79+
.then(data => {
80+
loader.style.display = 'none'; // Hide the loader
81+
if(data.recipe) {
82+
var converter = new showdown.Converter()
83+
var parsedHtml = converter.makeHtml(data.recipe);
84+
recipeOutput.innerHTML = parsedHtml; // Display the recipe
85+
} else if (data.error) {
86+
recipeOutput.innerHTML = `<p>Error: ${data.error}</p>`;
87+
}
88+
})
89+
.catch(error => {
90+
console.error('Error:', error);
91+
loader.style.display = 'none';
92+
recipeOutput.innerHTML = `<p>Error: ${error}</p>`;
93+
});
94+
};
95+
});
96+
</script>
97+
</div>
98+
99+
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.bundle.min.js" integrity="sha384-C6RzsynM9kWDrMNeT87bh95OGNyZPhcTNXj1NW7RuBCsyN/o0jlpcV8Qyq46cDfL" crossorigin="anonymous"></script>
100+
<script src="main.js"></script>
101+
</body>
102+
</html>

‎examples/recipegenerator/tool.gpt

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
Name: vision
2+
Description: Analyze a set of images using a given prompt and vision model.
3+
Args: detail: Fidelity to process images at. One of ["high", "low", "auto"]. Default is "auto".
4+
Args: max-tokens: The maximum number of tokens. Default is 2048.
5+
Args: model: The name of the model to use. Default is "gpt-4-vision-preview".
6+
Args: prompt: The text prompt based on which the GPT model will generate a response.
7+
Args: images: Space-delimited list of image URIs to analyze. Valid URI protocols are "http" and "https" for remote images, and "file" for local images. Only supports jpeg and png.
8+
9+
#!/bin/bash
10+
11+
node index.js "${PROMPT}" ${IMAGES}

0 commit comments

Comments
 (0)