WIP: 84 a solid version controlled copy of the prompts #86

liniiiiii · 2024-08-22T11:33:56Z

I need support for formatting the prompt codes to a package that can be implemented directly in .py file

/Ni

i-be-snek · 2024-08-23T10:10:03Z

.gitignore

@@ -9,6 +9,9 @@ results
 # ignore excel files
 **.xlsx

+# ignore openai api keys
+**.env


Good thinking :)

i-be-snek · 2024-08-23T10:14:20Z

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

+    info_box = str(item.get("Info_Box"))
+    Whole_text = process_whole_text(item)
+
+    prompt_building_damage_country_0715 = f"""Based on the provided article {info_box} {Whole_text},


Tip: This prompt never changes! So if you create a "template" for it in python to format, there would be no reaosn to add it to the for-loop like this.

i-be-snek · 2024-08-23T10:15:09Z

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

+response_gpt4o = []
+
+for item in data:
+    Event_ID = str(item.get("Event_ID"))


Tip: These loops are almost identical, so you can avoid repetition by having a function that does everything inside this forloop (gets the event_id and source, sets up the prompt, etc...)

i-be-snek · 2024-08-23T10:15:51Z

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

+
+# skip the multi events
+
+from json.decoder import JSONDecodeError


I guess you are reloading the json decoder here because you probably had this in many cells in jupyter. Now is the time to remove them.

i-be-snek · 2024-08-23T10:16:49Z

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

+    json.dump(response_gpt4o, json_file, indent=4)
+from json.decoder import JSONDecodeError
+
+response_gpt4o = []


It's good for this to also be in a function, because if you forget to reset it to [] in any of these runs, it will carry over items from the previous one.

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

+import json
+
+# Specify the file path
+file_path = input("File for prompting experiments:")


i-be-snek · 2024-08-23T10:18:27Z

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

+        answer_dict = json.loads(answer_str)
+        event_info.update(answer_dict)
+
+    except JSONDecodeError as e:


Good job with catching errors :)

i-be-snek · 2024-08-23T10:19:03Z

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

+openai.api_key = api_key
+
+
+def completion_4(prompt):


Just out of curioustiy, why is it called completion_4?

i-be-snek · 2024-08-23T10:19:41Z

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

+
+
+# Saving the results for all events to a JSON file
+with open(


Tip: This storage script is also repetitive. One recommendation would be to turn it to a small function.

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py

…r-content pair, also with the run_prompt.py file, in formatting the batch file process

liniiiiii · 2024-08-28T08:37:32Z

A general note:

Now I noticed that we have Database/Prompts but we also have Prompting/gpt4_o_experiment_1 which both contain some prompts. At this point it might be good to think about those two directories. Ultimately and ideally, prompts should go only in one easy-to-find place or directory, so it might be worth thinking about if you want to move these to either directories or maybe edit the contents of Prompting/gpt4_o_experiment_1.

As discussed, the Prompting/gpt4_o_experiment_1 will only contains the experiments, and the Database/Prompts will contain the end version of prompts and codes

i-be-snek · 2024-08-28T08:39:03Z

@liniiiiii

As discussed, the Prompting/gpt4_o_experiment_1 will only contains the experiments, and the Database/Prompts will contain the end version of prompts and codes

What is the difference between the two? It's a bit hard for me to understand.

@i-be-snek , the Prompting/gpt4_o_experiment_1 will be deleted in the end, it's for testing

i-be-snek · 2024-08-28T15:09:01Z

Database/Prompts/run_prompts.py

+        # Step 2: Load the JSON data into a Python dictionary
+        raw_text = json.load(file)
+
+    # notice that due to the different version of prompts applied, the keys may a bit different, below is the version V_3


I think it's good to only support V_3 here 🤔 since we mentioned that for V_1 (?) in the nlp4climate paper, all the prompts are in the appendix.

I will think about how to present it, because if we use the GPT4o-08-06 version, we need to define something else, and the prompt template will also change, maybe make it into separate functions, and according to the prompt version, choose to use the different processes

Could you show me an example of how these keys could differ, just to understand the problem better?
That small section can be a function on its own. It's good to try not to have a lot of repeated code.

@i-be-snek , yes, so separate functions mean that we may split the prompts into two, for example,

"affected_L1/L2": """ xxxx""" "affected_L3":"""xxx"""

then the key will change, and when we put them into the batch file, we need to append the key after the custom_id, which is better in the end to retrieve the results.

Database/Prompts/batch_output_retrivel.py

…riment versions

liniiiiii · 2024-09-04T11:26:15Z

Hi, @i-be-snek and @MurathanKurfali , I think this is finished with clear code, and we can merge to the main, thanks!

MurathanKurfali · 2024-09-05T10:15:41Z

Database/Prompts/run_prompts.py

@@ -7,7 +7,8 @@
 import openai
 from dotenv import load_dotenv

-from Database.Prompts.prompts import V_3  # change here to choose the version of prompts
+# the newest version of prompts are applied
+from Database.Prompts.prompts import V_3


you can do name the prompt dictionary you are importing as:

from Database.Prompts.prompts import V_3 as target_prompts

and use target_prompts instead V_3 later in the code. so each time you change the target prompt dictionary (say you wanted to use V_2 instead), you can only change the import and the rest of the code will not need any changes.

done! thanks!

liniiiiii added 2 commits August 20, 2024 16:03

add the prompt_affected

38a6abe

add V20240610/0715 prompts for L1-3

70f7645

liniiiiii requested a review from MurathanKurfali August 22, 2024 11:33

liniiiiii linked an issue Aug 22, 2024 that may be closed by this pull request

A solid, version-controlled copy of the prompts #84

Closed

2 tasks

i-be-snek reviewed Aug 23, 2024

View reviewed changes

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py Outdated

import json

# Specify the file path

file_path = input("File for prompting experiments:")

This comment was marked as resolved.

Sign in to view

i-be-snek reviewed Aug 23, 2024

View reviewed changes

Database/Prompts/single_prompts_L1-3/prompts_V20240715_GPT4o_V0513.py Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

update the prompts.py, in process

6191cab

i-be-snek changed the title ~~84 a solid version controlled copy of the prompts~~ WIP: 84 a solid version controlled copy of the prompts Aug 24, 2024

liniiiiii added 3 commits August 26, 2024 16:27

add the version controled prompts

6e13ac4

edit the run-prompt file

9760b9c

add the raw wikipedia articles for dev, test and full run in a heade…

b65d896

…r-content pair, also with the run_prompt.py file, in formatting the batch file process

This comment was marked as resolved.

Sign in to view

add the run_prompts.py, which is working for openai batch process

6481f72

This comment was marked as resolved.

Sign in to view

revise the code to import the prompt list

cd339d1

This comment was marked as resolved.

Sign in to view

i-be-snek reviewed Aug 28, 2024

View reviewed changes

Database/Prompts/batch_output_retrivel.py Show resolved Hide resolved

liniiiiii added 6 commits August 29, 2024 16:29

add the retrieval function

313c517

add the json command in model setting and the retrieval script

a82726f

add the batch process output from GPT4o model for the dev 70events

c6293ae

add the devtest and fullrun setting for run_prompts.py

f8dccc6

update the .gitignore file for env file

dac9179

update the run_prompts.py and batch_retrieve.py files for trace expe…

01eec60

…riment versions

liniiiiii self-assigned this Sep 4, 2024

MurathanKurfali reviewed Sep 5, 2024

View reviewed changes

change the V_3 to target_prompts

da0f1f7

liniiiiii merged commit 31510ab into main Sep 5, 2024
1 check passed

liniiiiii deleted the 84-a-solid-version-controlled-copy-of-the-prompts branch September 5, 2024 11:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: 84 a solid version controlled copy of the prompts #86

WIP: 84 a solid version controlled copy of the prompts #86

liniiiiii commented Aug 22, 2024

i-be-snek Aug 23, 2024

i-be-snek Aug 23, 2024

i-be-snek Aug 23, 2024

i-be-snek Aug 23, 2024

i-be-snek Aug 23, 2024

This comment was marked as resolved.

i-be-snek Aug 23, 2024

i-be-snek Aug 23, 2024

i-be-snek Aug 23, 2024

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

liniiiiii commented Aug 28, 2024

i-be-snek commented Aug 28, 2024 •

edited by liniiiiii

Loading

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

i-be-snek Aug 28, 2024

liniiiiii Aug 28, 2024

i-be-snek Aug 28, 2024

liniiiiii Aug 29, 2024

i-be-snek Aug 29, 2024

liniiiiii commented Sep 4, 2024

MurathanKurfali Sep 5, 2024

liniiiiii Sep 5, 2024


		# skip the multi events

		from json.decoder import JSONDecodeError



		# Saving the results for all events to a JSON file
		with open(

		openai.api_key = api_key


		def completion_4(prompt):

WIP: 84 a solid version controlled copy of the prompts #86

WIP: 84 a solid version controlled copy of the prompts #86

Conversation

liniiiiii commented Aug 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

liniiiiii commented Aug 28, 2024

i-be-snek commented Aug 28, 2024 • edited by liniiiiii Loading

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liniiiiii commented Sep 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

i-be-snek commented Aug 28, 2024 •

edited by liniiiiii

Loading