tester update

MikeGu721 · Jun 28, 2023 · 9f0f65d · 9f0f65d
1 parent f1d0e24
commit 9f0f65d
Show file tree

Hide file tree

Showing 7 changed files with 789 additions and 406 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,3 +1,8 @@
 ./venv
 ./resources/data/xiezhi_all
-.idea
+.idea
+./Tester/tasks/ceval/
+./Tester/tasks/ceval_train/
+./Tester/tasks/m3ke/
+./Tester/tasks/mmlu/
+./Tester/tasks/mmlu_train/
diff --git a/README.md b/README.md
@@ -14,6 +14,7 @@ We hope Xiezhi could help developers track the progress and analyze the importan
 - [Leaderboard](#leaderboard)
 - [Experiment Setting](#experiment-setting)
 - [Data](#data)
+- [How To Run Your Own Test](#how-to-run-your-own-test)
 - [TODO](#todo)
 - [Licenses](#licenses)
 - [Citation](#citation)
@@ -61,6 +62,9 @@ Example of question in Xiezhi Interdiscipline:
 Example of our few-shot learning setting:
 <p align="center"> <img src="resources/question-3shot.png" style="width: 100%;" id="question-3shot"></p>
 
+## How To Run Your Own Test
+- 
+
 ## TODO
 - [ ] add results of traditional 4 options experiments setting
 - [ ] add results of more API-based models

diff --git a/Tester/model_test.py b/Tester/model_test.py
diff --git a/resources/data/xiezhi_inter/xiezhi.v1.json → Tester/tasks/xiezhi_inter/xiezhi.v1.json b/resources/data/xiezhi_inter/xiezhi.v1.json → Tester/tasks/xiezhi_inter/xiezhi.v1.json
diff --git a/resources/data/xiezhi_spec/xiezhi.v1.json → Tester/tasks/xiezhi_spec/xiezhi.v1.json b/resources/data/xiezhi_spec/xiezhi.v1.json → Tester/tasks/xiezhi_spec/xiezhi.v1.json
diff --git a/resources/data/xiezhi_train/xiezhi.v1.json → Tester/tasks/xiezhi_train/xiezhi.v1.json b/resources/data/xiezhi_train/xiezhi.v1.json → Tester/tasks/xiezhi_train/xiezhi.v1.json
diff --git a/Tester/test.sh b/Tester/test.sh
@@ -0,0 +1 @@
+python -u model_test.py