Robot submissions

This script is designed to test the Robot Submissions feature. Robot users should be able to submit to bot-enabled competitions without being admitted as a participant.

Using the Scripts:

Setup:

• Create a competition with robot submissions enabled

  Example competition bundle

• Create a user and enable the bot user flag on the django admin page.

---

get_competition_details.py:

• Inside the get_competition_details.py script, configure these options:

• CODALAB_URL can be changed if not testing locally.

• Run the get_competition_details script with no arguments.

• Find the competition you want to test on.

• Run the get_competition_details script again with the competition id as the only argument.

• Find the phase you want to test on.

• If you want to use the task selection feature, run the script again with the competition id as the first argument and the phase id as the second argument.

• Select the task you would like to run your submission on.

• Use the phase id and task ids to configure example_submission.py.

---

example_submission.py:

• Inside the example_submission.py script, configure these options:

• CODALAB_URL can be changed if not testing locally.

• USERNAME and PASSWORD should correspond with the user being tested.

• PHASE_ID should correspond with the phase being tested on.

• TASK_LIST can be used to submit to specific tasks on a phase. If left blank, the submission will run on all tasks.

• SUBMISSION_ZIP_PATH should be changed if testing on anything but the default "Classify Wheat Seeds" competition. An example submission can be found here.

• Run this script in a python3 environment with requests installed.

---

get_submission_details.py

• Configure the get_submission_details.py options before running.

• CODALAB_URL can be changed if not testing locally.

• USERNAME and PASSWORD should correspond with the user being tested.

• Run the get_submission_details.py --phase <id> script with the ID of the phase containing the desired submission.

• Find the ID of the desired submission.

• Run the get_submission_details.py --submission <id> script with the desired submission ID.

  • The output of the script should be a submission object and a submission get_details object. This data can be used view scores, get prediction results, ect.

• Run the get_submission_details.py --submission <id> -v to save a zip containing previous info plus the original submission and logs.

  • --output <PATH> can be used to choose where to save the zip file. Otherwise it will be saved in the current directory.