|
| 1 | +# Code Evaluation Tool |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +The [Code Evaluation Tool]( https://microbit.makecode.com/--eval) is a mechanism for constructing a checklist of requirements for an assignment and running that list automatically against projects in quick succession. This allows teachers to build a checklist, then easily evaluate any number of projects based on that checklist. Projects are evaluated one at a time, but with auto-run enabled, you can update the loaded project by providing a new share link, at which point the rules will automatically be re-run on the new project. |
| 6 | + |
| 7 | +## Code Evaluation Tool Features |
| 8 | + |
| 9 | +### Creating, Editing, and Running a Checklist |
| 10 | + |
| 11 | +#### 1. Creating a new checklist |
| 12 | + |
| 13 | +Create a new checklist using the **New Checklist** card. If there is already an "in progress" checklist, a warning will appear asking if it is okay to overwrite it. |
| 14 | + |
| 15 | + |
| 16 | + |
| 17 | + |
| 18 | + |
| 19 | +#### 2. Naming a checklist |
| 20 | + |
| 21 | +The checklist is given a name. |
| 22 | + |
| 23 | + |
| 24 | + |
| 25 | +#### 3. Add Criteria |
| 26 | + |
| 27 | +One or more **_criteria_** are added from the catalog using the **Add Criteria** button. |
| 28 | + |
| 29 | + |
| 30 | + |
| 31 | +Some criteria (like `[block] used [count] times`) can be added multiple times, others (like `Read a GPIO pin` can only be added once). |
| 32 | + |
| 33 | + |
| 34 | + |
| 35 | +#### 4. Fill in Parameters |
| 36 | + |
| 37 | +Parameters for the criteria item are filled in for a criteria item. |
| 38 | + |
| 39 | +### ~ tip |
| 40 | + |
| 41 | +#### Parameter types |
| 42 | + |
| 43 | +From a technical perspective, criteria parameters have these types: |
| 44 | + |
| 45 | +- **Numeric** parameters have a small input and only allow number inputs. |
| 46 | +- **String** parameters can have medium and long sized inputs. |
| 47 | +- **Block** parameters should open a block-picker modal. |
| 48 | +- **Empty** parameters appear in an error state until they have values. |
| 49 | + |
| 50 | +### ~ |
| 51 | + |
| 52 | +Here a block is selected and used 3 times: |
| 53 | + |
| 54 | + |
| 55 | + |
| 56 | +Parameter options are displayed and then selected. |
| 57 | + |
| 58 | + |
| 59 | + |
| 60 | + |
| 61 | + |
| 62 | +#### 5. Ask AI |
| 63 | + |
| 64 | +You can also have an **Ask AI** question as a criteria item in the checklist. You are limited to up to 5 Ask AI questions per checklist. |
| 65 | + |
| 66 | + |
| 67 | + |
| 68 | +#### 6. Remove Criteria |
| 69 | + |
| 70 | +A criteria item is removed using the **trash** button. |
| 71 | + |
| 72 | + |
| 73 | + |
| 74 | +#### 7. Load a project |
| 75 | + |
| 76 | +Load a project into the project view by pasting in a share link or share ID. |
| 77 | + |
| 78 | + |
| 79 | + |
| 80 | +The project will load in read-only mode with the project title appearing at the top of the project view. |
| 81 | + |
| 82 | + |
| 83 | + |
| 84 | +#### 8. Run the checklist |
| 85 | + |
| 86 | +With a project loaded, the checklist can run. The results are shown after clicking the **Run** button. |
| 87 | + |
| 88 | + |
| 89 | + |
| 90 | +The results view lists each criteria with its outcome. |
| 91 | + |
| 92 | + |
| 93 | + |
| 94 | +**Note**: the **Run** Button is disabled without loaded project. |
| 95 | + |
| 96 | +### Editing Results |
| 97 | + |
| 98 | +#### 1. Add feedback and notes |
| 99 | + |
| 100 | +Feedback and notes are added using the **Add Notes** button. The feedback box should resize to fit its content as notes are added. The original feedback remains even if you re-run the rules using the **Run** button. |
| 101 | + |
| 102 | + |
| 103 | + |
| 104 | + |
| 105 | + |
| 106 | + |
| 107 | + |
| 108 | +#### 2. Edit outcomes |
| 109 | + |
| 110 | +An outcome is edited using the provided dropdown. |
| 111 | + |
| 112 | + |
| 113 | + |
| 114 | +The new selected outcome. |
| 115 | + |
| 116 | + |
| 117 | + |
| 118 | +### Result Clearing and Auto-Run |
| 119 | + |
| 120 | +#### 1. Toggling Auto-run |
| 121 | + |
| 122 | +Auto-run is toggled either **on** or **off** using the button in the menu. |
| 123 | + |
| 124 | + |
| 125 | + |
| 126 | +#### 2. Auto-run disabled |
| 127 | + |
| 128 | +If auto-run is **disabled**, a result's outcome (i.e. "Looks good", "Needs work", etc...) is set to "Not started" automatically if any of any of the following conditions are met: |
| 129 | + |
| 130 | +- It is newly added (defaults to the "Not Started" state). |
| 131 | +- A parameter in a rule is changed (only the affected rule enters the "Not Started" state). |
| 132 | +- The loaded project changes (all rules are be set to "Not started"). |
| 133 | + |
| 134 | +#### 3. Auto-run enabled |
| 135 | + |
| 136 | +If auto-run is **enabled**, any rules that enter the "Not started" state due to the conditions listed above are immediately and automatically re-run with their results updated. |
| 137 | + |
| 138 | +### Loading/Importing/Exporting Checklists |
| 139 | + |
| 140 | +#### 1. Pre-built checklists |
| 141 | + |
| 142 | +There are pre-built checklists are available on the welcome page. If a selected checklist is already in-progress, an overwrite confirmation prompt is given. |
| 143 | + |
| 144 | + |
| 145 | + |
| 146 | +#### 2. Export a checklist |
| 147 | + |
| 148 | +A checklist is exported using the vertical "..." menu near the "auto-run" button. Only the checklist is exported, not the results. For a copy of the results, use the [print](#other) function in the Results view. |
| 149 | + |
| 150 | + |
| 151 | + |
| 152 | +This will download a json file for the checklist. |
| 153 | + |
| 154 | + |
| 155 | + |
| 156 | +#### 3. Import a checklist |
| 157 | + |
| 158 | +User can import a checklist from a file using the same "..." menu, or from the card on the welcome page. |
| 159 | + |
| 160 | + |
| 161 | + |
| 162 | + |
| 163 | + |
| 164 | +Checklist file is selected using "Browse" or dropped directly into the popup. An overwrite confirmation prompted if there is currently an in-progress checklist. |
| 165 | + |
| 166 | + |
| 167 | + |
| 168 | + |
| 169 | + |
| 170 | +### Other |
| 171 | + |
| 172 | +If the page is refreshed (or if the browser closes/re-opens), the current checklist preserved. |
| 173 | + |
| 174 | +Use the print button to create a version of the results with the outcomes and feedback visible (the other UI elements are hidden). |
| 175 | + |
| 176 | + |
| 177 | + |
| 178 | +The checklist-view/project-view splitter can be resized. It can also be reset to 50/50 split with double-click. |
| 179 | + |
| 180 | + |
| 181 | + |
| 182 | +Slide the splitter to widen the view of the criteria and results. |
| 183 | + |
| 184 | + |
0 commit comments