Create multiple choice quizzes using ProtoWizard CLI (#560)

* WIP

* WIP: too much at once

* WIP cleanup

* rename file

* add clean quiz.js

* add selectLesson & promptFilesReady

* delete old draft messy_quiz.js

* add createQuiz and afterQuizCreate

* Add updateQuiz to API

* improve selectLesson UX

* add pristine status and overwrite warnings

* remove old comments

* Update scripts/commands/wizard/quiz.js

Co-authored-by: Zé Bateira <[email protected]>

* incorporate review feedback

* fix typos

* Update guide to reflect mult choice creation via ProtoWizard

* change npm run serve to npm start

* WIP add jest tests for quiz

* exclude specific logs from snapshot

* add TODOs

* add addl quiz tests

* fix overwrite quiz logic

* add assertUpdatedQuiz

* WIP add assertQuiz x2 versions

* WIP fix Jest 4.3, bugs in 4.1 & 4.2

* fix: spacing on assertQuizUnknownOrder

* fix async expression in createQuiz

* update snapshots for now-passing jest 4.2

* clean up console logs and comments

* simplify logging of quiz choices

* make ProtoWizard singular & more human

* WIP: add testing docs

* update wording in logCreateLater

* Update jest/helpers/asserts.js

Co-authored-by: Zé Bateira <[email protected]>

* Update scripts/commands/wizard/quiz.js

Co-authored-by: Zé Bateira <[email protected]>

* Update src/api/modules/lessons.js

Co-authored-by: Zé Bateira <[email protected]>

* Update scripts/__tests__/protowizard/quizzes.spec.js

Co-authored-by: Zé Bateira <[email protected]>

* apply review suggestions

Co-authored-by: Zé Bateira <[email protected]>
Co-authored-by: Ze Bateira <[email protected]>

Author: 3 people

DEVELOPING_TUTORIALS.md

@@ -79,7 +79,7 @@ There are four lesson formats available, which you may mix and match within your
 
 To build any of these lesson types, you'll need to use Markdown, a fairly simple way to style text on the web. [Learn more about Markdown formatting here.](https://guides.github.com/features/mastering-markdown/)
 
-To create text-based or multiple-choice lessons, no prior coding knowledge is required. Our text-only lessons are written exclusively in Markdown, and you'll only need to do some simple text replacements in our JavaScript boilerplate to set up questions and answers for multiple-choice quizzes.
+To create text-based or multiple-choice lessons, no prior coding knowledge is required, as the content of these lessons is written in Markdown. To create the question and answer choices for a multiple-choice quiz, you can either use the ProtoWizard or make some text edits to the provided JavaScript template.
 
 However, if you want to create code challenges (with or without file upload), you'll need to use JavaScript extensively to set up your default and solution code and validation. JavaScript is a scripting language for building interactive web pages, and you should have a solid understanding of JavaScript before building a code challenge for ProtoSchool.
 
@@ -124,7 +124,7 @@ $ npm install
 5. Run the dev server locally:
 
 ```sh
-$ npm run serve
+$ npm start
 ```
 
 6. Open a web browser to the following address to preview your work: http://localhost:3000
@@ -140,7 +140,7 @@ We've included instructions in this guide for doing all of this manually (in cas
 
 #### Using the ProtoWizard CLI (recommended)
 
-The ProtoWizard is a CLI (Command-Line Interface) that makes it easy to create the starter files and metadata needed for a new tutorial. You can build your full tutorial at once, creating your tutorial metadata (URL, description, etc.), lesson files, and resources in one go. Alternatively, you can create your tutorial framework and then come back to add lessons and resources as you're ready.
+The ProtoWizard is a CLI (Command-Line Interface) that makes it easy to create the starter files and metadata needed for a new tutorial. You can build your full tutorial at once, creating your tutorial metadata (URL, description, etc.), lesson files, and resources in one go. Alternatively, you can create your tutorial framework and then come back to add lessons and resources as you're ready. You can even summon the ProtoWizard to build the quizzes for your multiple-choice lessons.
 
 It's easy to use the ProtoWizard even if you're not familiar with the command line. It will ask you a few questions and let you type your answers or select options using arrow keys. Based on your answers, the ProtoWizard will build the directory, files, and metadata you'll need to create your tutorial.
 
@@ -157,12 +157,12 @@ First, install the wizard:
 $ npm run install-protowizard
 ```
 
-Once the wizard is installed, you'll be able to launch it repeatedly with the shortcut command:
+Once the wizard is installed, you'll be able to summon it repeatedly with the shortcut command:
 ```sh
 $ protowizard
 ```
 
-If you choose not to install it, you'll need to use this longer command to run the ProtoWizard:
+If you choose not to install it, you'll need to use this longer command to summon the ProtoWizard:
 ```sh
 $ npm run scripts:wizard
 ```
@@ -176,7 +176,7 @@ When you run either of the commands above to start the CLI, our friendly little
 
 ![screenshot](public/protowizard.png)
 
-If you're creating a new tutorial from scratch, you'll need to create a tutorial before adding lessons or resources to it. You can either add your lessons and resources immediately after creating the tutorial or exit the ProtoWizard and run it again later to create those items.
+If you're creating a new tutorial from scratch, you'll need to create a tutorial before adding lessons or resources to it. You can either add your lessons and resources immediately after creating the tutorial or exit the ProtoWizard and run it again later to create those items. To create a multiple-choice quiz, you'll need to have first created your tutorial and lesson files.
 
 By keeping your server running, you can preview your new tutorial in a web browser while using the ProtoWizard to add content.
 
@@ -190,16 +190,22 @@ While the ProtoWizard only supports the initial creation of this data, you can e
 
 Tell the ProtoWizard the name of your lesson and its type (text-only, multiple-choice, coding challenge with or without file upload) and it will create all the starter files you need inside your tutorial's directory. ([Learn more about the files required for each lesson type.](#create-lesson-files)) The ProtoWizard will provide you with the names of the files it's created so you'll know where to go to make your edits.
 
-If you have a solid outline, you can create these files all at once before editing them to create your content. Alternatively, you can set up just the files for your first lesson, build out that content to get familiar with the process, and return to the ProtoWizard later to create your next set of files. It's up to you.
+If you have a solid outline, you can create these files all at once before editing them to create your content. Alternatively, you can set up just the files for your first lesson, build out that content to get familiar with the process, and summon the ProtoWizard again later to create your next set of files. It's up to you.
 
 This guide includes detailed instructions on how to work within those files to [create your lesson content](#create-your-lesson-content) when you're ready.
 
 **Resources**
 
-Each ProtoSchool tutorial ends with a resources page where you can share suggestions of other learning materials relevant to your learners. The ProtoWizard will ask you a few quick questions about each resource in order to add the necessary details your tutorial's metadata. You can add all of your entries in the ProtoWizard at once or come back to it as you think of more resources.
+Each ProtoSchool tutorial ends with a resources page where you can share suggestions of other learning materials relevant to your learners. The ProtoWizard will ask you a few quick questions about each resource in order to add the necessary details your tutorial's metadata. You can add all of your entries at once or come summon the ProtoWizard repeatedly as you think of more resources.
 
 If you'd like to make edits to the resources you've created through the ProtoWizard, you can edit the details later in `src/static/tutorials.json`. [Learn how to create or edit resources manually.](#manage-your-tutorials-metadata)
 
+**Multiple-choice quizzes**
+
+Each multiple-choice lesson ends with a quiz to reinforce the lesson content. With the help of the ProtoWizard, you'll draft the question the learner needs to answer, then provide answer choices and feedback positive or negative feedback to display when each option is selected.
+
+The ProtoWizard will provide you with a link to the updated quiz file so you can make further edits as needed, or you can summon the ProtoWizard again to start from scratch and overwrite the quiz content. [Learn how to create or edit quizzes manually.](#create-multiple-choice-quizzes-skip-for-coding-challenges-and-text-only-lessons)
+
 #### Manually
 
 This section describes how to build the directory, lesson files, and metadata for your tutorial manually if you've chosen not to use the recommended ProtoWizard. If you've used the [ProtoWizard](#using-the-protowizard-cli-recommended) to create your tutorial, lesson files, and resources, please skip to [Create Your Lesson Content](#create-your-lesson-content).
@@ -432,7 +438,15 @@ Then in your lesson Markdown file, you can either add the image with regular Mar
 
 #### Create multiple-choice quizzes (skip for coding challenges and text-only lessons)
 
-When creating a multiple-choice lesson, you'll use your JavaScript (eg `01.js`) file to define the question and its answer choices.
+We recommend that you [use the ProtoWizard to create your multiple-choice quizzes](#using-the-protowizard-cli-recommended).
+
+However, you may instead choose to edit your JavaScript (eg `01.js`) file directly to define the question and its answer choices, as described below.
+
+When editing manually, please be sure to remove the following line of code:
+```
+// #PRISTINE# This file was auto-generated. Please remove this line when updating the file manually.
+```
+This will allow the ProtoWizard to identify the file as having been edited, so that it can warn you if you later ask to overwrite existing content.
 
 The `question` value must be a string:
 
@@ -448,26 +462,26 @@ const question = "What's the meaning of life, the universe, and everything?"
 ```js
  const choices = [
    {
-     answer: 'An incorrect answer',
+     answer: "An incorrect answer",
      correct: false,
-     feedback: 'Oops. Here\'s some clue about why that answer is wrong.'
+     feedback: "Oops. Here's some clue about why that answer is wrong."
    },
    {
-     answer: 'A correct answer.',
+     answer: "A correct answer.",
      correct: true,
-     feedback: 'Great job!'
+     feedback: "Great job!"
    },
    {
-     answer: 'A different incorrect answer',
+     answer: "A different incorrect answer",
      correct: false,
-     feedback: 'Sorry, here\'s some clue about why this choice is wrong.'
+     feedback: "Sorry, here's some clue about why this choice is wrong."
    }
  ]
 ```
 
 Please provide 3-5 answer choices per question. **You may only provide _one_ correct choice.**
 
-The answer choices will be presented in the order in which you list them. Be sure to vary the position of the correct answer from lesson to lesson.
+If you create the quiz manually, the answer choices will be presented in the order in which you list them. Be sure to vary the position of the correct answer from lesson to lesson. (When using the ProtoWizard, your choices will be shuffled automatically.)
 
 The `feedback` provided for each choice will be shown highlighted in red if incorrect or in green if correct, and the user will be able to advance to next lesson once they've made the right selection.
 

README.md

@@ -35,20 +35,24 @@ $ git clone https://github.com/ProtoSchool/protoschool.github.io.git
 $ cd protoschool.github.io
 $ git checkout existing-branch-name-from-PR
 $ npm install
-$ npm run serve
+$ npm start
 ```
 
 View the site on localhost at: http://localhost:3000
 
 ## Testing without affecting site analytics in Countly
 
-Only actions taken at the https://proto.school domain affect our main ProtoSchool analytics in Countly. 
+Only actions taken at the https://proto.school domain affect our main ProtoSchool analytics in Countly.
 
-You can safely experiment with any of the following to send stats to our TEST ProtoSchool dashboard in Countly instead: 
+You can safely experiment with any of the following to send stats to our TEST ProtoSchool dashboard in Countly instead:
 - served locally: http://localhost:3000
 - live site accessed directly through Fleek: https://protoschool.on.fleek.co/
 - preview for a specific PR via link found in PR testing section (e.g. https://bafybeihebd2p64xghtqjuewonkfm4l5n2lkvrllviz7nb33qtyir5q5fau.on.fleek.co/)
 
+## Testing suites
+
+We use Cypress and Jest for testing. Please check the documentation [here](scripts/README.md) for further details.
+
 ## Managing remote data
 
 Some data, such as the events list, is pulled at build-time from remote sources using Node.js scripts. Please check the documentation [here](scripts/README.md) for further details.

jest/helpers/asserts.js

@@ -1,5 +1,21 @@
+// ** SNAPSHOT FILTERING ***
+// prevent snapshots for logs that change on each instance, such as shuffled quiz options
+
+const excludedPhrases = [
+  "Cool! Here's what we get when we mix up the order of the answer choices:"
+]
+
+// returns true if log includes a string from the excludedPhrases array
+function isExcluded (log) {
+  return excludedPhrases.some(phrase => log.includes(phrase))
+}
+
+// add log to snapshot UNLESS it contains an excluded phrase
 function assertLogSnapshot (log) {
-  expect(log).toMatchSnapshot()
+  if (!isExcluded(log)) {
+    expect(log).toMatchSnapshot() // expect if log isn't excluded from assertion
+  }
+  // do nothing if log is excluded from assertion
 }
 
 module.exports = { assertLogSnapshot }

jest/helpers/fixtures.js

@@ -18,7 +18,7 @@ function createTutorial (config = { override: {}, lessons: 0, resources: 0 }) {
 
 // Generators: generate data to be used
 
-function generateTutorial (config = { override: {}, lessons: 0, resources: 0 }) {
+function generateTutorial (config = { override: {}, lessons: 0, resources: 0, lessonOverride: {} }) {
   const suffix = api.tutorials.getNextTutorialId()
 
   const tutorial = {
@@ -30,7 +30,12 @@ function generateTutorial (config = { override: {}, lessons: 0, resources: 0 })
   }
 
   const lessons = new Array(config.lessons).fill().map((_, i) => (
-    generateLesson({ override: { title: `Lesson ${i + 1}` } }).lesson
+    generateLesson({
+      override: {
+        title: `Lesson ${i + 1}`,
+        ...config.lessonOverride
+      }
+    }).lesson
   ))
 
   const resources = new Array(config.resources).fill().map((_, i) => (

public/protowizard.png

@@ -25,6 +25,24 @@ To update one or more ToCs on demand (without automatically saving), you can run
   - `npm run build:docs:develop` - update TOC in DEVELOPING_TUTORIALS.md
   - `npm run build:docs` - update ToC in both files
 
+## Testing
+We use a number of test suites to ensure the best possible user experience. These tests are run automatically for each pull request and build.
+
+When submitting a PR for a new feature, please include updated tests as well. You can use the following tools and commands to build your tests.
+
+### Cypress
+Cypress provides end-to-end testing that mimics the user's experience with our interface.
+
+TODO: add link to docs, most common commands and flags, incl how to keep window open, whether or not screenshots need to be committed, etc.
+
+
+### Jest
+Jest is used for snapshot testing of our CLI, the ProtoWizard.
+
+TODO: add link to docs, most common commands and flags, incl when and how to update (and when not to update) snapshots
+
+
+
 ## Event and Newsletter Data Import
 
 This project depends on some remote sources for data:

scripts/README.md

@@ -1,3 +1,5 @@
+const fs = require('fs')
+
 // Used with generateTutorial
 function assertNewTutorial ({ context, result, expected }) {
   expect(result).toBeInstanceOf(Object)
@@ -54,8 +56,28 @@ function assertNewResource ({ result, expected }) {
   expect(result.resources[0]).toMatchObject(expected.resource)
 }
 
+function assertQuizUnknownOrder ({ result, hardcodedData }) {
+  const fileContents = fs.readFileSync(result.files.js, 'utf8')
+  expect(fileContents).toContain(`question = "${hardcodedData.question}"`)
+  hardcodedData.choices.forEach(choice => {
+    expect(fileContents).toContain(`{
+    "answer": "${choice.answer}",
+    "correct": ${choice.correct},
+    "feedback": "${choice.feedback}"
+  }`)
+  })
+}
+
+function assertQuizKnownOrder ({ result }) {
+  const fileContents = fs.readFileSync(result.lessons[0].files.js, 'utf8')
+
+  expect(fileContents).toMatchSnapshot()
+}
+
 module.exports = {
   assertNewTutorial,
   assertNewLesson,
-  assertNewResource
+  assertNewResource,
+  assertQuizKnownOrder,
+  assertQuizUnknownOrder
 }