Testing changes made to Flubaroo
This page describes instructions for testing Flubaroo from
end-to-end. You don't need to follow all of these steps anytime you make a change, but you should at least test those that are related to the feature you're working on.
However, these steps must be followed completely before a new version of Flubaroo is published to the Gallery. This is necessary to make sure that new features don't break
existing functionality.
Setup
Before doing any testing, do this setup:
- Open the Flubaroo QA Spreadsheet (read-only).
- Paste
the contents of "Student Submissions" into a blank worksheet in your
own development environment, where your modified Flubaroo script lives.
- Grade
the data you pasted (you'll have to rename that blank worksheet to
"Student Submissions"). Use these options when grading:
- Q1: 3 points
- Q10: 4 points
- Q24: Skip Grading
- Q25: 5 points
- All other questions: 1 point (except student identifiers)
- Row 3 (with Full Name of "Answer Key") is the answer key to be used in Step 2.
1. Testing Grading / Regrading:
You'll
want to compare the results of your grading "Correct Grades" sheet,
which is also in the Flubaroo QA spreadsheet. This sheet has been
manually checked to be correct. Spend time to ensure that the 2
spreadsheets have identical content, including:
- Summary at top (number of submissions, average score, etc)
- Which rows have red font, for low scoring students.
- Which questions have overall low scores, with orange highlighting.
- Student(s) with more than one submission.
- Individual scores of questions (check a few submissions, at least).
Any
deviations b/n the 2 spreadsheets likely indicate that something you
did broke grading. An easy way to check all this is to open the 2
spreadsheets in adjacent tabs, and toggle back and forth to look for
differences.
2. Testing Reporting:
- Select "View Report" in the
Flubaroo menu in your dev environment. Compare the summary info and
histogram to the image stored in the "Correct Report" tab of the
Flubaroo QA spreadsheet. Make sure they match each other identically.
- Also,
in your report, count the number of total submissions from the
histogram (by reading off the vertical axis), and make sure it matches
what's in the summary of the report and spreadsheet.
- Email yourself the report, and make sure that the email:
- Has matching summary info and histogram.
- Has a working link to open the spreadsheet.
3. Testing Emailing of Grades:
You should send emails to a minimum of 4 different
students. You can just add email addresses manually in the Grades sheet,
depending on what accounts you have access to. An easy option is to
have all emails go to the same account by using the "+" operator in
gmail (e.g. dabouav+fltest1@gmail.com, dabouav+fltest2@gmail.com, etc).
Pick students that have a variety of scores (one near perfect, one with
most answers wrong, one in the middle).
Send emails 3 times, with these options for each:
- Try without list of questions and scores sent, and without any instructor message.
- Try with list of questions and scores, without answer key, and without any instructor message.
- Try with list of questions and scores, with answer key, and with multi-line instructor message.
For each email received:
- Ensure that the information matches what's in the Grades sheet, such as the student's personal info and their total score.
- For #3, ensure that the instructor's message is present.
- In particular for cases 2 and 3, make sure that:
- The "Your Answer" column matches that student's entry in the Grades sheet.
- The individual scores for each question match the Grades sheet, and have the proper color (green, red, or gray).
- The "Correct Answer" column matches the answer key chosen.
Also check that you receive an email receipt of the grading, with correct info and working links.
4. Testing "About Flubaroo" and Version:
- Call up "About Flubaroo" and check for the correct version number.