PACE is pleased to announce the launch of the Question Set Certification Program (QSCP) for the 2012-2013 quizbowl season. This program’s aim is to improve the overall quality of regular-difficulty and novice-difficulty house-written high school quizbowl sets by recognizing the top such sets written during the course of the year. PACE will use the following criteria to judge sets:
Evaluation Criteria
1. Difficulty control – is this set at the announced difficulty or easier? Does it avoid excessively hard material? Regardless of its stated difficulty can a wide range of teams score points on it (with most averaging several tossups per game and 10 or more points per bonus)?
2. Length control – Do these questions have enough clues to properly distinguish teams without being longer than the announced length or violating reasonable expectations?
3. Pyramidality and clue quality – Do these tossups use significant, uniquely-identifying, factually correct clues that high schoolers can know, in the order that high schoolers are likely to know them? Do these bonuses have an easy, middle, and harder part which are all possible and significant? Does it show through this set’s questions that more went into this set than cursory searches of previous packets and Wikipedia?
4. Comprehensibility and legibility – Does this set use complete, grammatical English sentences? Do these questions make sense to a normal English speaker when read aloud?
In order to submit a set for certification, send the entire set to PACE President Mike Bentley at mbentleypace@gmail.com. PACE will then provide a timely rating on whether the set is an “exemplary” set or not. You can also send partial sets to this address, but this will be for feedback purposes only—PACE will only judge a finalized, complete set.
PACE will recognize sets it has judged to be “exemplary” on its website, on the hsquizbowl forums, and at the awards ceremony of the NSC.
This service will be of greatest use to both question writers and teams if the sets are submitted to PACE ahead of the first mirror of a tournament. As such, PACE encourages (but does not require) question writers to submit their finalized sets at least two weeks before the initial mirror of a tournament. This will give the PACE certification committee enough time to thoroughly go through the set and offer commentary; it will give question writers enough time to make any needed changes to the set; and it will give tournament directors time to advertise the quality of the set (should it be deemed excellent).
Frequently Asked Questions:
Q: Will PACE be judging specific NAQT or HSAPQ sets?
A: No. PACE has confidence that NAQT and HSAPQ sets meet the above criteria and are thus “exemplary”.
Q: Does PACE offer any rankings besides “exemplary”?
A: Not at this time.
Q: Do sets have to be judged “exemplary” in order to meet the bar for “good” or “excellent” as specified in the qualification guidelines for NSC affiliated tournaments?
A: No. It is important for tournament directors to know ahead of time how many teams will qualify for the NSC, and the above tournament certification process will generally take place after a tournament has happened. Thus, PACE will pass a separate, less rigorous judgment on house-written sets for the purposes of determining if the set meets the bar for a “good” or “excellent” affiliated tournament. Any tournament that has been judged to be exemplary through the QSCP process will automatically meet the bar for an “excellent” tournament for certification purposes.
Q: Will PACE be judging collegiate or middle school sets?
A: No. This program extends only to sets written for high school teams (even if high school teams play end up playing on those sets).
Q: Will PACE be judging novice high school sets?
A: Yes. These sets will be judged similarly to regular difficulty sets, only with extra attention given to elements like difficulty control and length control.
Q: Can a set targeted for upper-echelon high school teams earn an “exemplary” rating?
A: PACE will only be judging sets written to be regular difficulty or novice sets, and thus “nationals prep” tournaments will not be certified as “exemplary”. PACE is still happy to provide feedback on these sets.
Q: Will PACE re-examine a set that has changed after it has been mirrored?
A: PACE will be happy to provide feedback on changes made to a set, but the judgment of whether a set is deemed excellent or not will be based on the initial version of the set. The reason for this is to encourage hosts to get the set right the first time.
Q: Can I still get feedback on my set, even if it’s not completely done yet?
A: Yes, PACE will provide feedback and guidance on partial sets. However, PACE will not issue a final judgment on a set until it is finished.
Q: Will PACE provide feedback for questions I’m writing on my own for non-high school tournaments (i.e. an ACF Fall packet)?
A: No, this program is just for people writing announced high school tournaments. If you want general question writing feedback, please consider the ACF Writing Feedback program.
Q: What if a set meets some, but not all, of the above criteria?
A: PACE will use its discretion when judging whether a set if excellent or not, but in general, sets must meet all of the above criteria in order to be deemed excellent.
Q: What are PACE’s motivation for this set certification program?
A: PACE hopes that this program will increase the overall quality of questions used at high school quizbowl tournaments by providing constructive feedback to question writers, and by specially recognizing the best question sets.