Creating Tests

General Guidelines

  • Keep questions concise.  The more superfluous information in a question, the more likely it is to waste students time and/or confuse them.
  • Be careful to avoid any potential biases when writing a test.  These can be cultural, racial, ethnic, sexual, or socio-economic.  For example, a question that refers to a ‘crew team’ may give an unfair advantage to wealthier students.
  • Make tests as culturally sensitive as possible. When creating hypothetical characters and situations, avoid known stereotypes or only using white male names.
  • Make test questions reflect course objectives.  Rather than focus on trivial or abstract details, create questions that deal with the important concepts.[1]
  • Avoid writing joke answers to questions.  They are a waste of valuable space and can confuse students or make them not take the rest of the exam seriously.
  • Avoid having test questions refer to each other.  By keeping questions independent of each other, test-wise students don’t gain an advantage.[2]

Validity and Reliability

  • Validity refers to the ability of a test to accurately measure what it is meant to measure.
    • For example, an algebra test that primarily covered geometric shapes would not be considered a valid test.
  • Reliability is how repeatable the test is when taken by a larger sample.
    • For example, if a test administered in three different semesters consistently has similar scores, it can be seen as reliable.  
  • When grading tests, carefully look for questions that the bulk of the class missed.  Consider removing test items that are commonly missed from later iterations of the exam.  It is likely those questions are poorly worded, misunderstood, or not strongly related to the course material.[3]
  • See also: Crafting Questions

Test Organization

  • Group like-questions together. If a test has numerous types of questions (like multiple choice, true-false, and matching) break all of the questions up by type.  At the start of each section, place the specific directions for the according question type.
  • Leave plenty of blank space on an open-response test.  By doing so, students will have ample room to work out problems and potentially show their thinking.  As well, with less clutter tests are generally easier to grade.[1]
  • If questions are worth varying point totals, label them clearly. Students will be able to clearly understand where they should spend the bulk of their time.[4]
  • Don’t split a question up over multiple pages.  If a question won’t fit on the end of one page, move the entire thing over.
  • Place the more difficult items towards the end of the test if questions are not the same relative difficulty.  By doing so, slower test-takers will cover more of the test if they should run out of time.[1]

Source Information

  1. Zimmerman, B. B., Sudweeks, R. R., Shelley, M. F., & Wood, B. (1990). How to Prepare Better Tests: Guidelines for University Faculty. Retrieved online: Department for Instructional Science, Brigham Young University Testing Services.
  2. Cohen, A. S., & Wollack, J. A. Handbook on Test Development: Helpful Tips for Creating Reliable and Valid Classroom Test. Retrieved online: Testing & Evaluation Services, University of Wisconsin-Madison.
  3. Callahan, M., & Logan, M. M. (2014). How Do I Create Tests for my Students?. Retrieved online: Teaching, Learning, & Professional Development Center, Texas Tech University.
  4. Eberly Center for Teaching Excellence, Carnegie Mellon University. Whys and Hows of Assessment: Creating Exams. Retrieved online.