Glossary of Scoring Terms

 

 

 

SAMPLE

Scoring Guide

 

The NYSESLAT scoring guide contains a description of the scoring process, rubrics, exemplars, and practice responses. The guide must be used by raters to ensure that every student's response is rated the same way throughout the state.

The constructed response items on the NYSESLAT test are scored holistically, which means that a student’s work is evaluated for its total, overall, or whole effect based on the rubric and accompanying exemplar responses.

Holistic scoring is similar to learning a new language or a new way of thinking, and it is crucial that all raters put aside their own beliefs, ideas, and theories about how to evaluate students’ work. For any large scale scoring project to be successful and have meaningful results, all raters must score using the established criteria.

Before beginning to score, raters need to understand and internalize the criteria as exemplified by the rubric, along with student exemplars for each score.

Accurate scoring comes from using the Scoring Guide effectively—the rubric description for a particular score point should always be referenced in conjunction with the exemplars for that score point. The exemplary responses act to elaborate upon the rubric and help the raters to interpret them correctly. The student exemplars can be used effectively for reference and comparison.

 

  1. Description of the Scoring Process

Follow your school's procedures for training raters. This process should include:

Introduction to the constructed response—

Introduction to the rubric and exemplars—

Practice scoring individually—

  1. When actual rating begins, each rater should record his or her individual rating for a student's

response on the rating sheet provided, not directly on the student's response or answer sheet. The rater should not correct the student's work by making insertions or changes of any kind.

3. Each writing test must be rated by at least two raters. One rater will rate one question, and the other will rate two questions. The speaking test will be rated by the teacher administering the test.

  1. General information—
  2. Rating test papers—

 

6. Sensitive papers. Occasionally, a teacher will read a student response that reveals a sensitive issue. A sensitive response would include:

The rater should score the response according to the ordinary rules. Then the rater should immediately notify the school principal.

 

Using Rubrics to Score

  1. The criteria in a rubric must be clearly described. The criteria consist of a fixed scale and a list of characteristics. A rubric acquires meaning in relation to student work.
  2. The two most frequently used rubric types are
  1. Holistic which considers a performance as a whole
  2. Analytic which examines a performance by breaking it into component parts

NYSESLAT will be scored with holistic rubrics.

  1. Scores are determined by matching evidence from the response to the language of the rubric.
  2. Rubric scoring is the introduction of objectivity into what might be a largely subjective process.
  3. Training to score is not the time to critique the test questions, the rubric, or the scoring decisions. The purpose of the training is to learn to apply the scoring criteria, not to give opinions of how to alter the test or the criteria.
  4. Use rubrics as an asset model rather than a deficit model. That is what's there, rather than what's missing.
  5. As you read a response, think in the rubric language, not in percentile language.
  6. Don’t be fooled by context. A good response may appear great if read or heard after weak responses.
  7. The highest point on a rubric scale does not measure a perfect response. Each point on a rubric scale measures a body of evidence, not a single response. Thus, responses at a certain rubric point, as a whole, show the characteristics described.
  8. Be aware of what rubric language is used to indicate the differences between adjacent score points.
  9. Responses can be high, middle or low in a score point (a high 5, a low 3, etc).
  10. On the written tests, handwriting does not count.
  11. Rubrics are powerful tools to improve performance and instruction.