The University of the State of New York
THE STATE EDUCATION DEPARTMENT
Albany, New York 12234

Information Booklet for Administering and Scoring
Regents Examinations in
Global History and Geography and
United States History and Government

GENERAL INFORMATION

The general procedures to be followed in administering Regents examinations are provided in the publications Regents Examinations, Regents Competency Tests and Proficiency Examinations: School Administratorís Manual, 2001 edition, and Directions for Administering and Scoring Regents Examinations. The school administratorís manual may be accessed on the Departmentís web site at http://www.p12.nysed.gov/ciai/assess.html.

Questions about general administration procedures for Regents examinations should be directed to the Office of State Assessment at 518: 474-8220 or 518: 474-5099. For information about the rating of the Regents examinations in Global History and Geography and United States History and Government, contact George Gregory or JoAnn Larson of the Office of Curriculum and Instruction at 518: 474-5922 or Gary Warren in the Office of State Assessment at 518: 474-5900.

All school personnel who will be involved in the administration and scoring of these examinations must have a copy of this information booklet. This information booklet may be photocopied.

ADMINISTERING THE EXAMINATION

Test Description

The Regents examinations in Global History and Geography and in United States History and Government assess student achievement of the social studies learning standards at the commencement level. Students are to be allowed a maximum of three hours to complete each examination.

Each examination has three parts. Students are to answer all questions in all parts; no choice is permitted.

  • Part I consists of 50 multiple-choice questions, for which the student is to select the correct answer from among the four choices given.
  • Part II contains one thematic essay.
  • Part III A contains several scaffold (open-ended) questions based on specific documents.
  • Part III B contains one document-based (DBQ) essay.

Students should use black or dark blue ink to write their answers to the essay questions and the scaffold questions. (If students are using a machine-scorable answer sheet for Part I, they will also need a No. 2 pencil.)

Test Materials

Each student should be given:

  • An examination booklet with a detachable answer sheet printed as the last page. Students are to record their answers to the multiple-choice questions on this answer sheet. The answer sheet includes a space for recording student scores for Parts I through III and for recording the studentís final score on the examination. (See page 8.)

Schools that choose to use machine-scorable answer sheets for Part I should refer to Regents Examinations, Regents Competency Tests and Proficiency Examinations: School Administratorís Manual, 2001 edition, for guidelines concerning the format and scoring of the answer sheets.

  • An essay booklet in which students are to write their final thematic and document-based essays. Essay booklets are provided to schools by the Department.
  • Scrap paper for planning essays. Scrap paper is provided by the school.

SCORING THE EXAMINATION

The Scoring Key and Rating Guide

The Scoring Key and Rating Guide contains:

  • Correct answers to the multiple-choice questions.
  • Specific scoring rubrics for both the thematic (Part II) and DBQ essays (Part III B) and the DBQ scaffold (open-ended) questions (Part III A).
  • Prescored anchor papers at each essay score level, with commentary explaining why a particular student paper was awarded that specific score.
  • Prescored practice papers.
  • A chart for converting the studentís multiple-choice (Part I) and DBQ scaffold question (Part III A) score and total essay (Part II and Part III B) score to a final examination score.

(Note: Each conversion chart is unique to a specific examination. Be sure to use the correct conversion chart for the examination you are scoring.)

Rating the Examination

The reliability of the scores is a fundamental concern in the measurement of the studentís achievement. Therefore, each essay must be scored by at least two qualified teachers. The short-answer document-based scaffold questions need only be scored by one qualified teacher. Qualified raters include teachers of grades 7Ė12 social studies and special education teachers who are knowledgeable about the Global History and Geography or United States History and Government curriculum. Raters should have previously received some school-level, district-level, or regional training on scoring social studies essays or scaffold (open-ended) questions as part of the turnkey training process.

It is recommended that schools with a small number of qualified social studies raters form a consortium of teachers to score as a group the answer papers from several schools.

In order to ensure reliable scoring, the principal of each high school administering the social studies Regents examinations must appoint a scoring coordinator who will:

  • Manage the training and logistics of the scoring process.
  • Provide task-specific training, including review of the rating guide just prior to scoring.
  • Assign two teachers to rate each essay response independently, with a third teacher available to resolve discrepant scores. (A discrepant score is one that varies by more than one point on a 5-point rubric.) Only one rater is needed for the scaffold questions. If staffing is sufficient, separate teams of teachers should rate the Part II thematic essay, the Part III A scaffold questions, and the Part III B essay.

Every effort should be made to avoid having a teacher rate his/her own studentsí responses. When this is not possible, a teacher should score no more than one part of his/her studentsí paper (i.e., a thematic essay, the scaffold questions, or a DBQ essay).

Organizing the Rating and Recording

Before student responses can be read and rated, each school must set up a procedure for collecting, arranging, and processing the answer papers and for maintaining records of the examination results. The procedure used in a particular school should be designed to produce a reliable score for each student and to facilitate maintenance of the schoolís records of each studentís score. A suggested procedure for managing the mechanics of the rating process is described on pages 4 and 5.

Scoring of Multiple-Choice Questions

Indicate by means of a checkmark each incorrect or omitted answer to multiple-choice questions on the designated answer sheet; do not place a checkmark beside a correct answer. Use only red ink or red pencil. In the appropriate space on the studentís answer sheet, record the number of multiple-choice questions the student answered correctly.

Detailed Directions for Training Raters to Score Student Responses

In training raters to score student responses for Part II and Part III of the examination, follow the procedures outlined below:

1. Introduction to the Task

The introduction to the task may take place once the administration of the examination has begun. However, the actual Scoring Key and Rating Guide for this examination may not be removed from the shrink-wrapped package of scoring keys for use by raters until after the Uniform Statewide Admission Deadline.

a. Raters read the task.

b. Raters identify answers to the task.

c. Raters discuss possible answers and summarize expectations for student responses.

2. Introduction to the Specific Rubric and Anchor Papers

The introduction to the specific rubric and anchor papers may take place once the uniform statewide deadline has passed.

  1. Trainer leads review of specific rubric with reference to the task.
  2. Trainer leads discussion of procedures for assigning holistic scores (i.e., by matching evidence from the response to the rubric).
  3. Trainer leads review of each anchor paper and commentary.

 

  1. Practice Scoring Individually


a. Raters score the practice papers independently without looking at the scores and commentaries provided after the papers.

b. Trainer records scores and leads discussion of scoring criteria until raters feel comfortable enough to move on to actual scoring.

c. If additional practice is required to reach scoring consensus, trainer may use a sample of student answer papers from the current administration of the examination.

Suggested Rating Procedure

The following procedure is recommended for managing the mechanics of the rating process. A copy of the Rating Sheet and the Record Sheet are included in the Appendix. You may photocopy as many copies as needed.

  1. The person assigned as the coordinator of the rating process, or other designated representative(s), will be responsible for coordinating the movement of papers, calculating a final score for each studentís essays, recording that information on the studentís Part I answer sheet, and determining the studentís final score for the examination.
  2. Set aside one room as a central rating room for collecting, sorting, circulating, and storing answer sheets/essay booklets and for preparing and maintaining records for these examinations.
  3. Provide a suitable location for rating of essays.
  4. Allow time to provide training for scoring the specific task for all raters immediately before the rating of the studentsí responses (about 2 hours per essay and about 30Ė45 minutes per document). It is strongly recommended that teachers be trained on one document and score those responses, then be trained and score the next documentís response(s), etc.
  5. Provide adequate time for rating (3Ė5 minutes per response for each essay, 1 minute per response for each scaffold question scored 0-2, and Ĺ minute per response for each scaffold question scored 0Ė1).
  6. For Part II and Part III B, continue with these procedures:

  7. Each essay must be scored by at least two qualified teachers. For each essay question, divide raters into two-person teams. Designate one team member as Rater 1 and the other as Rater 2. After the examination has been administered, either keep the essay booklets together and shift them between raters or separate the studentsí essay booklets into Part II and Part III B. If the essay answer booklet is separated, be sure to verify that the student has entered his/her name and the school name on the page where Part III (Document-Based Question) begins before separating the parts. After separating the essays, staple together all pages of the studentís Part II response and staple together all pages of the studentís Part III B response.
  8. Arrange the essay responses for each part according to a sequence, using whatever order is most convenient for your school, e.g., class period, alphabetical, or local identification number. Beginning with the first paper in the sequence, enter each studentís name on a copy of the Record Sheet. (Master for duplicating appears in the Appendix.)
  9. Divide each group of essays into bundles of 25 to 30 papers.
  10. Prepare a Rating Sheet for each bundle. (See sample rating sheet in the Appendix.) After recording the studentsí names on the rating sheet, photocopy the rating sheet. Each rater will need a separate rating sheet for each bundle of essay papers he/she rates. The second rater must not be aware of the score assigned by the previous rater.
  11. Distribute the bundles of essay papers to the rating teams, making sure that each rating team receives two rating sheets for each bundle of papers. Each rater on a team should rate one of the bundles and record his/her ratings on one of the rating sheets. The two raters should then exchange bundles. The second rater should only record his/her scores on the second rating sheet. No scores or corrections should be indicated on the essay papers.
  12. After each team has completed rating a bundle, the team should return those answer papers to the central rating room. Remove the rating sheets completed by each rater from the bundles and enter the scores on the Record Sheet. Make sure there are two independent ratings for each response. Enter the resolved scores in the appropriate columns on the Record Sheet.
  13. Review the two scores for each student to determine if the studentís scores for that essay are discrepant, i.e., a difference of more than one point between the two scores. Separate the studentsí responses with discrepant scores and make another bundle. Prepare a separate rating sheet for those discrepant papers. List the names of the students on a new rating sheet and attach the sheet to the corresponding bundle of student responses. Assign each of these bundles to a rater to obtain a third independent rating of the studentsí responses. Make sure that the third rater is not one of the original two raters of that task, and that the third rater has undergone the training for that task.
  14. After the necessary third ratings have been obtained, remove the rating sheets from the bundles of student responses and determine the resolved scores by using the method for resolving discrepant scores described on page 6. Enter the resolved scores in the appropriate columns on the Record Sheet.
  15. Transfer the resolved scores to the appropriate spaces on the studentsí Part I answer sheets.

For Part III A:

  1. Follow a similar procedure for processing the papers.
  2. The short-answer (open-ended) questions need only be scored by one qualified teacher.
  3. The scores for each scaffold question may be recorded in the studentsí examination booklet.
  4. Record the total Part III A score in the space provided on the studentís Part I answer sheet.

Method for Determining the Score for Each Essay

Two Ratings:

1. Compare the two ratings.

2. If the two ratings agree, the student receives that score.

3. If the two ratings are contiguous, average the two scores.

4. If the two ratings are not contiguous, a third rating is necessary.

Three Ratings:

1. Compare the three ratings.

2. If two of the three ratings agree, the student receives that score.

3. If the three ratings are different, the student receives the middle score.

Examples:

Rater 1

 

Rater 2

 

Rater 3

 

Resolved
Score*

Reason

 

2

 

2

 

ó

 

2

 

Two ratings agree. Use that score.

 

2

 

3

 

ó

 

2.5

 

Two ratings are contiguous. Average the two scores.

 

2

 

4

 

4

 

4

 

Two ratings are two or more points apart. Third rating is done. Two of the three ratings agree. Use that score.

 

2

 

4

 

3

 

3

 

Two ratings are two or more points apart. Third rating is done. Three ratings differ. Use middle score.

 

0

 

1

 

ó

 

0.5

 

Two ratings are contiguous. Average the two scores.

 

* If the final score ends in .5, do not round at this point.

Entering Essay Scores on Record Sheet

The examples below show how studentsí scores should be recorded on the Record Sheet.

 

Studentís Name

 

Part II Essay Scores

 

Part III B Essay Scores

 

Rater 1

 

Rater  2

 

Rater 3

 

Resolved Score

 

Rater 1

 

Rater 2

 

Rater 3

 

Resolved Score

 

Student A

 

4

 

4

 

ó

 

4

 

4

 

2

 

3

 

3

 

Student B

 

0

 

1

 

ó

 

0.5

 

1

 

4

 

2

 

2

 

Student C

 

3

 

5

 

4

 

4

 

4

 

3

 

ó

 

3.5

 

Determining the Studentís Final Examination Score

Record the number of correct answers for the multiple-choice questions and total points awarded for the scaffold (open-ended) questions on the appropriate lines of the studentís Part I answer paper. Add these two numbers together and write that score in the box labeled "Total Part I and Part III A Score."

Record the essay scores for Part II and Part III B on the appropriate lines. Add the two essay scores together. (If the total essay score ends in .5, that score should be rounded up to the nearest whole number at this time.) Write that score in the box labeled "Total Part II and Part III B Score."

To determine the studentís final examination score, use the chart located at the end of the Scoring Key and Rating Guide. Locate the studentís total multiple-choice and scaffold question score on the left side of the chart and the studentís total essay score across the top of the chart. The point where those two scores intersect is the studentís final examination score. The chart below illustrates the format of the chart. The chart included with the actual Scoring Key and Rating Guide will include scores ranging from 0 to 100 within the cells of the chart. Because the scaled scores corresponding to raw scores in the conversion chart and the number of scaffold questions may change from one examination administration to another, it is crucial that for each administration, the conversion chart provided in the scoring materials for that specific administration be used to determine the studentís final score.

Total Essay Score Ť    

 

 
                    Total Essay 
Score Ť
 
                      

0

 

1

 

2

 

3

 

4

 

5

 

6

 

7

 

8

 

9

 

10

 

0

 

1

 

2

 

3

 

4

 

5

 

6

 

7

 

8

 

9

 

10

 

Total Part I and Part III A Score

 

0

 

0

 
                   

Total 
Part I 
and 
Part III A Score
(continued)

 

31

 
                      

1

 
                     

32

 
                      

2

 
                     

33

 
                      

3

 
                     

34

 
                      

4

 
                     

35

 
                      

5

 
                     

36

 
                      

6

 
                     

37

 
                      

7

 
                     

38

 
                      

8

 
                     

39

 
                      

9

 
                     

40

 
                      

10

 
                     

41

 
                      

11

 
                     

42

 
                      

12

 
                     

43

 
                      

13

 
                     

44

 
                      

14

 
                     

45

 
                      

15

 
                     

46

 
                      

16

 
                     

47

 
                      

17

 
                     

48

 
                      

18

 
                     

49

 
                      

19

 
                     

50

 
                      

20

 
                     

51

 
                      

21

 
                     

52

 
                      

22

 
                     

53

 
                      

23

 
                     

54

 
                      

24

 
                     

55

 
                      

25

 
                     

56

 
                      

26

 
                     

57

 
                      

27

 
                     

58

 
                      

28

 
                     

59

 
                      

29

 
                     

60

 
                      

30

 
                     

61

 
                    

100

 

Appendix 

Rubrics
Rating Sheet
Record Sheet
 

Generic Scoring Rubric
Social Studies
Thematic Essay Thematic Essay

Score of 5:

  • Shows a thorough understanding of the theme or problem
  • Addresses all aspects of the task
  • Shows an ability to analyze, evaluate, compare and/or contrast issues and events
  • Richly supports the theme or problem with relevant facts, examples, and details
  • Is a well-developed essay, consistently demonstrating a logical and clear plan of organization
  • Introduces the theme or problem by establishing a framework that is beyond a simple restatement of the task and concludes with a summation of the theme or problem

Score of 4:

  • Shows a good understanding of the theme or problem
  • Addresses all aspects of the task
  • Shows an ability to analyze, evaluate, compare and/or contrast issues and events
  • Includes relevant facts, examples, and details, but may not support all aspects of the theme or problem evenly
  • Is a well-developed essay, demonstrating a logical and clear plan of organization
  • Introduces the theme or problem by establishing a framework that is beyond a simple restatement of the task and concludes with a summation of the theme or problem

Score of 3:

  • Shows a satisfactory understanding of the theme or problem
  • Addresses most aspects of the task or addresses all aspects in a limited way
  • Shows an ability to analyze or evaluate issues and events, but not in any depth
  • Includes some facts, examples, and details
  • Is a satisfactorily developed essay, demonstrating a general plan of organization
  • Introduces the theme or problem by repeating the task and concludes by repeating the theme or problem

Score of 2:

  • Shows limited understanding of the theme or problem
  • Attempts to address the task
  • Develops a faulty analysis or evaluation of issues and events
  • Includes few facts, examples, and details, and may include information that contains inaccuracies
  • Is a poorly organized essay, lacking focus
  • Fails to introduce or summarize the theme or problem

Score of 1:

  • Shows a very limited understanding of the theme or problem
  • Lacks an analysis or evaluation of the issues and events
  • Includes little or no accurate or relevant facts, examples, or details
  • Attempts to complete the task, but demonstrates a major weakness in organization
  • Fails to introduce or summarize the theme or problem

Score of 0: Fails to address the task, is illegible, or is a blank paper Fails to address the task, is illegible, or is a blank paper

Generic Scoring Rubric
Social Studies Document-Based Question

Score of 5:

  • Thoroughly addresses all aspects of the Task by accurately analyzing and interpreting at least ___ documents
  • Incorporates information from the documents in the body of the essay
  • Incorporates relevant outside information
  • Richly supports the theme or problem with relevant facts, examples, and details
  • Is a well-developed essay, consistently demonstrating a logical and clear plan of organization
  • Introduces the theme or problem by establishing a framework that is beyond a simple restatement of the Task or Historical Context and concludes with a summation of the theme or problem

Score of 4:

  • Addresses all aspects of the Task by accurately analyzing and interpreting at least ___ documents
  • Incorporates information from the documents in the body of the essay
  • Incorporates relevant outside information
  • Includes relevant facts, examples, and details, but discussion may be more descriptive than analytical
  • Is a well-developed essay, demonstrating a logical and clear plan of organization
  • Introduces the theme or problem by establishing a framework that is beyond a simple restatement of the Task or Historical Context and concludes with a summation of the theme or problem

Score of 3:

  • Addresses most aspects of the Task or addresses all aspects of the Task in a limited way, using some of the documents
  • Incorporates some information from the documents in the body of the essay
  • Incorporates limited or no relevant outside information
  • Includes some facts, examples, and details, but discussion is more descriptive than analytical
  • Is a satisfactorily developed essay, demonstrating a general plan of organization
  • Introduces the theme or problem by repeating the Task or Historical Context and concludes by simply repeating the theme or problem

Score of 2:

  • Attempts to address some aspects of the Task, making limited use of the documents
  • Presents no relevant outside information
  • Includes few facts, examples, and details; discussion restates contents of the documents
  • Is a poorly organized essay, lacking focus
  • Fails to introduce or summarize the theme or problem

Score of 1:

  • Shows limited understanding of the Task with vague, unclear references to the documents
  • Presents no relevant outside information
  • Includes little or no accurate or relevant facts, details, or examples
  • Attempts to complete the Task, but demonstrates a major weakness in organization
  • Fails to introduce or summarize the theme or problem

Score of 0: Fails to address the Task, is illegible, or is a blank paper Fails to address the Task, is illegible, or is a blank paper

Rating Sheet (Circle one examination title below.)

Regents Examination in: o Global History and Geography o United States History and Government

Examination Date: _________________ 

Raterís Name:

 

___________________________________

 
Choose One: 

Rater Number:

 
1 2 3 (circle one) 
Part II essay  

School:

 
   
Part III B essay 

Date:

 
   

Studentís Name

Essay Score

1.   
2.   
3.   
4.   
5.   
6.   
7.   
8.   
9.   
10.   
11.   
12.   
13.   
14.   
15.   
16.   
17.   
18.   
19.   
20.   
21.   
22.   
23.   
24.   
25.   

Record Sheet (Circle one examination title below.)

Regents Examination in: o Global History and Geography o United States History and Government

Examination Date

(month and year) ______________ School_________________________District_____________________

Studentís Name

 

Part II Essay Scores

 

Part III B Essay Scores

 

Rater
1

Rater
2

Rater
3

Resolved
Score

Rater
1

Rater
2

Rater
3

Resolved
Score

1.                 
2.                 
3.                 
4.                 
5.                 
6.                 
7.                 
8.                 
9.                 
10.                 
11.                 
12.                 
13.                 
14.                 
15.                 
16.                 
17.                 
18.                 
19.                 
20.                 
21.                 
22.                 
23.                 
24.                 
25.