EMSC Banner for Content Pages. The banner reads, "New York State Education Department Back to EMSC Home
SED Home
Disclaimers and Notices

Guided Questions

For Scoring State Level Social Studies Examinations
 

Demographic Profile

  • Number of students

    • How many tests need to be ordered? (Remember to order a few extras for use by special education/resource room teachers!)

  • Types of tests

    • What types of tests (foreign language, Braille, etc.) must be ordered?

  • Number of qualified staff including special education teachers

    • Do you have an adequate number of staff to score?

    • Who will be included in the scoring?

    • Are there resources (retirees, qualified substitutes, etc.) outside the district for scoring?

    • Will it be necessary to team with another district to accomplish the scoring task in a timely manner?

  • Language needs (bilingual)

    • Do you have enough bilingual teachers to score student papers?

  • Braille needs

    • Will you need teachers who can read Braille to score?

 
   
 

Logistics

  • Allocation of staff

  • Who will be scoring each section?

  • Site location

  • What space is available that will accommodate the task?

  • Is the space available for the time period when it is needed?

  • Time needed

  • When will it be scored?

  • On the basis of the number of papers and the number of available raters, can you determine how much time is needed to complete each section?

  • Are there any collective bargaining concerns associated with scoring?

  • Transportation and storage

  • How will the papers be sent to the scoring location?

  • How will the papers be secured?

  • Organization of papers

  • How will the papers be collected and bundled after the administration?

  • How will they be marked to identify them for scoring?

  • Preparation

  • Review the information from the New York State Education Departmentís School Administratorís Manual regarding the test being scored (NYSED SCORING MATERIALS)

  • Have all staff received and reviewed the booklet?

  • Have you reviewed the rating and record sheets?

  • Is there a need to create your own rating and record sheets?

  • Lists of exam takers

  • Do you have clerical staff to create class lists and scoring forms?

  • Who is providing this information?

  • Assignment of tasks

  • Are scoring assignments equitable?

  • Are readers receiving adequate time for breaks and refreshments?

  • Filling out of rating sheets and record sheets

  • Who is responsible for completing the rating sheets, the record sheets, and the final tabulation forms?

  • Where do these sheets go when they are completed?

  • Turnkey training

  • Where and when will training be held?

  • Who can help train the rest of the staff?

  • How will consistency be maintained among all raters?

  • What materials do you need to have at the training?

  • Proctoring schedules for raters

  • Will your raters have to proctor?

  • Are there any conflicts between scoring and proctoring?

  • Determine types of answer sheets for each section and how they will be processed

  • Do answer sheets need to be hand-scored or machine-scored?

  • Who is providing machine-scored answer sheets? (district, regional scoring center, etc.)

  •  
       
     

    Procedure

  • Training and calibration for tasks (constructed-response questions, document-based question Ė scaffolding and essay)

  • Have raters adequately reviewed the anchor papers?

  • Has brainstorming taken place, and have ideas been recorded and posted, so that raters can review them?

  • If rating is taking place over multiple days, are rubrics reviewed and reinforced each day?

  • Organize raters according to New York State Education Department regulations

  • Do you have at least two readers for each essay paper?

  • Is a third reader available?

  • Who will resolve any discrepant papers?

  • Who will contact the New York State Education Department concerning questions not addressed in the scoring guide?

  • Do raters understand the scoring rubrics?

  • Do raters understand the concept of holistic scoring?

  • Movement of scored papers

  • How will the papers be moved so that all parts and all papers are scored completely?

  • Where will papers be stored overnight for multiple day scoring?

  • Is a checklist necessary for routing purposes?

  • Multiple-choice scoring

  • Who is responsible for scoring and recording?

  • Is there a fail-safe procedure for double-checking accuracy?

  • Who receives the scored documents after they are completed?

  • Tabulation

  • Who performs final tabulations?

  • Is there a system in place to re-check tabulations?

  • Have marginal papers been identified and reviewed?

  • Post-scoring Activities

  • Data reporting and grade dissemination

  • In addition to teachers and administrators, who needs to receive this information?

  • Does your district have a form for reporting test results?

  • Packing and storing

  • Who is responsible for the packing and storing of examination papers?

  • How will papers be bundled and stored after scoring is completed?

  • Does your district have an established procedure for this task?

  • Do you know that papers must be stored for one year?

  • Debriefing

  • Have teachers received performance information?

  • Has an item analysis (school wide, district wide, regional information center, etc.) been performed?

  • Has a comparative analysis with previous years or within a cohort been compiled?

  • How will the information be used to improve instruction and performance?

  • Evaluation of the scoring session

  • Did you collect information on how the scoring session went?

  • Did you complete the scoring within the set time frame?

  • How many discrepant papers did you have?

  • How will you use this information to improve scoring next year?

  • Evaluation of the exam

  • Did teachers and raters fill out the NYSED evaluation form, and were the forms returned to NYSED?

  • Academic Intervention Services planning

  • How are students identified for AIS?

  • How does the information for AIS travel with specific students within the district? Outside of the district?

  • State call for papers

  • Who is responsible for packing and sending the papers?

  • What happens with the information that is returned from the State?

  •  
       

    Back to New York State Social Studies Assessment