EMSC Banner for Content Pages. The banner reads, "New York State Education Department Back to EMSC Home
SED Home
Disclaimers and Notices

Regional BOCES Scoring Case Study
Grade 5  Elementary-Level
Social Studies Test

Demographic Profile

The BOCES in this case scores approximately 3,000 tests per year from 15 different public and private school districts. The component districts order the tests, including exams that are in foreign languages and in Braille from the BOCES. Between 70 and 80 qualified teachers score the tests. The teachers come from the component districts. These districts send all scoring materials to BOCES.

 
   
 

Logistics

On the basis of the number of papers, BOCES determines the number of raters who are needed for scoring papers. Raters usually use a local restaurant or convention center suitable for large-scale scoring. Traditionally, scoring takes place the week after Thanksgiving. Scoring usually takes three days, but the time varies depending on the number of papers that need scoring. Papers are sent by a BOCES secured courier and secured at the scoring location.

On the Friday before scoring, papers are organized at BOCES. There are four to ten districts per table, depending on the number of papers. Each table scores a mix of papers from the component districts. Scoring is randomized by district.

Preparation

Records and rating sheets are generated by BOCES on the basis of input from component schools. Using information provided by component schools, the BOCES clerical staff develop the list of exam takers. The BOCES clerical staff is responsible for completing the rating sheets, the record sheets, and the final tabulation form. These sheets are returned to the component school districts when scoring is completed.

Veteran table leaders conduct the turnkey training. Scorers are divided into two groups, and further divided into pairs. Table leaders maintain consistency in the scoring rooms.

 
   
 

Procedure

Training is provided to calibrate the tasks (CRQs, DBQ, scaffolding, DBQ essay). The NYSED rating guide is reviewed for each section. The first two hours of the scoring session are set aside to discuss any issues related to the rating guides. Alternative answers are brainstormed. NYSED is phoned if, in our professional judgment, something from our brainstormed answers or student responses does not fit NYSEDís guidelines, or if we need further clarification. The person designated by the BOCES makes the phone call.

Papers are organized and read according to NYSED regulations. The papers are sorted and bundled. The bundles are moved between the teachers. As one teacher in the pair completes a bundle, it is handed to another teacher. Raters' sheets are kept rubber-banded to the bottom of the envelope.

If two raters have conflicting scores of more than one point, the paper is pulled from the bundle and placed on top of the envelope. The rating process is stopped, and the teacher who has not rated this set of paper reads it.

The table leader (facilitator) is responsible for the movement of completed papers. The papers are secured by BOCES overnight for multiple-day scoring sessions.

The multiple choice section is machine scored at BOCES. SCAN-TRON answer sheets are sent to BOCES by districts. Final tabulations are completed by the RIC at BOCES. Once the tabulations are complete, each component district will receive its papers.

Post-scoring Activities

Scores are reported to the BOCES curriculum council, which includes representatives from each district. Reports are sent to each district. Exams are sorted by district at BOCES and sent back to the districts in boxes. It is each districtís responsibility to store the examinations. Generally, a debriefing session is not conducted, but if there is a problem with a scorer, the district is notified. An item analysis is performed by the BOCES Regional Information Center (RIC). District permission is required to do a comparative analysis. Results are posted on the BOCES blackboard site and sent to component districts as well as to the BOCES curriculum council. Evaluations of the exam are distributed and collected at the scoring session and returned to the New York State Education Department.

 
   

 

Back to New York State Social Studies Assessment