Scaling and Scale Maintenance, Standard Setting, and Reliability/Validity Analyses of the New York State Regents Examination and Grades 4 and 8 Science Test Programs
The New York State Education Department (NYSED) Office of State Assessment is seeking proposals to conduct three psychometric tasks: (1) scaling and scale maintenance of field tests; (2) post-operational score collection and standard setting; and, (3) analyses to provide evidence regarding reliability and validity for selected High School Regents Examinations in English, Mathematics, Science, and Social Studies, and the Science examinations for Grades 4 and 8. The successful vendor will perform these three psychometric tasks and prepare technical documentation for the 2015–16, 2016–17, and 2017–18 school years. Vendors must bid on all services for any examinations selected by NYSED in the content areas listed in this RFP.
Scaling and scale maintenance will be required for all field test administrations, once per year, beginning in the spring of the 2015–16 school year and then again for the 2016–17 and 2017‑18 school years. Vendors must bid on the scaling and equating of all examinations listed on the chart in the Background section of 1) DESCRIPTION OF SERVICES TO BE PERFORMED in this Request for Proposal (RFP). It is anticipated that under the terms of this contract, two examinations (Algebra II and Global History and Geography), and possibly others as well, will require the establishment of a new operational baseline scale and a technical report with documentation of all analyses and results. If the optional two-year contract extension is exercised, up to six new science examinations and one new social studies examination may require the establishment of baseline scales during the extension period. The vendor will be required to create a field-test form equating plan that would maximize the flexibility of the operational form development; as well as produce a scoring table and conversion chart annually for each of the Grades 4 and 8 Science operational examinations and three administrations of the operational Regents Examinations.
The standard setting component of the scope of work will require that the vendor conduct a post-operational score collection during the June 2016, June 2017, and June 2018 test administration periods and conduct standard-setting meetings to set performance standards. It is anticipated that the vendor will conduct up to 3 standard-setting meetings per June administration period (total of up to 6 during the initial contract period). If the contract extension option is exercised, up to 4 additional standard-setting meetings per year may be required. At the discretion of NYSED, some of these activities may be cancelled or postponed. The vendor will be paid based on the actual volume of work performed.
Beginning in the fall of 2015, the vendor will be required to develop technical reports for each Regents Examination and for the Grades 4 and 8 Science Tests based on operational test data provided to the vendor by NYSED. The vendor will be required to conduct data analyses to provide evidence regarding reliability, validity, scaling, and equating for the operational results of each Regents Examination title including, but not limited to, those listed in this RFP and for the Grades 4 and 8 Science Tests for the 2015–16, 2016–17, and 2017–18 school years.
This RFP includes three services. SERVICE 1 includes all deliverables relating to scaling and scale maintenance. SERVICE 2 includes all deliverables relating to score collection and setting the performance standards through standard setting. SERVICE 3 includes all deliverables relating to conducting data analyses that are required to provide evidence regarding the reliability, validity, scaling, and equating for the operational results of the examinations as well as preparing annual technical reports for each examination administered in the preceding year using operational test data provided by NYSED.
The eligible applicants may be either for-profit or not-for-profit entities, including large testing companies, consultants, and/or educational organizations with demonstrated capacity to complete all required services for large-scale assessments and demonstrated experience in educational measurement, scaling and equating, score collection, standard setting, data analysis, and technical reporting.
Subcontracting will be limited to thirty percent (30%) of the annual contract budget. Subcontracting is defined as non-employee direct personal services and related incidental expenses, including travel.
The vendor must submit a valid, complete, and comprehensive three-year detailed work plan with a time line that provides an in-depth description of how the activities required in the RFP will be conducted. The work plan must be aligned with a detailed timeline for each year of the contract and be consistent with the Schedule of Deliverables provided in this RFP.
Vendors are required to comply with NYSED’s Minority and Women-Owned Business Enterprises (M/WBE) participation goals for this RFP through one of three methods. Compliance methods are discussed in detail in the Minority/Women-Owned Business Enterprise (M/WBE) Participation Goals section below.
Terms of Contract
NYSED will award one contract pursuant to this RFP with an initial term of three years, anticipated to begin October 1, 2015 and to end September 30, 2018, with the option to extend for one additional two-year period (October 2018 – September 2020).
Components contained in RFP Proposal #15-015 are as follows:
- Description Of Services To Be Performed
- Proposal Submission
- Evaluation Criteria and Method of Award
- Proposal submission documents packages A-E
Questions and Answers
Questions regarding the request must be submitted by e-mail to AssessmentRFP@nysed.gov no later than the close of business June 3, 2015. Questions regarding this request should be identified as Program, Fiscal or M/WBE. A Questions and Answers Summary be posted here no later than June 15, 2015
The following are the designated contacts for this procurement:
The following proposal submission document packages must be separately sealed, as detailed in 2) PROPOSAL SUBMISSION of the RFP, and must be received by NYSED no later than June 29, 2015 by 3:00 p.m.
- Submission Documents labeled Submission Documents – RFP#15-015 Do Not Open
- Technical Proposal labeled Technical Proposal – RFP#15-015 Do Not Open
- Cost Proposal labeled Cost Proposal – RFP#15-015 Do Not Open
- M/WBE Documents labeled M/WBE Documents – RFP#15-015 Do Not Open
- CD containing the technical, cost, submission and M/WBE proposals submitted using Microsoft Word. Place in a separate envelope labeled CD-ROM – RFP#15-015 Do Not Open.
The mailing address for all the above documentation is:
(Facsimile copies of the proposals are NOT acceptable)
NYS Education Department
Bureau of Fiscal Management
Attn: Jessica Hartjen, RFP#15-015
Contract Administration Unit
89 Washington Avenue, Room 503W EB
Albany, NY 12234