P-12

Prekindergarten through Grade 12 Education

RFP #10-021

Title: Development of State Assessments in Elementary, Intermediate (Middle), and High School Level English Language Arts (ELA) and Elementary and Intermediate (Middle) Level Mathematics

The New York State Education Department (NYSED) is seeking proposals to develop tests in English Language Arts in Grades 3, 4, 5, 6, 7, 8, 9, 10, and 11 and in Mathematics in Grades 3, 4, 5, 6, 7, and 8 for multiple school years as defined in this RFP. The requested services include conducting field testing using both stand alone field tests and items embedded within the operational tests, scoring the field tests, developing all operational test forms, ancillary materials, and scoring materials, and perform all analysis and research. Test development will start in January 2011 with the creation of field tests for administration in the 2010–11 school year and operational tests beginning in the 2011–12 school year. Bidders must bid on both content areas, ELA and Mathematics, and all requested grade levels. Eligible applicants are testing companies and/or educational organizations with demonstrated experience in developing secure, high volume, standardized assessments including both multiple-choice and constructed-response items.

Subcontracting is limited to 40% of the annual contract budget. NYSED reserves the right to approve all subcontractors. Subcontracting is defined as non-employee, direct, personal services and related incidental expenses, including travel. Bidders are required to comply with NYSED’s Minority and Women-Owned Business Enterprises (M/WBE) participation goals. The goals are 10% Minority Business Enterprise (MBE) and 10% Women-Owned Business Enterprise (WBE). These goals can be met by any combination of MBE and WBE participation. Please see the section on M/WBE Requirements for information regarding NYSED’s Minority/Women-Owned Business Enterprise Procurement Policy.

Bidders are required to submit separate costs for two test development options. Option 1 is for the development of Grades 3–11 ELA and Grades 3–8 Mathematics Tests to include multiple-choice and constructed-response items. Option 2 is for the development of Grades 3–8 ELA and Grades 3–8 Mathematics Tests to include multiple-choice and constructed-response items. Following the scoring of the technical proposals and cost proposals submitted for Option 1, NYSED will make a determination, based on available funding, whether to award a contract pursuant to Option 1 to the bidder who received the highest total score. If it is determined that the cost of the highest ranked proposal for Option 1 is above the amount of funding available, then a second ranking will be completed using the cost proposals submitted for Option 2. Please see the section on Method of Award for additional information.

The contract resulting from this RFP will be for a single award with a term of five years. The contract will begin on approximately January 1, 2011 and end on December 31, 2015. Bidders should refer to the Mandatory Requirements section of this RFP and to the Schedule of Deliverables located in Submission Documents when preparing their proposals. Bidders must adhere to all Mandatory Requirements and perform all required deliverables.

Mandatory Requirements

Bidders must agree to adhere to all of the Mandatory Requirements referenced in the Submission Documents section of this RFP. If a bidder fails to meet any of these mandatory requirements, its proposal will be disqualified and removed from further consideration. By signing the mandatory Requirements Certification Form located in the Submission Documents of this RFP, the vendor certifies that they agree to provide and/or meet all of the Mandatory Requirements listed.

Components contained in RFP Proposal #10-021 are as follows:

  1. Description of Services to be Performed
  2. Submission
  3. Evaluation Criteria, Method of Award, and Terms and Conditions
  4. Assurances
  5. Appendices
  6. Submission Documents, including M/WBE Documentation and Schedule of Deliverables
  7. Attachments

Mandatory Bidders’ Conference

All bidders must attend the mandatory Bidders’ Conference to be held on October 15, 2010 at 9:00 AM in Room 216 of the Education Building, 89 Washington Avenue, Albany, New York. An agency representative must attend this mandatory conference in order to be eligible to bid on this RFP. The purpose of this conference is to clarify any points in the RFP that may not be clearly understood. Confirm intent to attend by e-mailing the Office of State Assessment at EMSCRFPOSA@mail.nysed.gov by close of business October 12, 2010.

Questions and Answers PDF icon (74 KB) are now posted.

Contact Information for Questions

Program Matters

Fiscal Matters

M/WBE Matters

Jessica Hartjen Gary Yorkshire Joan Ramsey
(518) 474-5902 (518) 408-1715 (518) 486-6883
EMSCRFPOSA@mail.nysed.gov

The following documents must be received at NYSED no later than November 15, 2010 by 3:00 PM:

  1. Technical Proposal—Eight (8) copies (one bearing an original signature in blue ink) – Submitted in a sealed envelope labeled Technical Proposal - RFP#10-021 Do Not Open
  2. Two Cost Proposals—Three (3) copies (one bearing an original signature in blue ink) for each option. Each submitted in a separate sealed envelope labeled Cost Proposal - RFP #10-021 (Option 1 or Option 2) Do Not Open
  3. Electronic Version (Microsoft Word)—One (1) CD with the technical and both cost proposals – Included with the cost proposal
  4. M/WBE Documents—Three (3) copies (one bearing an original signature in blue ink) for each option-in a separate sealed envelope labeled M/WBE Documents - RFP #10-021 (Option 1 or Option 2) Do Not Open. Each set must include the following:
    1. M/WBE Cover Letter Signature Required
    2. M/WBE 100 Utilization Plan
    3. M/WBE 102 Notice of Intent to Participate
    4. EEO 100 Staffing Plan

The mailing address for all the above documentation is:

NYS Education Department
Bureau of Fiscal Management
Contract Administration Unit
89 Washington Avenue, Room 505W EB
Albany, NY 12234

Facsimile copies of the proposals are NOT acceptable.

Back to top

Go to Table of Contents


Table of Contents


Description of Services


Work Statement

This section of the bid package details the services and products to be acquired. Please note that the contract process also includes general New York State administrative terms and conditions, as well as terms and conditions required by New York State law. These terms and conditions address issues related to both the submission of bids and any subsequent contract; they are included separately in this bid package for your information. Please review all terms and conditions carefully.

Background Information

The New York State Board of Regents, which is responsible for setting educational policy, standards, and rules, has published learning standards in seven curricular areas: Mathematics, Science, and Technology; English Language Arts; The Arts; Languages Other Than English; Health, Physical Education, Family and Consumer Sciences; Social Studies; and Career Development and Occupational Studies. These standards are the foundation for a rigorous system of assessment designed to:

  • evaluate the achievement of the knowledge and skills delineated by the learning standards and their constituent key ideas and performance indicators;
  • provide information that helps teachers adapt instruction to students' strengths and needs;
  • provide useful information to students, parents, educators, and the general public about what students know and can do; and
  • serve as the basic source of data for a rigorous school, district, and State accountability system.

In January 1999, New York State administered the Elementary English Language Arts (ELA 4) examination for the first time. In June 1999, the Intermediate English Language Arts (ELA 8), Elementary Mathematics (Mathematics 4), and Intermediate Mathematics (Mathematics 8) tests were administered. Each of these tests provided information on the proficiency of each student with respect to the learning standards for those subject areas.

In January and March of 2006, New York State administered the English Language Arts and Mathematics Tests for Grades 3–8. These tests are designed to determine whether each student is making adequate progress in attaining the skills and knowledge expected of students in their grade level, as outlined in the New York State Core Curriculum. These proficiencies are gauged in scale scores demarcated into four levels: Level 1 (Below Standard), Level 2 (Meets Basic Standard), Level 3 (Meets Proficiency Standard), and Level 4 (Exceeds Proficiency Standard).

The Board of Regents adopted the Common Core State Standards in July 2010. (Please see http://www.corestandards.org/the-standards external link icon for more information.) This results in a phase-in strategy for the Grades 3–8 Tests to be implemented during the term of the contract resulting from this RFP. Operational tests in the 2010–2011 school year (not part of this contract) will continue to be aligned to the existing NYS Learning Standards. Operational tests administered in the 2011–2012 school year will also be developed based on the test specifications associated with the existing NYS Learning Standards. However, the vendor will prepare items maps indicating the alignment to the NYS Learning Standards, the Common Core State Standards, and the NAEP Frameworks. Beginning with the operational tests to be administered in the 2012–2013 school year, tests will be developed based on the Common Core State Standards, with item maps indicating the alignment to both the Common Core State Standards and the NAEP Frameworks.

Beginning with field testing in 2011 and operational testing in 2012, NYSED plans to expand the Grades 3–8 Testing Program to include tests in ELA in Grades 9–11.

Test Administration Schedule

NYSED is requesting proposals for the development of secure tests, aligned with the NYS Learning Standards, the Common Core State Standards, and the NAEP Frameworks, to assess the achievement of elementary, middle level, and high school students. The tests will be developed in:

  1. English Language Arts (Grade 3, 4, 5, 6, 7, 8, 9, 10, and 11)
  2. Mathematics (Grade 3, 4, 5, 6, 7, and 8)

Field tests and operational tests for both ELA and Mathematics will be developed and administered according to the following schedule:

Field Test and Operational Test Administration Schedule

Contract Year

Field Test Administration

Operational Test Administration

Year 1
January 1, 2011–
December 31, 2011
April/May 2011
(all items to be field tested)
NA
(Forms Development Only)
Year 2
January 1, 2012–
December 31, 2012
April/May 2012
(all constructed-response items and multiple-choice ELA listening items)

Embedded in Operational Test
(all multiple-choice items except those associated with ELA listening)
April or May 2012
Year 3
January 1, 2013–
December 31, 2013
April/May 2013
(all constructed-response items and multiple-choice ELA listening items)

Embedded in Operational Test
(all multiple-choice items except those associated with ELA listening)
April or May 2013
Year 4
January 1, 2014–
December 31, 2014
April/May 2014
(constructed-response items and multiple-choice ELA listening items)

Embedded in Operational Test
(all multiple-choice items except those associated with ELA listening)
April or May 2014
Year 5
January 1, 2015–
December 31, 2015
April/May 2015
(constructed-response items and multiple-choice ELA listening items)

Embedded in Operational Test
(all multiple-choice items except those associated with ELA listening)
April or May 2015

Test Development

Test Development Requirements

  1. Beginning with the tests administered in the 2011–12 school year, Books 1 and 2 of each operational test will include both operational test items and embedded multiple-choice items included for field test purposes. See the section on Field Testing beginning for a description of the embedded field test. Anchor items to be used for equating purposes will also be included in Books 1 and 2. Most, but not all, of the embedded anchor items will be provided to the vendor by NYSED from pre-established anchor items. Individual student test scores will be based solely on the operational test items. Please see the section on Forms Scaling and Equating for additional information on the anchor items.
  2. At the discretion of NYSED, the vendor must include among the embedded items on the operational tests, or include in the field tests, or both, audit items specified by NYSED. These items will be in addition to the operational and embedded field test and anchor items specified in the RFP and will be provided to the vendor by NYSED by August 5 proceeding the school year in which the test will be administered, or by another date agreed upon by the vendor and NYSED.
  3. The bidder must propose to include between 20 and 25 nationally normed multiple-choice items among the operational test items in each grade. These items will be included in Book 1 and 2. The nationally normed items must be included in field tests prior to inclusion on operational tests. These items will remain the property of the vendor and will not be publically released following operational administration. The nationally normed items must be items that have not been released or administered in vendor-furnished tests in New York State.
  4. With the exception of the nationally normed items, all operational test items must be unique items. No item may have appeared on a previously administered test in New York State or any other state.
  5. All items on the field tests and operational tests become the sole property of New York State, with the exception of the nationally normed items.
  6. All test content is required to be aligned with the NYS Learning Standards, the Common Core State Standards, the NAEP Frameworks, and the test specifications.
  7. Following the administration of field tests, an item bank must be available for NYSED review at any time. NYSED will own any and all items and materials in the item bank, excepting the vendor-owned nationally normed items. Furthermore, the vendor must provide the items in the bank to NYSED on a yearly basis, after item selection for operational tests, through a mutually agreed upon procedure.
  8. The vendor must provide NYSED with twenty business days for each step for the review of items, field test forms, operational test forms, scoring keys, rating guides, and related deliverables. This will allow sufficient time for NYSED to provide feedback to the vendor and give final approval on the deliverables. NYSED appreciates the time constraints inherent in developing field tests for administration in the first year and will endeavor to complete its review in less time whenever possible. The vendor is responsible for any costs associated with making modifications to materials and deliverables necessary to obtain NYSED approval.
  9. NYSED will review all items for approval, rejection, or revision before they are brought to teacher committees. Passages or items rejected by NYSED will not be brought for review by teachers. To the fullest extend feasible, the vendor must make changes requested by NYSED prior to committee review.
  10. NYSED will have the final authority over items selected by the committees and used in field tests.
  11. For both ELA and Mathematics, the vendor must maintain a full staff consisting at a minimum of three Mathematics content specialists, one each for Grades 3–4, 5–6, and 7–8, and four ELA content specialists, one each for Grades 3–4, 5–6, 7–8, and 9–11, for a total of seven content specialists, and at least one development supervisor for the duration of the contract. Each content specialist must have a bachelor’s degree or higher, at least 24 college credits in the content area, and at least three years of experience teaching in the content area. These staff members must be identified by name and experience in the bidder's proposal, and a resume for each staff person must be included.
  12. The vendor must develop items that are clear and age/grade appropriate for each test and that will cover the full range of the testing pool, from easy to difficult.
  13. The tests must include items with a full range of difficulties such that the tests will allow for valid and reliable distinctions between students at all four levels of proficiency for each grade level being tested.
  14. The tests must be standards-based tests, with a high degree of precision around all three achievement level cut scores. The tests are designed to assess knowledge and skills contained in the learning standards, and therefore are timed, but not speeded.

Teacher Committees

NYSED requires the vendor to involve New York State teachers, and for standard setting, college professors, in only the following activities related to the development of examinations. (See Attachment F for specifications for teacher committees.)

  • Item Review;
  • Final Eyes Review; and
  • Standard Setting

All activities involving New York State teachers must be held within a radius of 30 miles of the NYSED building at 89 Washington Avenue in Albany, New York.

NYSED has a database of selected teachers that it will provide to the vendor to use to contact teachers and determine their willingness and availability for participation in teacher committees. New names will be added by NYSED. Committees must be composed of teachers from all parts of the State, be diverse in terms of race, gender, and the needs/resource capacities of their schools, and be composed of a mixture of both experienced and new reviewers. A waiting list of teachers for each event must be established by the vendor in case of last-minute problems with attendance.

For all teacher committees, except as time constraints in the first year of the contract may preclude, the vendor must send out written invitations to teachers to participate at least two months in advance to allow sufficient time for teachers to plan accordingly, especially during the school year.

For each teacher committee the vendor is required to:

  1. Convene and conduct meetings and pay for participant expenses at State approved rates;
  2. Contact the teachers recommended by NYSED to determine willingness and availability to participate on the committee and to inform them of the logistics around participation;
  3. Arrange and pay for the meeting space within 30 miles of the NYSED building;
  4. Notify NYSED in advance of all committee meetings and provide all committees with the information needed to analyze and review assessment objectives and items;
  5. Record and summarize the recommendations of these committees, the result of those recommendations, and report this information in writing to NYSED;
  6. Provide evaluation forms for teachers to submit near the end of each meeting providing feedback on the quality of the vendor’s session; and
  7. Provide NYSED’s Office of State Assessment with teacher and staff attendance sheets and copies of the evaluation forms for each meeting.

English Language Arts

Tests will be developed for ELA Grades 3–11 that follow the test design in the table below including assessment of the three NYS Learning Standards within the reading, writing, and listening modalities, the Common Core State Standards for English Language Arts, and the NAEP Frameworks for Reading and Writing. Each Grades 3–11 ELA Test will be made up of four test books. There will be multiple-choice and constructed-response items on every test. Bidders are to assume development of questions with increasing problem solving and higher cognitive demands with the increasing cognitive maturity of the student.

English Language Arts Test Design

The following chart provides a description of the test design for each grade.

English Language Arts Test Design

Grade

Book

Number and Type of Passages

Number of 1-Credit Multiple-Choice Items*

Number of 2-Credit Constructed-Response Items

Number of 4-Credit Constructed-Response Items

3

1

5-7 Reading

33

0

0

3

2

5-7 Reading

33

0

0

3

3

1 Listening

5

3

1

3

4

4-5 Reading

0

4

1

4

1

6-8 Reading

36

0

0

4

2

6-8 Reading

36

0

0

4

3

1 Listening

5

3

1

4

4

4-6 Reading

0

4

1

5

1

6-8 Reading

36

0

0

5

2

6-8 Reading

36

0

0

5

3

1 Listening

5

3

1

5

4

4-6 Reading

0

4

1

6

1

6-8 Reading

36

0

0

6

2

6-8 Reading

36

0

0

6

3

1 Listening

5

3

1

6

4

4-6 Reading

0

4

1

7

1

6-8 Reading

36

0

0

7

2

6-8 Reading

36

0

0

7

3

1 Listening

5

3

1

7

4

4-6 Reading

0

4

1

8

1

6-8 Reading

36

0

0

8

2

6-8 Reading

36

0

0

8

3

1 Listening

5

3

1

8

4

4-6 Reading

0

4

1

9

1

6-8 Reading

36

0

0

9

2

6-8 Reading

36

0

0

9

3

1 Listening

5

3

1

9

4

4-6 Reading

0

4

1

10

1

6-8 Reading

36

0

0

10

2

6-8 Reading

36

0

0

10

3

1 Listening

5

3

1

10

4

4-6 Reading

0

4

1

11

1

6-8 Reading

36

0

0

11

2

6-8 Reading

36

0

0

11

3

1 Listening

5

3

1

11

4

4-6 Reading

0

4

1

* Approximately one-third of the items and passages shown in Books 1 and 2 in each grade will be embedded field test and anchor items to be used for developing future operational test forms and for equating purposes. Please see Attachment G and the sections on "Field Testing" and "Forms Scaling and Equating" for additional information.

Approximate Percentage of Credits Assessing Each Standard

Bidders must propose tests aligned to the NYS Learning Standards, the Common Core State Standards, and the NAEP Frameworks. The current approximate percentage of credits assessing each of the three NYS English Language Arts Learning Standards and the Core Performance Indicators (see http://www.p12.nysed.gov/ciai/cores.html) is shown below; however, this breakdown may change following the adoption and implementation of the Common Core State Standards.

Approximate Percentage of Credits Assessing Each Standard

New York State Standard/Core Performance Indicators

Grade

3 4 5 6 7 8 9 10 11

Standard 1. Reading, Writing, and Listening for Information and Understanding

33%

36%

39%

36%

39%

39%

18%

18%

18%

Standard 2.  Reading, Writing, and Listening for Literary Response and Expression

47%

44.5%

39%

44.5%

39%

39%

19%

19%

19%

Standard 3.  Reading, Writing, and Listening for Critical Analysis and Evaluation

20%

19.5%

22%

19.5%

22%

22%

27%

27%

27%

Core Performance Indicators

 

 

 

 

 

 

36%

36%

36%


Question Formats for the ELA Tests

The ELA Tests contain questions with a variety of formats, including multiple-choice, short response, extended response, and graphic organizers. For multiple-choice questions, students select the correct response from four answer choices. For short- and extended-response questions, students write an answer to an open-ended question. The extended responses are scored for writing as well as for reading comprehension.

Passages on ELA Tests

  1. A variety of material must be used on the ELA tests. Approximately 50 percent of the reading passages on each test may be commissioned pieces and approximately 50 percent of the reading passages must be authentic literature.
  2. The material must be of general interest, have a familiarity to most students, have characters that are portrayed as positive role models, have a positive message, and be well written.
  3. The vendor must obtain a large diversity of material to bring to teacher review; most genres should be included yearly.
  4. Illustrations or photographs that accompany passages should be used to the extent possible.
  5. Authentic published literature used on each ELA test must come from a variety of books and magazines. These pieces may only be abridged from longer pieces if the permission of the author is obtained prior to the item being field tested.
  6. In order to measure students’ growth towards college and career readiness, the assessments should contain both literary and informational passages. In Grades 3–5, approximately half of the passages should be literary texts and half should be informational texts. In Grades 6–11, approximately one-third of the passages should be literary texts, one-third should be history/social studies texts, and one-third should be science/technical texts.
  7. At each grade level, the assessments must contain sufficiently complex passages worthy of reading. If possible, the tests should provide information on the level of text complexity at which each student demonstrates proficiency.
  8. NYSED staff will review and approve passages prior to their being reviewed by teachers and will have the final authority over passages selected for use on the tests.

Requests for Permission

Requests for Permission obtained by the vendor from copyright holders to use their copyrighted works and materials on any NYSED ELA operational tests administered under this contract must include explicit permission from the author for NYSED to use the copyrighted work in NYS operational tests, whether or not developed by the vendor. For at least 75% of the copyrighted works, excluding those associated with the nationally normed items and the embedded anchor items, the written agreement between the vendor and the copyright holders, for works used in field tests and operational tests, must expressly state that NYSED is being granted permission to post the test containing the copyrighted work on the NYSED web site for no less than three years following the administration of the operational test.

The vendor must obtain permission of the author or copyright holder for use of copyrighted passages and Internet posting prior to associated items being submitted to NYSED for review. If the vendor cannot obtain the copyright holder’s approval to use the copyrighted work on NYSED’s operational tests, then the passage and all associated items cannot be used for any purpose, including, but not limited to field tests and operational tests.

Items on ELA Tests

  1. NYSED prefers that items be written by individuals with a bachelor’s degree or above, at least 24 college credits in the ELA content area, knowledge of appropriate questions for the target audience, experience in writing and editing items, and knowledge and training in the NYS Learning Standards, the Common Core State Standards, and the NAEP Frameworks.
  2. The vendor must develop at least eight multiple-choice items per passage, so that operational tests may be constructed using items that have been found appropriate through NYSED and teacher committee review and have been successfully field tested.
  3. Items and item types must reflect the best educational research available. Items must match the cognitive demands defined for each standard and be clear to the reader.
  4. Multiple-choice questions must have four choices, labeled A, B, C, and D. Phrases such as “none of the above” or “all of the above” may not be used. Items having negative words such as “not” may only be used when necessary. There must be one and only one correct answer choice.
  5. Constructed-response items must clearly define the expectations for the students, such as the necessity of providing details from a story or other piece of literature as support.
  6. Items on the assessment must be text dependent questions that require a close attention to the text and no information or evidence from outside the text. A text dependent question establishes what follows and what does not follow from the text itself.
  7. The vendor must convene item review committees. The committees do not select items but instead advise the vendor and NYSED on which items to include in the field tests. See the section on “Teacher Committees” and Attachment F for additional information.

Reading

In the Reading section of the ELA Tests, students will read several passages representing a variety of genres. The reading passages that appear on the test are similar to the kinds of materials students read both in the classroom and for homework assignments. Reading passages are carefully selected for grade level appropriate vocabulary and content.

For each passage, students will apply the skills and knowledge gained in the classroom by answering reading comprehension questions that demonstrate their understanding of the passages. Each passage will give the student all of the information that is needed to successfully answer the attached questions without using their background knowledge of the subject. Higher order thinking skills are required on some questions that accompany reading passages, including summarizing, analyzing, inferring, interpreting, predicting, and evaluating information.

Tests at every grade level will contain both literary and informational reading passages. Literary passages may include short stories, folk tales, poetry, or other forms of literary writing; informational passages may include articles, excerpts from biographies or autobiographies, essays, or other forms of informational writing. A minimum of five genres must be used on each test for interest and literary diversity. Genres for the tests will include:

  • Folktales
  • Reports
  • Letters
  • Stories
  • Informative pieces
  • Poems
  • Literary pieces
  • Interviews
  • How-to pieces
  • Classical works
  • Plays

Listening

In the Listening section of the ELA Tests, students will listen to a passage and apply the skills and knowledge that they gained in the classroom to answer comprehension questions. The questions are designed to assess student understanding of the passages.

Listening sections will represent a variety of genres. The listening passages that appear on the test are similar to the kinds of materials students read and hear in the classroom. Listening passages are carefully selected for grade level appropriate vocabulary and content. Students will not see the questions prior to hearing the listening selection so that they focus on the passage being read rather than on the finding answers to the questions.

Writing

Each grade level test will include both 2-credit and 4-credit constructed-response items. The students will be required to demonstrate knowledge of three types of writing: narrative, persuasive, and informative. Students will write from a variety of prompts and for many different audiences. Student writing will be evaluated on how well it communicates to the reader.

Writing prompts should require writing from sources, rather than writing without sources. The focus should be on the students’ ability to draw evidence from the given texts and present it clearly.

Back to top

Go to Table of Contents


Mathematics

Tests developed for Grades 3–8 Mathematics must be aligned to the five content strands and five process strands in the NYS 3–8 Mathematics Core Curriculum, the Common Core State Standards for Mathematics, and the NAEP Framework for Mathematics. The specific content to be assessed may change during the course of the contract period as New York State transitions to the Common Core State Standards. Bidders are to assume development of questions with increasing problem solving and higher cognitive demands with the increasing cognitive maturity of the student.

Mathematics Test Design

The following chart provides a description of the test design for each grade.

Mathematics Test Design

Grade

Book

Number of 1-Credit Multiple-Choice Items*

Number of 2-Credit Constructed-Response Items

Number of 3-Credit Constructed-Response Items

3

1

36

0

0

3

2

35

0

0

3

3

0

4

3

4

1

38

0

0

4

2

37

0

0

4

3

0

5

4

5

1

38

0

0

5

2

37

0

0

5

3

0

4

5

6

1

38

0

0

6

2

37

0

0

6

3

0

6

5

7

1

38

0

0

7

2

37

0

0

7

3

0

6

5

8

1

38

0

0

8

2

37

0

0

8

3

0

8

5

* Approximately one-third of the items shown in Books 1 and 2 in each grade will be embedded field test and anchor items to be used for developing future operational test forms and for equating purposes. Please see Attachment G and the sections on “Field Testing” and “Forms Scaling and Equating” for additional information.

Approximate Percentage of Credits Assessing Each Strand

The questions on Grades 3–8 Mathematics Tests currently assess both the content and the process strands of NYS Mathematics Standard 3. Each question is aligned to one content performance indicator for reporting purposes, but is also aligned to one or more process performance indicators, as appropriate for the concepts embodied in the task. As a result, the tests assess conceptual understanding, procedural fluency, and problem-solving abilities, rather than assessing knowledge of isolated skills and facts. Each operational test must reflect coverage of a wide range of performance indicators by randomly selecting the tested performance indicators from among all testable performance indicators. This process should ensure that a significant number of performance indicators assessed on each operational test were not tested on the previous year’s test. Bidders must propose tests aligned to the NYS Learning Standards, the Common Core State Standards, and the NAEP Frameworks. The current approximate percentage of credits assessing each strand of the NYS Mathematics Standard is shown below; however, this breakdown may change following the adoption and implementation of the Common Core State Standards.

Approximate Percentage of Credits Assessing Each Strand

Strand

Grade 3

Grade 4

Grade 5

Grade 6

Grade 7

Grade 8

1. Number Sense and Operations

48%

45%

39%

37%

30%

11%

2. Algebra

13%

14%

11%

19%

12%

44%

3. Geometry

13%

12%

25%

17%

14%

35%

4. Measurement

13%

17%

14%

11%

14%

10%

5. Statistics and Probability

13%

12%

11%

16%

30%

0%

When developing new test specifications following the implementation of the Common Core State Standards, the domains in any given grade need not, and generally should not, contribute equally to the total score. Vendors should propose methods for providing subscores for specific domains. The vendor may be required to retroactively analyze previous years’ test data to reveal past performance in these domains (excluding data from questions which are not in alignment with the Common Core State Standards).

Question Formats for Mathematics Tests

  1. The Mathematics Tests will contain multiple-choice items, 2-credit and 3-credit constructed-response items.
  2. For all multiple-choice questions, students select the correct response from four answer choices, labeled A, B, C, and D.

Back to top

Go to Table of Contents


Field Testing for ELA and Mathematics

The vendor will conduct field testing for ELA and mathematics in the 2010–11, 2011–12, 2012–13, 2013–14, and 2014–15 school years to assess the difficulty, fairness, and appropriateness of the items proposed for operational tests. In the first year (spring 2011), all field testing will be conducted through stand alone field tests, administered separately from the operational tests. In subsequent years, all new multiple-choice items, except those associated with the listening selections for ELA, will be embedded in the operational test. Field testing of constructed-response items for ELA and mathematics, and all items associated with ELA listening passages, will continue to be accomplished through stand alone field tests.

As part of the proposal submitted for this RFP, the bidder must describe how each field test and related materials will be constructed and submit a timeline for each test. The proposed plan for field testing provides specifics on major events, phases, and deliverables. The timeline for the development of operational Book 1 and 2 with embedded multiple-choice items must be consistent with timelines specified elsewhere in this RFP for providing camera ready copies of operational test books to NYSED.

Bidders may propose to assemble field tests that will include newly-developed items consistent with the test specifications. Bidders may also propose to include items on field tests that are taken from an existing item bank that the vendor may own that are consistent with the test specifications and the NYS Learning Standards, the Common Core State Standards and the NAEP Frameworks, but must understand that, except for vendor-owned nationally normed items, all items included on the tests become the property of NYSED. Items used from the vendor’s bank must be field tested before being included in the operational test.

For all field tests, the vendor must:

  1. Carefully word all field test items based on the New York State style guide. The vendor must edit and proofread all test items thoroughly before submitting them to NYSED for review and approval.
  2. Convene item review committees for each of the tests, in order to review test items based on both content and sensitivity before field testing, ensure their quality, and advise NYSED to accept, reject, or modify items. All judgments made by committee members must be consistent with criteria documented in Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education). See the section on “Teacher Committees” and Attachment F for additional information.
  3. Develop a field test item pool large enough that the operational test forms can be created using “successfully” field tested items, and those items found unusable for an immediate operational test can be revised and reworked in order to be used in future field test cycles, thus building an item bank for NYSED’s use.
  4. Field test at least 1.75 times the number of multiple-choice items and 2.5 times the number of constructed-response items needed per operational test and build an item bank that contains 50% more items than needed each year during operational content selection to allow for the best possible selection of items. As indicated in item #3 of the Test Development Requirements section of the RFP, all operational test items, including the nationally normed items, must be field tested.
  5. Include in the field tests each year items which cover the full breadth of the performance indicators in the core curricula and, when there are multiple components, the various components within each performance indicator. The vendor must also ensure that items included in the field tests are not simply questions which mirror the manner in which a performance indicator was tested in a prior field test or operational test. The vendor must ensure variety in its writing of items to prevent “item cloning,” where nearly identical items with minor syntactical or numerical revisions occur on tests across multiple years. The vendor should not treat any performance indicator as untestable except by mutual agreement between NYSED and the vendor. The vendor must include as much variability as possible in the performance indicators tested by field test questions such that new operational tests can be developed with similar variability from year to year.
  6. Maintain the security of all field test items and test forms.
  7. Score the field tests, verify the results, forward the score results to NYSED in a mutually agreed-upon format, and produce a Field Test Technical Report for each year of field testing.
  8. Retain all field test data throughout the contract and for an additional five years thereafter.

For the stand alone field tests, the vendor must:

  1. Administer field tests within one month of the administration of the operational examination.
  2. Develop, print, and ship field tests, teacher directions, student test booklets, and answer sheets and directions for administering and returning the tests to the vendor.
  3. Organize field tests in mini-forms such that students can generally complete the form in a 40-minute class period.
  4. Design and print the field test answer sheets or use scannable field test booklets for students to record their responses.
  5. Provide each participating school with an easy-to-follow set of instructions that explains only how to administer a field test in the grade band and subject assigned to that school. Therefore, the instructions need to be provided in separate booklets by grade band and subject.
  6. For Mathematics Tests in 2011 only, translate the English language multiple-choice items into Spanish and provide a method for those schools selected for field testing in mathematics to order Spanish versions if necessary. Schools are not specifically selected for populations using the Spanish version. The vendor is not required to translate constructed-response items for field testing. During the administration of constructed-response mathematics field test forms, those students using a Spanish version will be administered multiple-choice items in Spanish.
  7. For field tests in 2011 only, provide one multiple-choice field test form per grade in large-type and braille and provide a method for those schools selected for field testing to order large-type and braille versions if necessary.
  8. Provide notice to schools on each page of the field test forms and administration directions that photocopying materials and any reference to test content is strictly prohibited. The notice must read as follows:
    Secure materials. All rights reserved. No part of this booklet may be reproduced or transmitted by any means. Use of these materials is expressly limited to the New York State Education Department.
  9. Collect a minimum of 2,000 usable field test answer papers per field test form. Since field test participation in NYS is not mandated, the vendor must oversample in order to achieve the target of 2,000 answer papers written for each field test form.
  10. Ensure that the data drawn from each study has sufficient reliability across the student population of the State and within student sub-populations required for accountability under the No Child Left Behind Act (NCLB).
  11. In collaboration with NYSED, select the schools to participate in the stand alone field testing based on NYSED’s sample design (see Attachment H). Field tests are administered by New York State public schools (including charter schools) and nonpublic schools.
  12. Notify the selected schools in writing of the field tests. The solicitation by the vendor must be explicit as to the grade levels and subject for which the school is asked to participate and must be approved by NYSED.
  13. Send written notification to schools selected for field test participation that have failed to submit their enrollments for participation at least two weeks prior to the established deadline for responding to field test solicitation.
  14. Provide all schools that respond affirmatively to the field test solicitation with the appropriate quantities of field test materials for that grade(s) and subject even if it appears to the vendor that more than the target number of answer papers will be achieved.
  15. Pay for the shipping costs for schools to return the field tests to the vendor.
  16. Serialize all secure materials distributed to schools as part of the field tests. The vendor’s records and the shipping manifest must indicate the range of serial numbers shipped to schools. The vendor must also account for each individualized secure item in checking returned materials. Follow-up correspondence must include the serial number of the missing materials.
  17. Assure that all field tests and related materials are returned in a timely and secure manner, and notify schools repeatedly, if necessary, that do not return materials by the due date. The vendor must provide a list to NYSED of school districts, charter, and nonpublic schools that have failed to return materials within 2 weeks after the due date and again after the vendor has made a minimum of three concerted efforts to effect their return. The vendor must provide documentation to NYSED of its efforts to contact schools and have materials returned.
  18. Perform all activities needed for the handscoring of constructed-response items. NYSED must approve the vendor’s plan for rangefinding and the scoring process, including the scoring rubrics and training materials (practice sets and annotated anchor sets).
  19. Employ only scorers with a minimum of a bachelor’s degree. For scoring ELA Tests, such scorers must have at least nine college credits in English and/or Education. For scoring Mathematics Tests, such scorers must have at least nine college credits in Mathematics and/or Education. The vendor must provide all training for scorers.
  20. Propose a plan to equate those items administered in the stand alone field tests to the operational test scale. Field tests must be matched by student to their operational test results to permit common scaling of items to the operational scale.
  21. Retain all student responses to constructed-response items, on paper or as scanned images, through the end of the contract.

For the embedded field tests, the vendor must:

  1. Divide all multiple-choice items to be field tested, except those associated with the ELA listening passages, into four equal sets of field test items. The full census of schools across New York State will be divided into four representative groups of schools by NYSED through random selection, and each of the four representative groups of schools will receive an operational test with a different set of field test items embedded in Books 1 and 2. (See Attachment G.)
  2. Embed field test items in different locations each year, such that it is not obvious which items are part of the field test and which are operational items.
  3. Provide NYSED with operational test forms including the embedded field test items according to the timelines and specifications provided elsewhere in the RFP. See the section on Operational Forms Preparation for additional information.
  4. Provide all field test items in large-type and braille formats. The large-type editions will be provided in camera-ready format for printing by NYSED. The vendor will provide NYSED with the number of braille booklets requested by schools plus 25 additional copies of each booklet. (See Attachment B: Specifications for Large-Type and Attachment C: Specifications for Contractor/Subcontractor Transcribing Braille.)
  5. Translate all mathematics field test items into the five alternate languages required on operational tests: Chinese (Traditional), Haitian-Creole, Korean, Russian, and Spanish. All translated items must be back translated. The vendor and its subcontractor, if there is one, must refer to bilingual mathematics glossaries and past translations of Grades 3–8 Mathematics Tests, which are posted on NYSED’s web site at http://www.nysedregents.org/, for guidance as to NYSED’s preferred word usages, formatting, and style for all translations.
  6. Receive raw data files containing student responses from the Student Information Repository System (SIRS), in the same way such files are received for operational tests. (See items #2, 4, and 5 of the Local Scoring of Operational Tests section of the RFP for information.) The vendor must score all student responses and conduct all analyses.

Field Test Sample Design

A stratified random sampling design will be used for the stand alone field tests. All public, nonpublic, and charter schools with Grades 3–11 enrollment will be required to participate in the annual field tests. Each school will be asked to administer mini field test forms in only one grade.

A five-year field test schedule is designed to accommodate all New York State elementary and intermediate school field tests in 2010–11 through 2014–15 school year (see Matrix in Attachment H). The Matrix contains 10 pre-selected elementary school samples (ES1 to ES10) and eight Intermediate school samples (MS1 to MS8). NYSED will be revising the high school field test matrix to accommodate the Grades 3–11 ELA Tests. Every school with Grades 3–11 enrollment will be assigned to one field test sample but not more than one sample. All samples represent stratified random samples that are representative of New York State schools. Schools that are assigned to the field test samples (called strands) will remain in the same strands for the duration of the five-year field test schedule. However, their field test assignments will be rotated from year to year. The vendor may select a subsample from each strand depending on the field test needs.

NYSED will provide the vendor with a complete list of New York State schools and related demographic variables for sample selection. The vendor is responsible for selecting the stratified random samples. Vendor must ensure that all samples are representative of the New York State schools with regard to number of schools and students N counts by grade, distribution across the Need/Resource Capacity Categories (for more information see page 100 of the 655 Report found at http://www.p12.nysed.gov/irts/chapter655/2006/volume1.pdf), ethnic composition, school type, and mean scale scores in Grades 3–11 ELA and Grades 3–8 Mathematics Tests. The sample file must be submitted to NYSED for review and approval.

Analysis of Field Tests

The vendor will provide statistical analyses for the field test administration. The analyses must include, but not be limited to, the following:

  • Analyses of field test participation rates;
  • Evaluation of the field test sample representativeness;
  • Analyses of classical item statistics;
  • Analyses of Item Response Theory (IRT) item parameter estimate including the item difficulty and discrimination;
  • Evaluation of the item fit and model fit statistics;
  • Evaluation of the classical and IRT Differential Item Functioning (DIF) results;
  • Overall statistical indices of field test forms reliability;
  • Reliability and validity information; and
  • Inter-rater reliability.

The Mantel-Haenszel process for selected-response questions, a derivative process for constructed-response questions, or an IRT-derived process for both item types, must be used to evaluate DIF on field test items for males and females and for population focal groups of sufficient size identified by NYSED. DIF analysis will also be conducted on the field test versions of mathematics test items field tested in English compared to Spanish, plus any other alternate language edition where the N size is 200 or greater. All high DIF items will be evaluated by the vendor and NYSED for suitability for inclusion in operational tests. A copy of each research file containing all field test item responses must be provided to NYSED after each field test.

Field Test Technical Report

The vendor will develop and submit to NYSED a technical report for each field test administration that will include, but not be limited to, the following information:

  • Field test design;
  • Field test sample and participation rate;
  • Field test and operational data match;
  • Classical analysis (item analysis, inter-rater reliability, and DIF analysis);
  • IRT Scaling (item calibration, item equating, item fit evaluation and IRT DIF); and
  • Item flag summary and research guide for item selection.

Back to top

Go to Table of Contents


Operational Test Administration

The vendor must describe how each operational test will be prepared and submit a timeline for each test that provides specifics on major events, phases, and deliverables. The plan must include appropriate time for NYSED to review and approve carefully edited copies of all test items, teacher directions, and scoring materials.

Operational Forms Preparation

  1. The vendor must provide to NYSED, for its approval, a style guide for the Grades 3–11 ELA and Grades 3–8 Mathematics Tests delineating the content specific terminology, phrases, graphics, symbols, and grammatical style. In addition, the vendor must refer to the Core Curricula, the Instructional Recommendations for Elementary and Intermediate Mathematics Instruction, and the Lists of Suggested Mathematics Vocabulary for Grades 3–8 for guidance on terminology.
  2. Vendor must annually update the style guide with approval from NYSED.
  3. Materials to be produced include all test booklets, including braille, large-type, and translated editions; teacher directions and school administrator manuals; dictation copies; scoring guides for local scoring; and any other examination-related materials necessary for valid administration. Instructions in manuals for teachers and administrators must be clear, complete, and specific. Whenever possible, directions across grade levels and within content areas should be consistent.
  4. The vendor must provide NYSED with both camera-ready and PDF format files with all fonts embedded in the PDF file. All pages must be single-sided and numbered (except the cover). The PDF files must be in gray scale with no colors and at 600 dpi. In addition, the vendor must submit the final QuarkXpress file of the examination and all related art files (either in Freehand MX or Illustrator CS2 or higher). The vendor must use Mac version QuarkXpress 6.5 or 7 (OS X) to typeset the examinations. The vendor will be responsible for verifying the requirements at the time of delivery. This requirement applies in all instances where the term "camera-ready" is used.
  5. Grade and subject designations, as well as New York State Testing Program must appear on all test materials. Grade appropriate graphics, content, style, typeface, and font size must be determined by the vendor and approved by NYSED.
  6. The vendor will provide NYSED with the materials noted in this section prior to the date of test administration according to a mutually agreed upon schedule. All products and procedures proposed will be reviewed and approved in draft and final form by NYSED, with a minimum of 20 days for review. Materials for NYSED approval will be provided by the vendor in a finished form.
  7. The vendor must propose/construct the operational test forms based on the field test results and research guidelines for item selection. The vendor must also propose additional field tested items for possible replacements.
  8. Item selection for operational test forms will be held at the NYSED building with both vendor and NYSED staff in order to review the proposed forms and make necessary edits. The vendor must provide proposed forms and all field test statistical data for review. At least two weeks prior to item selection, the vendor must provide NYSED with all items and item statistics, including classical and IRT statistics for the items selected and the replacement items. These materials may be sent using the secure FTP site. At the time of item selection, the vendor must also provide NYSED staff with a mapping of all proposed items to the test specifications as well as the performance indicators associated with the possible replacement items.
  9. The final operational forms will be jointly reviewed by the vendor and NYSED and approved by NYSED.
  10. Annually, the vendor must convene final eyes committees, comprised of four to six teachers per grade band (3–5, 6–8, and 9–11 for ELA and 3–5 and 6–8 for Mathematics), who represent the state demographically and geographically, to review the proposed final form of each test before printing. (See Attachment F for specifications.) At the conclusion of the final eyes committees, the vendor must conduct debriefings with NYSED to resolve any concerns raised by the committees.
  11. The vendor will prepare and ship to NYSED the quantities that schools request of braille editions of each test plus an additional 25 braille copies of each test booklet. It is expected that the vendor will be required to provide a total of 125 booklets to NYSED in braille for each test booklet. NYSED will provide the vendor with exact quantities to be produced after schools submit their requests. The braille editions must be accurately labeled. The large-type editions must be prepared in camera-ready format for NYSED to print. (See Attachment B: Specifications for Large-Type and Attachment C: Specifications for Contractor/Subcontractor Transcribing Braille.)
  12. Mathematics tests will be translated into Chinese (traditional), Haitian Creole, Korean, Russian, and Spanish. Translated editions must be back translated. The ELA tests are not translated. The vendor and any subcontractor must refer to bilingual mathematics glossaries and past translations, which are posted on NYSED’s web site at http://www.nysedregents.org/, for guidance as to NYSED’s preferred word usages, formatting, and style.

Local Scoring of Operational Tests

  1. The vendor will provide a toll-free number which is staffed from 7:30 A.M.–4:30 P.M. EST each business day to handle all test-related questions for the duration of the scoring period, which begins after each testing period ends. In addition, the vendor will ensure that a toll-free FAX number, with a minimum of three FAX machines, shall be operational 24 hours a day for the duration of the scoring period.
  2. All operational test multiple-choice questions, including embedded field test and anchor items, will be scanned by the Regional Information Centers (RICs) and the large city (Buffalo, New York City, Rochester, Syracuse, and Yonkers) school district scanning centers. RICs are data processing centers operated by twelve Boards of Cooperative Educational Services (BOCES) that provide answer sheets and scanning services for schools outside the five large city school districts.
  3. All operational constructed-response questions are scored by New York State teachers in regional or district-level groups and scanned at the RICs, including the large city school districts. New York State is allowing local districts to choose regional scoring at sites identified by the BOCES and the large city school districts, or choose district-level scoring. NYSED may choose to switch to a centralized scoring model during the term of the contract resulting from this RFP.
  4. Bidders must propose a plan to NYSED to collect data and provide quality control for all data records. In this plan, all student answer documents must be scanned at the RICs where both the multiple-choice and constructed-response questions are added to an electronic file. These electronic records are sent to a central location, New York State’s Student Information Repository System (SIRS), for aggregation into one data file. This file is then provided to the vendor for scoring and scaling analysis. The vendor will share a copy of a research file containing statewide item level responses for each operational examination with NYSED.
  5. Student operational test answer sheets are developed by the RICs and large city school districts scanning centers. The vendor must cooperate with the RICs and large city school district scanning centers to facilitate the development of the most appropriate student operational test answer sheets. The vendor must also work cooperatively with the SIRS to facilitate the transfer of statewide student operational test data. NYSED will provide the vendor with contact information for the RICs, large city school district scanning centers, and the SIRS.

Scoring Materials for Operational Test Constructed-Response Items

  1. The vendor will provide clear, complete, and specific scoring guides that are derived from the vendor-prepared field test scoring materials. The vendor will also provide rubrics, student paper exemplars at each score point, practice sets for scorer training and consistency assurance sets.
  2. The vendor will develop two levels of scoring rubrics for constructed-response items. The first level of rubric, which will be public, is a generic rubric applicable to all questions with the same possible score range, (e.g., 0–2). Generic rubrics will communicate how point values will be assigned for each level of response in terms of the general competency displayed.
  3. The second level of rubric is an item-specific rubric. These rubrics are considered secure until the operational test forms to which they apply are scored. Item specific rubrics detail how the scoring principles expressed in the generic rubrics are applied to the specific responses for each constructed-response question.
  4. Camera-ready scoring materials must be delivered to NYSED by the first week in October for ELA and the first week in November for mathematics.
  5. The vendor-prepared camera-ready scoring materials will be printed and disseminated to schools by NYSED or posted by NYSED on NYSED’s web site. The vendor must provide all scoring materials in a format that can be posted online.
  6. In addition, the vendor must provide audio training materials on CDs for the scoring of each test. The vendor must ensure an error-free alignment between the scoring information provided on the CDs in spoken word and the information provided in the written scoring materials. The vendor will be responsible for developing the training material, producing the CDs, and providing 7,500 copies to NYSED for each ELA and Mathematics Test to be disseminated by NYSED along with the printed scoring materials.

Turnkey Training for Statewide Scoring

The vendor will provide initial regional turnkey training during the 2011–2012 school year on how to use the generic rubrics for scoring, the design and administration of the Grades 3–11 ELA and Grades 3–8 Mathematics Tests, and how to score the constructed-response items. Administrators, teachers, and BOCES representatives will be trained as turnkey trainers, and will then train teachers at the school level to score the constructed-response items.

The training shall provide examples from sample tests or other sources of student work for each score point on the generic rubric. The training shall demonstrate how to complete the required forms from the user's manuals to implement the program and record the results. Training materials to be used will be developed by the vendor, and must be approved by NYSED. Test and scoring materials that are disseminated for the scorer training prior to the administration of the operational test must not contain any secure test materials.

Fourteen one-day training sessions for each content area must be held. Sites must be geographically distributed to cover the State with at least four sites in New York City. The vendor must have at least six staff members present at each session, including staff development, scoring, and program management staff. NYSED staff may be present, largely as observers, during the training sessions. Up to 100 teachers may attend each training session.

The training sessions must be completed by October 2011 for ELA and November 2011 for Mathematics. Specific dates will be determined in collaboration with NYSED, BOCES, and the large city school districts.

The vendor will be responsible for arranging and paying for training sites and for all associated costs of the initial turnkey training. Teachers will not be paid honoraria for the time spent in training or be reimbursed by the vendor or by NYSED for their travel costs. The BOCES and large city school districts can be contacted to assist with the planning of sites for training. NYSED will provide the necessary contact information for the BOCES and large city school districts to the vendor.

The vendor will be responsible for producing a technical report on the turnkey training for statewide scoring. The technical report must include:

  • The materials used to conduct the turnkey training;
  • Locations of the turnkey training;
  • Dates and times of the training at each location;
  • Number of participants at each location;
  • Demographic information concerning each participant including school and district name and city, ethnicity, and current teaching assignment, e.g. Grade 3;
  • Summary of the evaluations of the training by participants; and
  • Names, qualifications, and titles of the trainers.

Updating of the Guide

The vendor must annually update the online addition of the “Guide to the Grades 3–11 English Language Arts and Grades 3–8 Mathematics Tests” found at http://www.p12.nysed.gov/osa/ei/gr3-8guide10.pdf. The guide must address: how the tests were developed; how they are administered; how scores will be used and in what format they will be reported; how many test books there will be by grade and the length of testing time for each book; what types of test questions (items) will be included (with examples); what standards will be assessed; and test specifications. In the first year, the guide must include test samplers for the ELA Tests in Grades 9–11. In subsequent years, sample questions should be included in the guide to reflect any changes to the test specifications.

Back to top

Go to Table of Contents


Research

Forms Scaling and Equating

Vendors must use IRT 3-Parameter Logistic Model for multiple-choice items and propose to use either partial credit model or graded response model for constructed-response items to calibrate, equate, and scale the field test and operational test forms. The equating process must allow for year-to-year comparisons for each test through the development of a common scale across years.

Post equating methodology must be used to equate the operational test forms using the non-equivalent groups anchor test design. Most, but not all, of the anchor items will be provided to the vendor by NYSED from pre-established anchor items that have been used by NYSED for a similar purpose. The vendor must revise or augment the anchor items as necessary to ensure alignment to any changes to the learning standards or test specifications. The full set of anchor items for each grade must resemble the operational test with regard to the test contents and statistical specifications. In spring 2011, the anchor items will be compiled into field test anchor item forms and spiraled within classrooms with other field test forms.

Starting in spring 2012, the anchor items will be embedded into the operational test forms. The anchor form in each grade will be divided into four mini sets of anchor items, each containing six multiple-choice items. The full census of schools across New York State will be divided into four representative groups of schools by NYSED through random selection, and each group of schools will receive an operational test with a different set of six anchor items embedded in Book 1 or 2 (see chart below and Attachment G). In the ELA Tests, these six items will be connected to one reading passage. Consequently, across the four groups 24 anchor items will be embedded in the operational test administered at each grade level for each of the two test content areas (ELA and Mathematics). These anchor items are not counted toward students’ test scores and will not be released to the public after the test is administered.

Anchor Items

Year

Anchor Items included in
Stand Alone Field Tests

Total Anchor Items Embedded
in Operational Tests

Anchor Items
Embedded in Each Operational Test Form

2011

24 items in each grade

NA (not part of this contract)

NA (not part of this contract)

2012

None

24 items in each grade

6 items in each test form

2013

None

24 items in each grade

6 items in each test form

2014

None

24 items in each grade

6 items in each test form

2015

None

24 items in each grade

6 items in each test form

Each operational test form can be placed onto the operational scale through the secure anchor items using either Fixed Common Item Parameters (FCIP) or Stocking and Lord Method. The stability of the anchor items must be examined before being used as anchors in the equating process using the delta plots and Lord’s chi-square.

Constructed-response items and all items associated with ELA listening passages will require a separate field test administration for future use as operational items using the specified field test matrix sampling design (Attachment H). The bidders must propose the most appropriate approach to equate the stand alone field test forms to the operational scale under the post equating design.

Operational test items can be selected from the item bank established through embedded field test items and stand alone field tests based on content coverage and statistical specifications. Operational test data must be used for item calibrations and equating and for creating the operational conversion tables. Number correct scoring approach must be used to produce the raw score to scale score table using the test characteristic curve (TCC). The vendor must submit to NYSED the data file used for equating and calibration for review and verification.

If the vendor is using proprietary software to scale and equate the test forms, the vendor must provide NYSED’s research staff with access to the software and training in its use at no additional charge to NYSED for the duration of the contract and three years after the end of the contract.

Linking Study

Vendor must propose an appropriate method and conduct a study to link the old and new Grades 3–8 ELA and Mathematics Tests in order to provide continuity of the testing program for accountability purposes. The assumptions and requirements of linking methodology, including test content, condition of measurement, and examinee population, should first be examined. Different types of linking can be considered including test equating, linking tests in transition, concordance, etc. (Dorans, Pommerich, and Holland, 2007). The vendor must decide the most appropriate linking method in conjunction with NYSED. The linking study should be able to address the questions listed in Linn, McLaughlin, and Thissen’s 2009 paper, Utility and Validity of NAEP linking efforts (p. 17), “there are at least six different kinds of questions that can be answered through test linking that cannot be answered without linking.” In addition, the degree of accuracy and the Percent Above Cut (PAC) should be studied with proximate results in the linking study. The design and results of the linking study must be provided to NYSED as part of the operational test technical report.

Operational Test Technical Report

Vendor must submit a comprehensive technical report after each operational test administration. The technical report must include, but not be limited to, all the technical information such as test design, test development, scoring, test reliability and validity, and analysis of the operational test data.

All research files must be submitted to NYSED containing complete student item-level data as well as scale scores and the demographic information necessary to replicate all analysis.

The vendor must retain all operational test data through the contract period and for five years after the end of the contract.

The vendor will provide a comprehensive technical report for each test (see Attachment I) to include:

  • Disaggregating test results at various levels;
  • All psychometric and statistical data;
  • Reliability data, including internal consistency and decision reliability, and validity data;
  • Documentation of item linkages to the standards;
  • Intercorrelations by content-specific sections and item types;
  • Generalizability analyses must be provided; and
  • Item-level data must also be provided.

Examples of technical reports prepared for prior years’ tests are provided on NYSED’s web site at: http://www.p12.nysed.gov/osa/reports/.

Standard Setting

As part of the process for determining appropriate cut scores, the vendor will be required to conduct formal standard setting studies following the administration of the 2012 operational tests. The vendor will also be required to implement Vertically Moderated Standards (VMS) in 2012. VMS design requires separate scales across grades, and must include comparable proficiency cuts that show a smooth and rational pattern of percentages of students scoring at each of the proficiency levels across grades.

The Standard Setting process for the Grades 3–11 ELA and Grades 3–8 Mathematics Tests consists of four activities: the Pre-Measurement Review Meeting, the Empirical Review, the Instructional Impact Review/Standard Setting, and the Post-Measurement Review Meeting. The main purpose of these meetings is to obtain cut score recommendations for NYSED for each grade level.

Following the administration of the 2010 operational tests, NYSED piloted an approach to setting the cut scores for the Grades 3–8 Tests using statistical analyses of college readiness. (See http://usny.nysed.gov/scoring_changes/ for more information.) NYSED will share this approach and the necessary data with the vendor. Bidders must propose an approach, based upon the approach used in 2010, to align cut scores for the Grades 3–11 ELA and Grades 3–8 Mathematics Tests with college-ready performance. The standard setting process must include both an empirical review and an instructional impact review/standard setting by teachers. The standard setting must be completed within one month after the tests have been scored and scanned.

For each content area (ELA and mathematics), the vendor will be expected to conduct Pre-and Post-Measurement Review Meetings in Albany about two months before and immediately after the Instructional Impact Review/Standard Setting for the Grades 3–11 operational tests in ELA and the Grades 3–8 operational tests in mathematics. These meetings will be comprised of approximately 38 New York State administrators and policy makers, as well as vendor staff and up to six NYSED staff. Participants at these meetings will not receive honoraria, but the vendor will be responsible for paying for travel, meals, and lodging. The vendor will not provide reimbursement to any NYSED staff for travel, meals, or lodging costs.

At the Pre-Measurement Review Meeting, the vendor will provide participants with background information and data (e.g., historical assessment results), training on the standard setting process, and a review and explanation of the standard setting materials. NYSED will provide the vendor with names of individuals to contact to determine availability and willingness to participate.

A three day Instructional Impact Review/Standard Setting is to be conducted by the vendor in Albany, for each ELA and mathematics, to be completed following the Empirical Review. For mathematics, the meeting will include approximately 36 teachers, including 12 college professors, who will be divided into groups of 18 teachers in two rooms. The teachers in the first room will be developing performance level descriptors and offering guidance on the cut scores for the tests administered in Grades 3, 4, and 5; teachers in the second room will address Grades 6, 7, and 8. For ELA, the meeting will include approximately 54 teachers, including 18 college professors, divided into three rooms: 18 for Grades 3, 4, and 5; 18 for Grades 6, 7, and 8; and 18 for Grades 9, 10, and 11. Each room will need at least four tables to accommodate the participants, vendor, and NYSED staff. NYSED will provide the names and contact information of the teachers and college professors that the vendor will contact to determine availability and willingness to participate. Each of the teacher tables will have a table leader selected by NYSED. The leader’s role is to facilitate table discussion, ensure participation of all table members and report summaries of table discussions to the larger group in each room.

In the afternoon of the third day of Instructional Impact Review/Standard Setting, the vendor will convene a group composed of all the table leaders for all rooms. Their purpose is to review the performance level descriptors and guidance on cut scores arrived at for each grade level in the separate rooms and to make recommendations on revisions to those determinations that help to provide a logical vertical articulation across the spectrum of Grades 3–11 for ELA or Grades 3–8 for mathematics.

The next day the vendor is to conduct the Post-Measurement Review Meeting with the same attendees who attended the Pre-Measurement Review Meeting. The purpose of this meeting is to develop performance level descriptors and offer guidance on cut scores to NYSED based in a review and integration of the recommendations that had been provided at the Pre-Measurement Review Meeting as well as at the Instructional Impact Review/Standard Setting.

The vendor is responsible for:

  1. arranging for conference room space in Albany where these meetings will take place and where participants residing more than 50 miles from Albany will be given lodging.
  2. contacting and sending invitations to the participants whose names NYSED has provided.
  3. securing signature of participants on the non-disclosure agreements provided by NYSED.
  4. paying for conference room space, meals, and travel reimbursements. In addition, the vendor is to provide honoraria to teacher participants in the Instructional Impact Review/Standard Setting but not to participants in the Pre-and Post-Measurement Review Meetings.
  5. developing the agenda, handouts, Power Point presentations, and sign-in and sign-out sheets.
  6. maintaining the security of materials and arranging for secure destruction services at the conclusion of Standard Setting.
  7. submitting the Standard Setting Technical Report which includes sections on the Pre-and Post-Measurement Review Meetings, the Empirical Review, and the Instructional Impact Review Meetings to NYSED.

A comprehensive final report on the Standard Setting study is required. A draft of the technical report must be submitted to NYSED within two weeks of the completion of the standard setting. The final technical report must be submitted within two weeks of receiving requested changes from NYSED.

Back to top

Go to Table of Contents


Program Management

The vendor must submit documentation demonstrating personnel experiences and expertise congruent with staff assignments for all aspects of the tasks outlined in the RFP. The vendor shall provide plans and procedures for ensuring that staffing is commensurate with the project’s scope of work throughout the duration of the contract and immediately notify NYSED of any changes in staffing including any subcontractors. NYSED reserves the right to review and approve any proposed changes in key staffing and/or subcontractors.

Detailed and sufficient information must be presented to lead evaluators to the efficacy of the proposed staffing solutions for this project and organizational arrangements of the same. This information must include:

  1. An organizational diagram that indicates staffing arrangements for this project.
  2. The time commitments of the proposed staff. Time commitments must be described in terms of projected hours per week devoted exclusively to the NYSED project.
  3. Provide a detailed description of responsibilities for all proposed staff.
  4. Detailed resumes for all staff members associated with this project.

The bidder must identify two program managers (one each for ELA and mathematics) to be the central points of contact with NYSED for the contract. The program managers and all other key staff, including the four ELA content specialists, three mathematics content specialists, development supervisor, and research staff, must be identified by name and experience in writing. Current resumes for these staff members and any other key staff must be included as part of the proposal.

Bidders are expected to demonstrate that the individuals identified as program managers have sufficient authority across departments within the organization to ensure that the work of the contract has the necessary priority to be completed with the highest quality and on time.

Program Manager Requirements

  1. The two program managers for this contract must not have primary responsibility for any other contract with NYSED or for a large scale testing program in another state.
  2. A detailed resume and two professional references that can attest to the individuals’ qualifications and experience must be included with the proposal.
  3. At least three years of experience managing large scale assessment projects from conception through completion following industry recognized project management methodology is required.
  4. Previous experience in a Grades 3–8 education project is required.
  5. A demonstrated knowledge of testing procedures and strong organizational, managerial, and communication skills are required.
  6. A bachelor’s degree is required. A master’s degree or above is preferred.
  7. Project management certification through the Project Management Institute (PMI) as a Project Management Professional (PMP), or other recognized program management certification is preferred. If the proposed program managers hold such certification, evidence of the certification must be included in the proposal.

Program Manager Duties

The program managers will:

  1. Travel to Albany at least four times a year to meet with NYSED program staff in the NYSED building. A start-up meeting must be held in Albany at the beginning of each contract year.
  2. Develop and submit an annual detailed project plan to NYSED for review and approval. The purpose of the plan is to provide NYSED with an overall analysis of the methods the vendor will utilize to perform all aspects of the RFP and contract in the required timeframe. The vendor will not perform work on the project until NYSED has approved the vendor’s project plan.
  3. Maintain accurate, up-to-date information of the current status of all vendor and subcontractor(s) work on the project, and timely communicate such information to NYSED.
  4. Coordinate and participate in quarterly review meetings and weekly conference calls to discuss the project status and any issues related thereto. Within one week following each quarterly review meeting, the vendor must provide meeting minutes to NYSED. The vendor must also provide a written statement on all important decisions made during weekly conference calls.
  5. Submit bi-weekly status reports electronically updating NYSED on all phases of the development and implementation of the project and progress made in accordance with the detailed project plans and contractual requirements. Such bi-weekly reports must include the following:
    • A current detailed listing of all work and activities completed and in progress with corresponding dates;
    • A detailed list of upcoming work;
    • A list of all information needed from NYSED in order for work to proceed;
    • External dependencies, if any, which may affect the schedule, and the variance by which they would affect the schedule; and
    • All dates must correspond to all timelines and schedules for deliverables.
  6. Submit all deliverables and other work products to NYSED in the manner designated by NYSED for review and approval prior to any release or distribution by the vendor. All materials must be appropriately proofed by the vendor for errors. The program manager must submit materials to NYSED according to a mutually agreed upon timeline to allow for appropriate review.
  7. Oversee the development and implementation of changes as necessary to ensure that the projects remain within specified scope and are within time, cost, and quality objectives.

Notification Procedures

In the event that a problem or potential problem arises with regard to quality, timeliness, or any other issue with respect to deliverables and services at any time during the contract term, regardless of when the problem arises, the program manager must immediately notify the Director of State Assessment, or his/her designee, via telephone and in writing of the issue and the vendor’s proposed solution and shall also include the issue and NYSED approved solution on any subsequent report(s).

NYSED Approval

NYSED shall have approval authority over all aspects of the Grades 3–11 ELA and 3–8 Mathematics Testing Programs including, but not limited to, the following:

  1. All material, products, and services produced by the vendor, including, but not limited to, all test items and passages, test forms (test booklets and answer documents), brochures, guides, student information labels and answer sheet identifiers, manuals, teacher’s directions, and any and all other printed materials produced for this project. In addition, all test forms produced in braille or large type must be reviewed and approved by NYSED before final copies are provided to NYSED.
  2. All development and operational plans for field testing items.
  3. All plans and timelines for item development, item replenishment, scoring, and reporting.
  4. All schedules, including but not limited to training schedules, rollout schedules, implementation schedules, scoring/reporting schedules and item review schedules.
  5. All reports, including but not limited to, technical reports (ongoing and post-operational), score reports, and field test reports.
  6. Any changes to project staff, including program managers, content, or research staff on and after award of the contract and commencement of work for the duration of the contract.
  7. Any change orders. Any changes to services or deliverables under this procurement must be in accordance with a written description clearly justifying the need for the change and identifying all rational for the change (Change Order). The New York State Office of the State Comptroller is the only entity that has the authority to approve the modification of an agreement between NYSED and a vendor. No work that is not already contracted for should be started by the vendor prior to its official approval by NYSED and the Office of the State Comptroller.

Security

Security of individual student data and test items, including the thorough retrieval and accounting for all field test booklets, is essential. The vendor must adhere to all of NYSED’s security protocols regarding transmission and shipping of secure materials. The vendor must use encrypted files and design, host, and maintain a secure file transfer protocol (FTP) site as a means of file transfer. Access to the NYS test information on this site must be limited to the vendor and NYSED unless further sharing with other parties is authorized in writing by NYSED. Any other electronic transfer via e-mail, Internet, or facsimile (FAX) of individual student information or any secure test materials is not permitted unless authorized by NYSED to do so on a case-by-case basis. When shipping secure materials, the vendor must use a delivery service with online ground tracking capabilities. All shipments between the vendor and NYSED or the vendor’s subcontractors must be in locked boxes, which will be supplied by NYSED. Bidders must describe all aspects of security in their proposals, which must, at a minimum, be consistent with Attachment D.

Qualifications and References

Bidders must document experience in the development of secure, high volume, standardized assessment programs including both multiple-choice and constructed-response items. Qualifications for test development, field testing, scoring, research, test material preparation, and test administration must also be documented as appropriate. Bidders must demonstrate the capacity to complete this project using their own hardware and software.

The proposal should include a minimum of three current professional references to substantiate these qualifications. Current shall mean relevant within the past three years. References must include name, title, company name, address, phone number, and e-mail address. Do not use NYSED staff as references. The proposal should include the dates of service and a brief summary of the services provided.

Contract Period

The contract resulting from this RFP will be for a single award with a term of five (5) years. The contract will begin on January 1, 2011 and end on December 31, 2015.

At the end of any contract term otherwise provided for herein, if a replacement contract has not yet been approved in accordance with State law, any contract awarded hereunder may be extended unilaterally by the State, upon notice to the contractor, at the same terms and conditions, including all contract pricing, for a period of one month, if funding is available and/or services are needed. Additionally, this extension may be for a period of up to three months with the concurrence of the contractor. However, any extension will terminate immediately upon approval of the replacement contract except where a period for transition of contractors has been previously provided for.

Terms and Conditions

  1. All tests and test-related materials submitted to NYSED pursuant to this proposal are required to be original materials developed solely by the bidder for use in satisfaction of this contract, with the exception of the nationally normed items included on field and operational tests.
  2. All examinations (including field and operational test items in both print and computer file format) and examination-related materials, with the exception of the nationally normed test items to appear on field and operational tests, are the exclusive property of NYSED and cannot be used by the vendor or any subcontractor for any purpose other than what is defined in the contract.
  3. Except as indicated above, NYSED is the sole owner of outputs resulting from the work proposed in the RFP, including, but not limited to: item banks, technical documents, test questions developed for field tests, operational tests, all scoring materials, test samplers, manuals, scoring rubrics, score scales and all guides, booklets, papers and reports generated under this contract. The vendor must be prepared to deliver all or part of these outputs to NYSED at any point during the term of the contract. Except as indicated above, these materials can only be used by NYSED. Ownership rights are more fully set forth in Appendix A-1 which appears in the contract.
  4. All such outputs are to be turned over to the State in print/or electronic form, as specified by NYSED, prior to the final payment to be issued under this agreement. The State may accordingly, at its discretion, make public, and may publish on its web site, all examinations and related materials noted in the preceding paragraph after administration of the operational examination.
  5. All materials are to be delivered, as appropriate, in copyright form. Where original works are included in the body of the material, an acknowledgment statement must be included, setting forth the copyright information with respect to such original works.
  6. All materials written or revised under this Agreement are to be held strictly confidential and may not be copied, duplicated, or disseminated in any manner other than as authorized in writing by NYSED. The vendor must obtain permission from copyright holders to use copyrighted work on the tests. With the exception of the nationally normed items appearing on field and operational tests, permissions from copyright holders regarding the use of 75% of the copyrighted works appearing on the tests must include permission to post these materials on NYSED’s web site.
  7. The vendor (and its employees, agents, consultants, or subcontractors arising out of this contract) will not disclose any information pertaining to NYSED's test content, specifications, or development to any person(s) or entity that does not need to know such information for the purpose of carrying out the tasks specified by the contract.
  8. If the vendor also develops or publishes textbooks or practice tests for adoption within the State of New York, the vendor agrees that, when promoting to New York State customers such textbooks, ancillary materials, and/or practice tests, the vendor will not make any reference to the vendor having been awarded the contract or the vendor's performance of the services for NYSED contemplated by the contract.
  9. If the vendor is under ownership with another entity that develops or publishes textbooks or practice tests for adoption within the State of New York: a) ("Related Publisher"), the vendor reiterates the restriction against disclosure as stated; b) the Related Publisher is restricted from providing any information to the vendor pertaining in any way to content uses, or proposed to be used, in the Related Publisher's development of textbooks or practice tests for adoption within the State of New York; and c) the Related Publisher is restricted, when promoting to New York State customers, the Related Publisher's practice tests and/or textbooks developed for adoption within the State of New York, from making any reference to the common ownership of the Related Publisher and the vendor or any reference to the vendor's performance.

Cooperation with Other NYSED Contractors

NYSED currently has, or may develop, contracts for certain test-related activities that are associated with the Grades 3–11 ELA and Grades 3–8 Mathematics Tests. These services include re-scoring of a sample of students’ responses to constructed-response questions and the development of individual student score reports. In the event that any of these contracts are held or awarded to other vendors, the vendor awarded the contract tied to this RFP must cooperate fully and in a timely manner with those vendor(s) providing other services associated with these examinations.

Considerations and Other Issues

Upon completion or termination of the contract awarded as a result of this RFP, the vendor will use its best efforts to assist NYSED in completing a seamless transition to any successive vendor and/or NYSED, including, but not be limited to, assisting NYSED in developing and implementing a feasible transition plan in advance of the anticipated expiration, cancellation, or termination of the contract.

The vendor agrees to cooperate fully with any successive vendor and NYSED and refrain from any activity that would interfere with the implementation of the transition plan and a seamless transition. The vendor shall provide all items, reports, materials, data, and equipment owned by NYSED in the vendor’s possession, and any information useful to and requested by NYSED in developing a RFP for a successive vendor, prior to the expiration, cancellation, or termination of the contract.

The vendor will work with NYSED to determine the optimum format for providing data and information that will optimize the reuse of this data and information by any successive vendor. The vendor will also provide NYSED with a list of all computer programs and software tools necessary to allow an end user to read and export any data provided by the vendor under this contract.

Electronic Processing of Payments

In accordance with a directive dated January 22, 2010 by the Director of State Operations - Office of Taxpayer Accountability, all state agency contracts, grants, and purchase orders executed after February 28, 2010 shall contain a provision requiring that contractors and grantees accept electronic payments. Additional information and authorization forms are available at the State Comptroller’s web site at www.osc.state.ny.us/epay/index.htm external link icon.

Subcontracting

Subcontracting will be limited to forty percent (40%) of the annual contract budget. Subcontracting is defined as non-employee direct personal services and related incidental expenses, including travel. All subcontractors must be listed on the Subcontracting Form located in the Submission Documents. In addition, the cost of subcontractor payments for each deliverable must be listed under “Purchased Services” in the Bid Form – Schedule of Deliverables (Attachments A-1 and A-2).

The vendor is required to use New York State-certified Minority and Women-Owned Business Enterprise (M/WBE) for an amount greater or equal to 10% MBE and 10% WBE of the total dollar amount of the contract. These goals can be met by any combination of MBE and WBE participation. Bidders are required to comply with NYSED’s Minority and Women-Owned Business Enterprise goals by completing and submitting the forms described in Mandatory Submission Requirement #7 of this RFP. These subcontractors must be certified with the NYS Division of Minority and Women-owned Business Development before beginning any work on this contract. The submitted budget must clearly define how and when this requirement will be achieved over the life of the contract and what vendors will be providing these services. See www.nylovesmwbe.ny.gov external link icon for additional information and a listing of currently certified M/WBEs.

If the vendor proposes to change subcontractors during the contract period, NYSED must be notified prior to the change. NYSED reserves the right to reject any replacement subcontractors proposed by the vendor and reserves the right to approve all changes in subcontractors. The Subcontracting Form located in the Submission Documents must be updated annually and submitted to NYSED. Using this form, the vendor must also report to NYSED, on an annual basis, actual expenditures incurred for all subcontractors and indicate which subcontracting costs are associated with M/WBE.

M/WBE AND EQUAL EMPLOYMENT OPPORTUNITIES REQUIREMENTS CONTRACTOR REQUIREMENT AND OBLIGATION UNDER NEW YORK STATE EXECUTIVE LAW, ARTICLE 15-A (PARTICIPATION BY MINORITY GROUP MEMBERS AND WOMEN WITH RESPECT TO STATE CONTRACTS)

In an effort to eradicate barriers that have historically impeded access by minority group members and women in State contracting activities, Article 15-A, of the New York State Executive Law §310-318, (Participation By Minority Group Members and Women With Respect To State Contracts) was enacted to promote equality of economic opportunities for minority group members and women.

The New York State Education Department (“NYSED”) has enacted its policies Equal Opportunity, Non-Discrimination and Affirmative Action and on Minority and Women-Owned Business Enterprise Procurements, consistent with the requirements as set forth under the provisions of Article 15-A (the “Article”) incorporated by reference, requiring Contracting Agencies to implement procedures to ensure that the “Contractor” (as defined under Article 15-A, §310.3 shall mean an individual, a business enterprise, including a sole proprietorship, a partnership, a corporation, a not-for-profit corporation, or any other party to a state contract, or a bidder in conjunction with the award of a state contract or a proposed party to a state contract, complies with requirements to ensure Equal Employment Opportunities for Minority Group Members and Women, in addition to providing Opportunities for Minority and Women-Owned Business Enterprises on all covered state contracts.

In keeping with the intent of the Law, it is the expectation of the Commissioner and the responsibility of all contractors participating in and/or selected for procurement opportunities with NYSED, to fulfill their obligations to comply with the requirements of the Article and its implementing regulations.

In accordance with these requirements, the contractor hereby agrees to make every good faith effort to promote and assist the participation of certified Minority and Women-Owned Business Enterprises (“M/WBE”) as subcontractors and suppliers on this project for the provision of services and materials in an amount at least equal to the M/WBE goal (Included in the procurement document) as a percentage of the total dollar value of this project. In addition, the contractor shall ensure the following:

  1. All state contracts and all documents soliciting bids or proposals for state contracts contain or make reference to the following provisions:
    1. The contractor will not discriminate against employees or applicants for employment because of race, creed, color, national origin, sex, age, disability, marital status, gender, religion, veteran status, sexual orientation, genetic disposition or carrier status and will undertake or continue existing programs of affirmative action to ensure that minority group members and women are afforded equal employment opportunities without discrimination.

      For purposes of the Article, affirmative action shall mean recruitment, employment, job assignment, promotion, upgrading, demotion, transfer, layoff or termination and rate of pay or other forms of compensation.
    2. The contractor shall request each employment agency, labor union, or authorized representative of workers with which it has a collective bargaining or other agreement or understanding, to furnish a written statement that such employment agency, labor union or representative will not discriminate on the basis of race, creed, color, national origin, sex, age, disability, marital status, gender, religion, veteran status, sexual orientation, genetic disposition or carrier status and that such union or representative will affirmatively cooperate in the implementation of the contractor’s obligation herein.
    3. The contractor shall state in all solicitations or advertisements for employees, that, in the performance of the State contract, all qualified applicants will be afforded equal employment opportunities without discrimination because of race, creed, color, national origin, sex, age, disability, marital status, gender, religion, veteran status, sexual orientation, genetic disposition or carrier status.
  2. The contractor will include the provisions of subdivision one of this section in every subcontract as defined under §310.14, except as provided under §312.6 of the Article, in such a manner that the provisions will be binding upon each subcontractor as to work in connection with the State contract.
  3. Contractors or subcontractors shall comply with the requirements of any federal law concerning equal employment opportunity, which effectuates the purpose of this section.
  4. Contractors and subcontractors shall undertake programs of affirmative action and equal employment opportunity as required by this section*. In accordance with the provision of the Article, the bidder will submit, with their proposal, Staffing Plan (EEO 100).
  5. Certified businesses (as defined under Article 15-A, §310.1 means a business verified as a minority or women-owned business enterprise pursuant to §314 of the Article) shall be given the opportunity for meaningful participation in the performance of this contract, to actively and affirmatively promote and assist their participation in the performance of this contract, so as to facilitate the award of a fair share of this contract to such businesses**.
  6. Contractor shall make a good faith effort to solicit active participation by enterprises identified in the Empire State Development (“ESD”) directory of certified businesses, which can be viewed at: www.nylovesmwbe.ny.gov external link icon
  7. Contractor shall agree, as a condition of entering into said contract, to be bound by the provisions of Article 15-A, §316.
  8. Contractor shall include the provisions set forth in paragraphs (6) and (7) above, in every subcontract in a manner that the provisions will be binding upon each subcontractor as to work in connection with this contract.
  9. Contractor shall comply with the requirements of any federal law concerning opportunities for M/WBEs which effectuates the purpose of this section.
  10. Contractor shall submit M/WBE Utilization Plan*** (M/WBE 100) as part of their proposal in response to NYSED procurement.
  11. The percentage goals established for this RFP are based on the overall availability of M/WBEs certified in the particular areas of expertise identified under this RFP. These goals should not be construed as rigid and inflexible quotas which must be met, but as targets reasonably attainable by means of applying every good faith effort to make all aspects of the entire Minority and Women-Owned Business Program work.
  12. Contractor shall ensure that enterprises have been identified (M/WBE 102) within the Utilization Plan, and the contractor shall attempt, in good faith, to utilize such enterprise(s) at least to the extent indicated in the plan, as to what measures and procedures contractor intends to take to comply with the provisions of the Article.
  13. Contractor shall upon written notification from NYSED M/WBE Program Unit as to any deficiencies and required remedies thereof, the contractor, within the period of time specified, will submit compliance reports documenting remedial actions taken and other information relating to the operation and implementation of the Utilization Plan.
  14. Where it appears that a contractor cannot, after a good faith effort, comply with the M/WBE participation requirements, contractor may file a written application with NYSED M/WBE Program Unit requesting a partial or total waiver (M/WBE 101) of such requirements setting forth the reasons for such contractor’s inability to meet any or all of the participation requirements, together with an explanation of the efforts undertaken by the contractor to obtain the required M/WBE participation.

* Notice – Contractors are provided with notice herein, NYSED may require a contractor to submit proof of an equal opportunity program after the proposal opening and prior to the award of any contract. In accordance with regulations set forth under Article 15-A §312.5, contractors and/or subcontractors will be required to submit compliance reports relating to the contractor’s and/or subcontractor’s program in effect as of the date the contract is executed.
** Should the contractor identify a firm that is not currently certified as an M/WBE, it should request that the firm submit a certification application to the NYSED M/WBE Program Unit by the deadline for submission of proposals for eligibility determination. NYSED will work with ESD to expedite the application, however, it is the responsibility of the contractor to ensure that a sufficient number of certified M/WBE firms have been identified in response to this procurement, in order to facilitate full M/WBE participation.
*** A Utilization Plan, as defined under Article 15-A, shall mean a plan prepared by a contractor and submitted in connection with a proposed state contract. In developing the Utilization Plan bidders should consider the goals and established time frames needed to achieve results which could reasonably be expected by putting forth every good faith effort to achieve the overall prescribed M/WBE participation percentage (%) goals as set forth under the procurement.

For purposes of determining a contractor’s good faith efforts to comply with the requirements of this section or be entitled to a waiver, NYSED shall consider at the least the following:

  1. Whether the contractor has advertised in general circulation media, trade association publications and minority-focused and women-focused media and, in such event;
    1. Whether or not the certified M/WBEs which have been solicited by the contractor exhibited interest in submitting proposals for a particular project by attending a pre-bid conference; and
    2. Whether certified businesses solicited by the contractor responded in a timely fashion to the contractor’s solicitations for timely competitive bid quotations prior to the contracting agency’s deadline for submission of proposals.
  2. Whether there has been written notification to appropriate certified M/WBEs that appear in the Empire State Development website, found at: www.nylovesmwbe.ny.gov external link icon

All required Affirmative Action, EEO, and M/WBE forms to be submitted along with bids and/or proposals for NYSED procurements are attached hereto. These forms are to be submitted without change to goals specified in the RFP. All M/WBE firms are required to be certified by Empire State Development (ESD) or must be in the process of obtaining certification from ESD.

Failure to comply with the requirements of Article 15-A as set forth under this procurement and in conjunction with the corresponding contract, will result in the withholding of associated funds and other enforcement proceedings set forth under Article 15-A.

Back to top

Go to Table of Contents


Submission


Documents to be submitted with this proposal

This section details the submission documents to be transmitted to NYSED in response to this RFP. Except for the nationally normed test questions included in field and operational tests, NYSED shall own all materials, processes, and products (software, code, documentation and other written materials) developed under this contract. Materials prepared under this contract shall be in a form that will be ready for copyright in the name of NYSED. Any subcontractor is also bound by these terms. The submission of these documents will become the basis on which NYSED will judge the respondent’s ability to perform the services specified in the RFP.

Project Submission:

Bidders must submit one technical proposal (to include both test development options) and two separate cost proposals (one for each option). Option 1 is for the development of Grades 3–11 ELA and Grades 3–8 Mathematics Tests to include multiple-choice and constructed-response questions. Option 2 is for the development of Grades 3–8 ELA and Grades 3–8 Mathematics Tests to include multiple-choice and constructed-response questions. Each proposal submitted in response to this RFP must contain eight (8) copies of the technical proposal (one bearing an original signature in blue ink), three (3) copies of each cost proposal (one bearing an original signature in blue ink), one (1) electronic version in Microsoft Word (CD format) of both the technical and cost proposals, and three copies of each set (one set for option 1 and one set for option 2) of the M/WBE documents (one bearing an original signature in blue ink). All proposals must be received by November 15, 2010 by 3:00 PM.

The proposal must communicate an understanding of the deliverables of the RFP, describe how the tasks are to be performed and identify potential problems in the conduct of the deliverables and methods to identify and solve such problems. Bidders must specify all details and dates required to evaluate the technical proposal and should limit the aspects of the project plan which are to be determined only after the award of a contract. Optional deliverables to be provided only at an additional cost should not be detailed and will not be considered. Contractual terms, conditions, and assumptions should not be included in the proposal.

Each proposal should be prepared simply and economically, avoiding the use of promotional materials beyond those sufficient to provide a complete presentation. If supplemental materials are a necessary part of the technical proposal, the vendor should reference these materials in the technical proposal identifying the document(s) and citing the appropriate sections and page(s) to be reviewed.

Technical Proposal: 70 Points

Eight copies (one bearing an original signature in blue ink) of the Technical Proposal should be organized in a binder with tabs clearly labeling each section as detailed below, and must include the following:

  1. Response Sheet for Bids (Signature Required)
  2. All Forms and Assurances located in Submission Documents including original signatures where necessary
  3. Mandatory Requirements Certification Form (Signature Required)
  4. Attachment E -Non-Disclosure Agreement (Signature Required)
  5. Project Description as outlined below:

Bidders will not receive points for meeting any of the Mandatory Requirements related to staff/vendor experience and qualifications as outlined in the RFP. These are scored on a pass/fail basis.

  1. Staffing and Program Management (10 points)

    The proposal must identify all key staff, including the program managers, content staff, and research staff, by name and provide details about the proposed staffing plan, including an organizational diagram. For each staff member, the bidder must provide a resume and a detailed description of responsibilities, qualifications, and the number of hours per week spent exclusively on the NYSED project. The proposal must provide sufficient detail to enable reviewers to evaluate the appropriateness of the proposed staff. The proposal must describe how different staff members and divisions will interact with one another, subcontractors, NYSED staff, teacher committees, schools, and other contractors working with NYSED. The proposal must describe how program managers will work with NYSED staff, including planning and coordinating status meetings and conference calls, providing summaries and minutes of meetings, developing status reports and project plans, notifying NYSED of any potential problems or changes to dates or deliverables, and submitting all deliverables according to the agreed upon schedule with appropriate time for NYSED review and approval. The proposal must describe how the bidder will ensure that the work of the contract has the necessary priority within the organization to be completed with the highest quality and on time.
  2. Test Development (10 points)

    The proposal must describe the bidder’s experience in developing passages and both multiple-choice and constructed-response items for large-scale assessment programs and its plans and capacity to develop the passages, multiple-choice, and constructed-response items required by the RFP. The bidder must detail its plans for assembling tests according to the test design and other specifications of the RFP using items from the bidder’s existing bank, newly developed items, or a combination of both. The proposal must include developing more items than will be needed for the operational tests, as specified in the RFP, in order to build operational tests from successfully field tested items, as well as an item bank for NYSED’s use. The bidder should describe how it will develop a variety of passages and items in order to ensure that multiple genres are reflected in each ELA operational test and that the items on both the ELA and mathematics operational tests cover the widest range of performance indicators possible. The proposal must include specifics on the required teacher review committees and the operational forms construction meetings to be held with NYSED staff. The proposal must also describe the nationally normed items to be included on each test.
  3. Preparation of Test Materials (10 points)

    The proposal must describe the bidder’s experience in preparing materials for large-scale assessment programs and its plans and capacity to develop all the materials required by the RFP, including printed field test materials (test booklets, answer sheets, teacher directions, and instructions for administration) and camera-ready operational test materials (test booklets, school administrators manuals, teacher directions, listening selections, and scoring guides and materials). The proposal must also include the development of translated, braille, and large-type versions of the tests and audio scoring materials. The bidder must describe the development and revision of a style guide for the Grades 3–11 ELA and Grades 3–8 Mathematics Tests. The proposal must demonstrate that the bidder has the technical capacity and staffing levels needed to provide quality control and to produce all materials in the timeframe required.
  4. Field Testing (10 points)

    The proposal must describe the bidder’s experience in field testing large-scale assessments and its plans and capacity to conduct field testing as required by the RFP, including both stand alone and embedded field tests. The proposal must include the bidder’s plans for selecting field test samples for stand alone field tests, ensuring they are representative of NYS schools, and oversampling to ensure that the minimum number of actual student responses is obtained for each field test form. The proposal must contain the development, printing, shipping, and retrieval of all stand alone field test materials, and communication with schools in order to notify them of the field tests and work to obtain any unreturned field test materials. The proposal must include a description of how the bidder will score constructed responses, including rangefinding, developing scorer training materials, and recruiting and training appropriate scorers. The proposal must also include a description of how multiple-choice field test items will be embedded into operational test forms.
  5. Research (15 points)

    The proposal must describe the bidder’s experience in conducting research for large-scale assessments and its plans and capacity to perform the research required by the RFP. The proposal must include appropriate field test analyses, forms scaling and equating, embedding anchor forms, conducting a linking study, conducting standard setting, providing all data files to NYSED, and preparing all required technical reports. The bidder must describe in detail the proposed methods to be used and the rationale for such methods.
  6. Test Administration and Field Support (5 points)

    The proposal must describe the bidder’s experience in test administration and providing field support for large-scale testing programs and its plans and capacity to perform all services required by the RFP. The proposal must describe the development of all administration manuals, teacher’s directions, and scoring materials needed for local scoring of the tests and include plans to annually update the “Guide to the Grades 3–11 English Language Arts and Grades 3–8 Mathematics Tests.” The bidder must describe its plans for conducting turnkey training for the local scoring and for providing a scoring hotline to answer test-related questions during the local scoring period. The proposal must also describe how the bidder will work with local scanning centers (RICS and large city school district scanning centers) and SIRS to coordinate the development of answer sheets and the transfer of student test data. The proposal must include plans for receiving, scoring, and analyzing student test data and providing appropriate quality controls for all data.
  7. Security (5 points)

    The proposal must describe in detail the bidder’s facilities, infrastructure, and security protocols, including specifics about each stage of the test development and administration process. The proposal must specifically address how the bidder will maintain the security of test items prior to operational test administration, including during committee meetings and field testing, and how the bidder will maintain the security of all test and individual student data. All requirements of Attachment D must be addressed.
  8. Qualifications and References (5 points)

    The proposal must describe how the bidder meets the qualification requirements for the development of large-scale assessment programs, including test development, field testing, scoring, research, material preparation, and administration. The proposal must demonstrate that the bidder has the hardware, software, and staffing to complete all required tasks. The proposal should include a minimum of three current professional references to substantiate these qualifications. Current shall mean relevant within the past three years. NYSED staff may not be used as references. Each reference should include the name, title, organization, address, telephone number, and e-mail address. The dates of service and a brief summary of the services provided should be included for each reference. The bidder should ensure that contact information is current and accurate.

Cost Proposal: 30 Points

The original plus two copies of each Cost Proposal should be mailed in a separate envelope labeled RFP #10-021 (Option 1 or Option 2) Cost Proposal Do Not Open and should include the following:

  1. Bid Form Cost Proposal—Schedule of Deliverables – for both option 1 and option 2.
  2. Five Year Budget Summary for both option 1 and option 2 (1/1/11–12/31/15) Signature Required
  3. Subcontracting Form for both option 1 and option 2.

Bidders must provide a cost proposal reflecting the cost and timeline for each activity within each content area. The cost is to be submitted on forms Attachment A-1a and A-1b for Option 1 and A-2a and A-2b for Option 2.

The Financial Criteria portion of the RFP will be scored based upon the grand total on the five year Budget Summary Form for either Option 1 or Option 2.

M/WBE Documents

Bidders must submit three (3) copies (one bearing an original signature in blue ink) for each option-in a separate sealed envelope labeled M/WBE Documents - RFP #10-021 (Option 1 or Option 2) Do Not Open. Each set must include the following:

  1. M/WBE Cover Letter Signature Required
  2. M/WBE 100 Utilization Plan
  3. M/WBE 102 Notice of Intent to Participate
  4. EEO 100 Staffing Plan

Back to top

Go to Table of Contents


Evaluation Criteria and Method of Award

This section begins with the criteria to be used to evaluate bids, and closes with the method of award or how the contractor will be selected. This will be followed by various terms and conditions that reflect the specific needs of this project as well as State contract guidelines and requirements.

Criteria for Evaluating Bids

An evaluation committee of NYSED staff will complete a review of all technical proposals submitted that meet the requirements outlined in this RFP. The committee will review each bid based upon the submitted proposal and the requirements of the RFP only. Bidders should not assume that reviewers will be familiar with the current program or have previous experience with the bidder. Appropriate description should be included to inform reviewers about the bidder’s qualifications and capacity to perform all required deliverables. The review committee will determine compliance with the Mandatory Requirements described in the RFP. NYSED retains the right to determine whether any deviation from the requirements of the RFP is substantial in nature and may reject in whole or in part any and all proposals, waive minor irregularities, and conduct discussions with all responsible bidders.

Proposals receiving at least 60% of the points available for the technical proposal (a total of 42 out of 70 points) will move to the next step of the process. Proposals with a score of less than 42 points in the Technical Proposal section will be eliminated from further consideration.

Technical Criteria: (70 Points)

Each proposal received by the deadline will be reviewed and rated on the quality and extent to which the bidder meets the following criteria as outlined in the previous section:

  1. Staffing and Program Management (10 Points)
  2. Test Development (10 Points)
  3. Preparation of Test Materials (10 Points)
  4. Field Testing (10 Points)
  5. Research (15 Points)
  6. Test Administration and Field Support (5 Points)
  7. Security (5 Points)
  8. Qualifications and References (5 Points)

Financial Criteria: (30 Points)

The Financial Criteria portion of the RFP will be scored based upon the grand total of the five year Budget Summary. The cost proposal will be awarded points pursuant to a formula, which awards the highest score of thirty points to the proposal with the lowest overall cost. The remaining cost proposals will be awarded points based on the relative difference of each proposal against the lowest budget submitted. The resulting percentage is then applied to the maximum point value of thirty points. This calculation will be computed by the Contract Administration Unit upon completion of the technical scoring by the technical review panel.

Method of Award

All proposals which are received by the bid due date and meet the Mandatory Requirements of the RFP will be reviewed by the technical evaluation committee as described above. The cost component for Option 1 will then be calculated for all proposals which receive at least the minimum number of technical points. The total score for each proposal will represent the 70/30 percent split between technical and cost scores. The successful bidder will be that bidder whose aggregate technical and cost scores is the highest among all the proposals rated. In the event that more than one proposal obtains the highest aggregate score, the successful bidder will be the bidder in that group of highest aggregate scores whose proposal reflects the lowest cost.

Prior to making an award based on the original cost scoring, NYSED reserves the right to request that all bidders submit a Final Best Price for their proposal. If Final Best Prices are requested, these prices will be used to calculate the final cost score.

After determining the successful bidder, NYSED will evaluate the cost of the successful bidder in comparison to the maximum feasible cost determined by NYSED given the availability of funding and budget constraints. If the cost proposed by the successful bidder is equal to or below the maximum feasible cost established by NYSED, NYSED will proceed with an award to the successful bidder pursuant to Option 1, and will not consider costs submitted for Options 2. If the cost proposed by the successful bidder is above the maximum feasible cost established by NYSED, no award will be made pursuant to Option 1 and NYSED will score cost proposals submitted in response to Option 2.

When calculating the total aggregate scores for Option 2, the same technical scores will be used for Option 2 that were used for Option 1. Total aggregate scores for Option 2 will be calculated using the same method described above for Option 1. After obtaining any requested Final Best Prices and determining the successful bidder pursuant to Option 2, NYSED will evaluate the cost proposed by the successful bidder in comparison to the maximum feasible cost determined by NYSED. If the cost proposed by the successful bidder is equal to or below the maximum feasible cost established by NYSED, NYSED will proceed with an award to the successful bidder pursuant to Option 2.

NYSED reserves the right to reject all proposals and cancel the RFP if it is in NYSED’s best interest.

NYSED’s Reservation of Rights

NYSED reserves the right to: (1) reject any or all proposals received in response to the RFP; (2) withdraw the RFP at any time, at the agency’s sole discretion; (3) make an award under the RFP in whole or in part; (4) disqualify any bidder whose conduct and/or proposal fails to conform to the requirements of the RFP; (5) seek clarifications of proposals; (6) use proposal information obtained through site visits, management interviews and the state’s investigation of a bidder’s qualifications, experience, ability or financial standing, and any material or information submitted by the bidder in response to the agency’s request for clarifying information in the course of evaluation and/or selection under the RFP; (7) prior to the bid opening, amend the RFP specifications to correct errors or oversights, or to supply additional information, as it becomes available; (8) prior to the bid opening, direct bidders to submit proposal modifications addressing subsequent RFP amendments; (9) change any of the scheduled dates; (10) eliminate any mandatory, non-material specifications that cannot be complied with by all of the prospective bidders; (11) waive any requirements that are not material; (12) negotiate with the successful bidder within the scope of the RFP in the best interests of the state; (13) conduct contract negotiations with the next responsible bidder, should the agency be unsuccessful in negotiating with the selected bidder; (14) utilize any and all ideas submitted in the proposals received; (15) unless otherwise specified in the solicitation, every offer is firm and not revocable for a period of 120 days from the bid opening; (16) require clarification at any time during the procurement process and/or require correction of arithmetic or other apparent errors for the purpose of assuring a full and complete understanding of an bidder’s proposal and/or to determine an bidder’s compliance with the requirements of the solicitation; (17) to request best and final offers.

Post Selection Procedures

Upon selection, the successful bidder will receive a proposed contract from NYSED. All terms set forth in the selected bidder's technical proposal will be final. The selected bidder may be given an opportunity to reduce its cost proposal in accordance with the agency's right to request best and final offers. The contents of this RFP, any subsequent correspondence during the proposal evaluation period, and such other stipulations as agreed upon may be made a part of the final contract prepared by NYSED. Successful bidders may be subject to audit and should ensure that adequate controls are in place to document the allowable activities and expenditure of State funds.

Debriefing Procedures

All unsuccessful bidders may request a debriefing within five (5) business days of receiving notice from NYSED. Bidders may request a debriefing letter on the selection process regarding this RFP by submitting a written request to the Fiscal Contact person at:

NYS Education Department
Contract Administration Unit
89 Washington Avenue
Room 505W EB
Albany, NY 12234

The Fiscal Contact person will make arrangements with program staff to provide a written summary of the proposal’s strengths and weaknesses, as well as recommendations for improvement. Within ten (10) business days, the program staff will issue a written debriefing letter to the bidder.

Contract Award Protest Procedures

Bidders who receive a notice of non-award may protest the NYSED award decision subject to the following:

  1. The protest must be in writing and must contain specific factual and/or legal allegations setting forth the basis on which the protesting party challenges the contract award by NYSED.
  2. The protest must be filed within ten (10) business days of receipt of a debriefing letter. The protest letter must be filed with:

    NYS Education Department
    Contract Administration Unit
    89 Washington Avenue
    Room 505W EB
    Albany, NY 12234

  3. The NYSED Contract Administration Unit (CAU) will convene a review team that will include at least one staff member from each of NYSED’s Office of Counsel, CAU, and the Program Office. The review team will review and consider the merits of the protest and will decide whether the protest is approved or denied. Counsel’s Office will provide the bidder with written notification of the review team’s decision within seven (7) business days of the receipt of the protest. The original protest and decision will be filed with OSC when the contract procurement record is submitted for approval and CAU will advise OSC that a protest was filed.
  4. The NYSED Contract Administration Unit (CAU) may summarily deny a protest that fails to contain specific factual or legal allegations, or where the protest only raises issues of law that have already been decided by the courts.

Vendor Responsibility

State law requires that the award of state contracts be made to responsible vendors. Before an award is made to a not-for-profit entity, a for-profit entity, a private college or university or a public entity not exempted by the Office of the State Comptroller, NYSED must make an affirmative responsibility determination. The factors to be considered include: legal authority to do business in New York State; integrity; capacity- both organizational and financial; and previous performance. Before an award of $100,000 or greater can be made to a covered entity, the entity will be required to complete and submit a Vendor Responsibility Questionnaire. School districts, charter schools, BOCES, public colleges and universities, public libraries, and the Research Foundation for SUNY and CUNY are some of the exempt entities. For a complete list, see: http://www.osc.state.ny.us/vendrep/documents/vrdocrules.pdf external link icon.

Vendors are invited to file the required Vendor Responsibility Questionnaire online via the New York State VendRep System. To enroll in and use the New York State VendRep System, see the VendRep System instructions at: http://www.osc.state.ny.us/vendrep/vendor_index.htm external link icon or go directly to the VendRep System online at https://portal.osc.state.ny.us/wps/portal external link icon.

For direct VendRep System user assistance, the OSC Help Desk may be reached at 866-370-4672 or 518-408-4672 or by email at helpdesk@osc.state.ny.us.

Vendors opting to file a paper questionnaire can obtain the appropriate questionnaire from the VendRep web site: http://www.osc.state.ny.us/vendrep/forms_vendor.htm external link icon or will receive it with the award letter.

Note: Bidders must acknowledge whether they filed their questionnaire online by checking the appropriate box on the Response Sheet for Bids in Submission Documents).

Procurement Lobbying Law

Pursuant to State Finance Law §§139-j and 139-k, this solicitation includes and imposes certain restrictions on communications between NYSED and an Offerer/bidder during the procurement process. An Offerer/bidder is restricted from making contacts from the earliest notice of the solicitation through final award and approval of the Procurement Contract by NYSED and, if applicable, Office of the State Comptroller (“restricted period”) to other than designated staff unless it is a contact that is included among certain statutory exceptions set forth in State Finance Law §139-j(3)(a). Designated staff, as of the date hereof, is identified below. NYSED employees are also required to obtain certain information when contacted during the restricted period and make a determination of the responsibility of the Offerer/bidder pursuant to these two statutes. Certain findings of non-responsibility can result in rejection for contract award and in the event of two findings within a four year period, the Offerer/bidder is debarred from obtaining governmental Procurement Contracts. Further information about these requirements can be found at the following web link: http://www.oms.nysed.gov/fiscal/cau/PLL/procurementpolicy.htm.

Designated Contacts for NYSED
Program Office – (Jessica Hartjen)
Contract Administration Unit – (Gary Yorkshire)

Consultant Disclosure Legislation

Effective June 19, 2006, new reporting requirements became effective for State contractors, as the result of an amendment to State Finance Law §§ 8 and 163.  As a result of these changes in law, State contractors will be required to disclose, by employment category, the number of persons employed to provide services under a contract for consulting services, the number of hours worked and the amount paid to the contractor by the State as compensation for work performed by these employees. This will include information on any persons working under any subcontracts with the State contractor.

Chapter 10 of the Laws of 2006 expands the definition of contracts for consulting services to include any contract entered into by a State agency for analysis, evaluation, research, training, data processing, computer programming, engineering, environmental, health, and mental health services, accounting, auditing, paralegal, legal, or similar services.

To enable compliance with the law, State agencies must include in the Procurement Record submitted to OSC for new consultant contracts, the State Consultant Services Contractor’s Planned Employment From Contract Start Date Through the End of the Contract Term (Form A external link icon - see link below). The completed form must include information for all employees providing service under the contract whether employed by the contractor or a subcontractor. Please note that the form captures the necessary planned employment information prospectively from the start date of the contract through the end of the contract term.

Form A: http://www.osc.state.ny.us/agencies/gbull/g226forma.doc external link icon

Chapter 10 of the Laws of 2006 mandates that State agencies must now require State contractors to report annually on the employment information described above, including work performed by subcontractors. The legislation mandates that the annual employment reports are to be submitted by the contractor to the contracting agency, to OSC and to the Department of Civil Service.  State Consultant Services Contractor’s Annual Employment Report (Form B external link icon - see link below) is to be used to report the information for all procurement contracts above $15,000.  Please note that, in contrast to the information to be included on Form A, which is a one-time report of planned employment data for the entire term of a consulting contract on a projected basis, Form B will be submitted each year the contract is in effect and will capture historical information, detailing actual employment data for the most recently concluded State fiscal year (April 1 – March 31).

Form B: http://www.osc.state.ny.us/agencies/gbull/g226formb.doc external link icon

For more information, please visit the OSC web site for G-Bulletin 226 at: http://www.osc.state.ny.us/agencies/gbull/g-226.htm external link icon.

Public Officer’s Law Section 73

All bidders must comply with Public Officer’s Law Section 73 (4)(a), as follows:

4. (a) No statewide elected official, state officer or employee, member of the legislature, legislative employee or political party chairman or firm or association of which such person is a member, or corporation, ten per centum or more of the stock of which is owned or controlled directly or indirectly by such person, shall (i) sell any goods or services having a value in excess of twenty-five dollars to any state agency, or (ii) contract for or provide such goods or services with or to any private entity where the power to contract, appoint or retain on behalf of such private entity is exercised, directly or indirectly, by a state agency or officer thereof, unless such goods or services are provided pursuant to an award or contract let after public notice and competitive bidding. This paragraph shall not apply to the publication of resolutions, advertisements or other legal propositions or notices in newspapers designated pursuant to law for such purpose and for which the rates are fixed pursuant to law.

(i) The term "state officer or employee" shall mean:
(i) heads of state departments and their deputies and assistants other than members of the board of regents of the university of the state of New York who receive no compensation or are compensated on a per diem basis;
(ii) officers and employees of statewide elected officials;
(iii) officers and employees of state departments, boards, bureaus, divisions, commissions, councils or other state agencies other than officers of such boards, commissions or councils who receive no compensation or are compensated on a per diem basis; and
(iv) members or directors of public authorities, other than multistate authorities, public benefit corporations and commissions at least one of whose members is appointed by the governor, who receive compensation other than on a per diem basis, and employees of such authorities, corporations and commissions.

Public Officer’s Law Section 73 can be found at http://www.nyintegrity.org/law/ethc/POL73.html external link icon.

Back to top

Go to Table of Contents


Assurances

Appendix A – Standard Clause for all New York State Contracts will be included in any contract that results from this RFP.  Vendors who are unable to complete or abide by these assurances should not respond to this request.

The documents listed below are included in Submission Documents, which must be signed by the Chief Administrative Officer. Please review the terms and conditions. Certain documents will become part of the resulting contract that will be executed between the successful bidder and the NYS Education Department.

Appendix A – Standard Clause for all New York State Contracts contains the following three (3) certifications

  • Non-Collusion Certification
    (Signature Required - the form is included in Submission Documents)
  • MacBride Certification
    (Signature Required - the form is included in Submission Documents)
  • Certification-Omnibus Procurement Act of 1992
    (Signature Required - the form is included in Submission Documents)

Appendix A-1 – NYSED Contract Clauses

Certification Regarding Lobbying; Debarment and Suspension; and Drug-Free Workplace Requirements (Signature Required - the form is included in Submission Documents)

Offerer Disclosure of Prior Non-Responsibility Determinations (Signature Required - the form is included in Submission Documents)

M/WBE Documents – the forms below are included in Submission Documents) and can also be found at: www.oms.nysed.gov/fiscal/MWBE/forms.html.

  1. M/WBE Cover Letter (Signature Required)
  2. M/WBE 100 Utilization Plan
  3. M/WBE 102 Notice of Intent to Participate
  4. EEO 100 Staffing Plan

Appendix A - Standard Clauses for NYS Contracts

Appendix A-1

Back to top

Go to Table of Contents


Submission Documents

Back to top

Go to Table of Contents


Attachments


Attachment A-1a: Schedule of Deliverables - English Language Arts Option 1

Attachment A-1b: Schedule of Deliverables – Mathematics Option 1

Attachment A-2a: Schedule of Deliverables - English Language Arts Option 2

Attachment A-2b: Schedule of Deliverables – Mathematics Option 2

Minority- and Women-Owned Business: Required Submission Documents

Attachment B: Specifications for Large Type

Attachment C: Specifications for Contractor/Subcontractor Transcribing Braille

Attachment D: Security Guidelines for the New York State Assessment Program

Attachment E: Non-Disclosure Agreement

Attachment F: Teacher Committees

Attachment G: Embedded Field Test Design

Attachment H: Field Test Schedule

Attachment I: Technical Manual Outline

Back to top

Go to Table of Contents

Last Updated: October 27, 2010