Office of Accountability

DTSDE Frequently Asked Questions

FAQ Printable Version pdf(377KB) DTSDE

 

Topics:

Frequently Used Terms

A. Overview of the School and District Review Process

B. The DTSDE Process

C. Models and Exemplars

D. Outside Educational Experts (OEE)

E. Regional Bilingual Education Resource Networks (RBE-RNs)

F. Special Education School Improvement Specialist (SESIS)                                              

G. Surveys

H. Report Writing

I. School/District Rating

J. Timeline

K. English Language Learners (ELLs)

L. DTSDE Rubric  

M. Calibration Assurance Team (CAT)

N. DTSDE Handbook

O. Schedules

P. Interfacing with other Review Protocols

Q. District-led Review Using DTSDE Protocols and School Reviews (Focused on DTSDE Tenet 3) with District Oversight Roles and Responsibilities

R. District Comprehensive Improvement Plans (DCIP)

S. Training

T, Other

 

Frequently Used Terms

SOP:  Statement of Practice
CAT: Calibration Assurance Team
IIT:  Integrated Intervention Team
OEE: Outside Educational Expert
HEDI: Highly, Effective, Developing, Ineffective
DTSDE: Diagnostic Tool for School and District Effectiveness
DCIP:  District Comprehensive Improvement Plans
SCEP:  School Comprehensive Education Plan
SED: New York State Education Department
NYSED: New York State Education Department

Back to the Top

A. Overview of the School and District Review Process

A.1. What types of reviews are conducted in Focus Districts?

There are three types of reviews conducted in Focus Districts:

On-site review led by the New York State Education Department (NYSED): An on-site review led by NYSED will be carried out by an Integrated Intervention Team (IIT) composed of NYSED reviewers and/or consultant reviewers over a period of one, two or three days. The length of the review will depend on the accountability identification status or the size of the school.  More information about on-site school reviews can be found at: http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/DTSDEHandbook.html

District led-review using the DTSDE protocol:  A District led review using the Diagnostic Tool for School and District Effectiveness (DTSDE) protocol consists of a district assigned lead reviewer as well as a team of individuals who will be responsible for conducting the school review.  The district lead reviewer is expected to be a person who has participated in the NYSED mandated DTSDE training or has otherwise been certified by NYSED.  Other district reviewers, such as content specialists, may participate after receiving appropriate turnkey district training.

School review (focused on DTSDE Tenet 3) with district oversight:   This school review with district oversight is focused on the creation and enactment of curriculum as defined in Tenet 3 of the DTSDE.  The school principal, in collaboration with any school based management-shared decision entity authorized by the district pursuant to Part 100.11 of Commissioner’s Regulations to participate in developing a school comprehensive education plan, completes the DTSDE School Self-Assessment document, which can be found on the SED website at  http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/DTSDESchoolReviewwithDistrictOversight.html.  The instructional leader and others with instructional expertise conduct classroom visits using the appropriate DTSDE Classroom Visitation Tool (there are targeted tools for different sub-groups of students), which can be found on the NYSED website at http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/DTSDEClassroomVisitationTool.docx.  The instructional leader and designated staff also use the Tools to Guide the Collection of Evidence of Shifts in Practice, which can be found under Downloadable Resources at http://engageny.org/resource/tools-to-guide-the-collection-of-evidence-of-shifts-in-practice and  the Tristate Rubric, which can be found at http://engageny.org/resource/tri-state-quality-review-rubric-and-rating-process/ to determine the school’s progress in alignment of its curriculum to the Common Core Learning Standards (CCLS) and the implementation of instructional practices that reflect the six Common Core Shifts.  The school then summarizes the results of its review against the Tenet 3 rubric and the implications for development of its next School Comprehensive Education Plan (SCEP). The district will complete a similar summary in terms of the District Comprehensive Improvement Plan (DCIP).  This option may be of particular interest for schools that received a School Quality Review (SQR), External School Curriculum Audit (ESCA), or Joint Intervention Team (JIT) review in 2011-12.

Districts that already have in place an internal school review process that addresses at least three of the five tenets of the DTSDE and can be used to determine the school’s progress in alignment of its curriculum with the Common Core Learning Standards (CCLS) and the implementation of instructional practices that reflect the six Common Core Shifts, with Department approval may use that process as the basis for conduct of the School Review (Focused on DTSDE Tenet 3) with District Oversight. 

A.2. Will districts with only one Focus School be visited by a NYSED-led IIT?

Yes.  An NYSED IIT will conduct a review of the district and the one school.

A.3. Do all Focus Districts receive funding for implementation? 

All Focus Districts that receive Title I funds are eligible 1003(a) School Improvement Grants (SIG).  Please see http://www.p12.nysed.gov/funding/2012-13-1003a/home.html.

A.4. Who leads the IIT review, the District-led Review using the DTSDE Protocol, and the School Review (Focused on DTSDE Tenet 3) with District Oversight?

The NYSED Lead Reviewer and the Outside Educational Expert are co-leaders for the State-led IIT review. For the district-led review, the lead is whomever the district assigns as its District Lead Reviewer. (Please note that the District Lead Reviewer must have participated in the SED mandated DTSDE training or be otherwise certified as a District Lead Reviewer.) A district may use an OEE to lead its district-led reviews. When the district is identified for the students with disabilities subgroup and a Special Education School Improvement Specialist (SESIS) is a member of the team, the SESIS may not be assigned the responsibility of leading the district-led review.

School self-reviews are led by the principal in collaboration with any such school leadership team and/or school based management/shared decision-making team as may have been established pursuant to Part 100.11 of Commissioner’s Regulations and tasked with participating in the development of the SCEP.  District oversight is provided by a person selected by the district who has been trained in the DTSDE protocol.

A.5. Can you define each of the team members' roles on the IIT?

The IIT will be comprised of an SED lead reviewer, Outside Educational Expert (OEE), district representative, and possibly a Regional Bilingual Education Resource Network (RBE-RN) staff member and/or a Regional Special Education Technical Assistance Support Center (RSE-TASC) staff member depending on the reason for the district's identification as a Focus District and the performance of the school's English language learner (ELL) and student with disabilities subgroups. If students with disabilities are one of the accountability subgroups, a Special Education School Improvement Specialist (SESIS) from the RSE-TASC will participate on IIT as a subgroup specialist.  See Question and Answer 46 for additional information on the role of the SESIS.

The SED reviewer will lead the events and interviews of the school and/or district review that will culminate in a School and/or District Final Report. The OEE works as a co-lead alongside the SED lead in the school reviews and will conduct interviews, focus group meetings, and classroom visitations and participate in the various school visit events. OEEs will not be involved in the district component of the review unless there is a written request from the district to the Office of Accountability that describes the rationale for the OEEs involvement in the district review. 

A.6. What is the number of recommended district staff who should participate on the IIT?

One or two district staff should participate on the IIT.

A.7. Who are required participants for each IIT school and district review team?

For the reviews led by the State's IIT:

The SED lead, the district representative(s) and the OEE are required to participate in the school review.  The RBE-RNs and RSE-TASC staff members will also be members of the IIT depending on the reason for the identification of the district.

The SED lead and SED reviewer(s), are required to participate in the district component of the IIT review.  If requested by the district, the OEE may also participate in the District review. If students with disabilities are one of the accountability subgroups, a RSE-TASC staff member is required to participate as a subgroup specialist.   If the school has high percentages or high numbers of English language learners who are not proficient in ELA and/or mathematics, staff from the RBE-RN will participate as a subgroup specialist.

RBE-RN staff are expected to participate in visits to schools in Focus Districts that have been identified for the performance of English language learners or as AMO districts and RSE-TASC staff if the district has been identified as a Focus District because of the performance of students with disabilities.

A.8. Can building level administrators (principals) be on the IIT or District-led review team?

Principals may not be the district representative on the IIT or District-led review team that will review their school.  They may participate on an IIT or District-led review team that reviews another school in the district.

A.9. Which Schools in Focus Districts will be visited in the 2012-13 school year?

The IIT will visit a select group of Priority and Focus Schools. In general, the IIT will visit:

  • those Priority Schools that the district has designated to implement a whole school reform model in the 2013-14 school year;

  • one Focus School at the elementary or middle level if there is an identified elementary or middle school in the district;

  • one Focus School at the high school level if there is an identified high school in the district; and

The maximum number of Focus Schools that will be visited in any district is two (an elementary or middle school and a high school).  In districts that have both elementary Focus Schools and a junior-senior high school designated as a Focus School, SED will generally only visit the junior-senior high school. 

Districts that would prefer that some of their Priority Schools that are designated to implement a whole school reform model in the 2013-14 school year and/or districts that are scheduled for two Focus School visits, but would prefer to have one of these schools have a District Led Review using the DTSDE protocol or a School Review (focused on DTSDE Tenet 3) with district oversight in the 2012-13 school year should contact SED to discuss this option.

A.10. Are all Focus Districts visited each year?  

Focus Districts will be visited each year that they remain so designated, except that in instances where all schools in a district made the required progress toward removal from Priority or Focus Designation, the District may request to conduct a District-led review using the DTSDE protocols or a School Review (focused on DTSDE Tenet 3) with District Oversight in lieu of a visit from an IIT.

A.11. How is the new diagnostic tool different from SQR?

The School Quality Review (SQR) was a support intervention for a school newly identified for the Improvement phase of the State accountability continuum.  The Quality Indicators (QI) document was used in Improvement Basic reviews for schools to self-assess their effectiveness in terms of various research-based criteria that are essential for the operation of successful schools. Improvement Focused and Comprehensive reviews consisted of two to three-day on-site reviews by a team of reviewers who formulated a collective perspective of the school’s teaching and learning and operations based on the evidence obtained from the use of the QI document, interviews, class reviews and documentation.  The findings and recommendations noted in the report were intended to guide the school’s planning and development of the Comprehensive Educational Plan (CEP) for the following school year.

The DTSDE, part of New York State’s ESEA waiver, focuses on a school and district review process using a single diagnostic tool. The six tenets of the DTSDE, i.e., District Leadership and Capacity; School Leadership Practices and Decisions; Curriculum Development and Support; Teacher Practices and Decisions; Student Social and Emotional Developmental Health; Family and Community Engagement, reflect the core concepts of an effective school and are aligned with the Federal Principles for School Turnaround.  The components of the review consist of pre-review of documentation, student and teacher surveys, interviews, focus groups, teacher meetings and classroom visitations using the DTSDE and evidence to rate each statement of practice according to the HEDI scoring framework.  This leads to an overall rating for each tenet that is Highly Effective, Effective, Developing or Ineffective. Recommendations are provided for each of the six tenets. The results of the district and school reviews will inform the development of DCIPs and SCEPs. 

A significant difference between the SQR and DTSDE protocols is that SQR was used for schools in the Improvement phase only, while schools in the corrective action and restructuring phases were subject to the ESCA and JIT protocols respectively.  By contrast the DTSDE is used with both Focus and Priority Schools. In addition, the DTSDE contains a district review component and a survey requirement that were not part of the /SQR protocol.  Finally, the DTSDE report provides specific ratings on a continuum from highly effective to ineffective for each school and its district in the six tenets, while SQR reports provided only a series of bulleted findings and recommendations for each Quality Indicator.

A.12. Will the district be held accountable for recommendations contained in past reviews (e.g., JIT, SIG, QIP)?

Yes, the DCIP and any SCEP and Quality Improvement Plan (QIP) developed by the district for the 2012-13 school year should address the findings of prior SQR, ESCA, JIT, and SIG visits as well as any Title III Improvement Plan or Corrective Action Plan or IDEA QIP. Once the DTSDE review takes place, if there is a new set of strengths and areas for improvement uncovered as part of the review, the district and school should take the opportunity to reflect on prior activities that are already underway and discern how these activities/practices can address any new recommendations.  The district and school are expected to modify the DCIP and SCEP to improve practices based on the findings of the DTSDE.

A.13. What is an IIT? Are schools visited by the IIT the only ones required to do a survey and have OEEs?

The IIT is an acronym for the Integrated Intervention Team that conducts the State led reviews of selected Priority and Focus Schools. The team is comprised of the NYSED reviewers, district reviewers, an OEE, as well as RBE-RNs (when the district is identified for ELLs) and RSE TASC staff (when the district is identified for students with disabilities).

All of the aforementioned team members, except for the OEE and district reviewers, will also participate in the District component of the review. The OEE will participate in the District component of the review upon written request by the district to the Office of Accountability.  When a district is identified for students with disabilities, the RSE-TASC coordinator or designee will participate in the district (tenet 1) review, as capacity and scheduling allows. 

Please refer to the survey section of this FAQ for further guidance on the survey administration process and to the District-led review visits section for further guidance on the use of OEEs.

Back to the Top

B. The DTSDE Process

B.1. What does "explicit theory of action" mean?  Does it refer to a district implementing a research based management practice (e.g., Marzano Bolman & Deal)?

A theory of action refers to the district having an action plan for school improvement to accelerate student learning that is founded on a theory (hypothesis or well thought-out idea) derived from the analysis of school data and linked to research on effective practices whenever possible. The theory provides the basis for the logic statement "If we make X changes, then we will see Y results."  What makes the theory "explicit" is the district's ability to fully and clearly communicate the theory so that it is widely understood.

B.2. To whom does the term "school community" refer?

The school community are those stakeholders that a particular school has defined as its members, i.e., the principal, teachers, other school staff, students, parents and caregivers and, if applicable, any community based partners who participate in student and/family life at the school.

B.3. How is each statement of practice explicitly linked to improved student outcomes?

The Mental Model for the DTSDE process clearly and explicitly maps out the theory of action behind the DTSDE process. When both districts and schools embrace a vision, build capacity, and establish routines grounded in the statements of practice, then schools can deliver consistently effective student learning experiences and achieve superior learning outcomes. Please refer to Ron Ferguson's presentation of the Mental Model at http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/DTSDEPresentations.html to view the full explanation of how the DTSDE process is linked to student outcomes.

B.4. Why are the NYS social emotional guidelines or the NYS Mental Health standards not included?

The NYS social/emotional development and learning guidelines, which are referred to in the crosswalk document as “SEDL.” were used to develop Tenet 5.

B.5. If schools have already completed SCEPs, why do the schools need to complete the school self-assessment?

The SCEP is the plan that the school has developed for the 2012-13 school year. The School Self-Assessment is the document that the school principal and/or school leadership team will complete in advance of the IIT visit. The self- assessment was created so that the school leader can present an accurate, real-time picture of the school that will allow the reviewer to see evidence and artifacts of work that the school has been doing to create an effective and coherent education experience for the school community.  The school leader will also provide a rating on how the school staff perceives themselves to be performing in each of the statements of practice.

B.6. Where can a copy of the district information sheet be found?

The district information sheet is embedded in the DTSDE district self-assessment document and is the form that the district will complete detailing the demographics, overall accountability status, and personnel data for the district. This form should be completed prior to the district visit and submitted to the IIT for review. The district information sheet will also be included in the district final report. The district will have an opportunity to review the district information sheet and final report before the information is published on the NYSED website. Districts should be sure to include in the district information sheet information on students who have been placed out of the district.

B.7. On the School Information Sheet found both in the DTSDE Self-Assessment and the School Final Report, is the data to be presented from the current year or previous year when the school was identified? 

The School Information Sheet should reflect the most current available, verified data.

B.8. How should the curriculum be submitted and should it be submitted for all content areas? 

If the school curriculum is not saved digitally, it can be submitted to the IIT lead reviewer and logistics team liaison assigned to the review as a hard copy in advance of the visit or shared with the IIT during the in-school document review.  Yes, curriculum should be submitted in all content areas.

B.9. How will findings be used to improve teacher preparation/effectiveness throughout the State?

The results of the DTSDE will be used to inform professional development that SED provides to school districts.  The results of the DTSDE may also inform policy development by the Board of Regents and how SED structures grant opportunities for school districts relating to teacher and leader effectiveness.

B.10. How much time will teachers need to be out of the classroom during the DTSDE visit?

Teachers will participate in:
  • The vertical teacher focus group interview;
  • The grade level observation meeting (in large high schools these are likely to be subject area meetings rather than grade level meetings); and
  • If teachers are a part of the Student Support Team at the school, then they will also participate in that focus group meeting as well.

All of the aforementioned events range from 45 minutes to an hour, depending on how they are scheduled.

B.11. How does this rubric align with the Danielson rubric?

Danielson's rubric provides a framework on teaching and learning along four domains of planning and preparation, classroom environment, instruction, and professional responsibilities.  While a crosswalk does not currently exist between the DTSDE rubric and the Danielson framework, there is significant alignment between Danielson's four domains and the statements of practice within Tenets 3, 4 and 5.

B.12. Are there specific questions asked of teachers, school leaders and students during the IIT review? Do you have samples?

Yes, there are suggested questions for all of the interviews that are to take place during the school review that are targeted to a statement of practice for the convenience of the reviewer. Those sample questions can be found in the DTSDE interview templates that are posted on-line at: http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/DTSDEHandbook.html.

B.13. What are "virtuous feedback loops"? How are they measured?

Virtuous feedback loops are when the communication between school leader and school staff is reciprocal, honest, free-flowing, and free of bias and ill-intent.  Virtuous feedback loops are purposely cultivated through actions and are measured as a result of the information that is gleaned during the various interviews and focus group discussions that take place as part of the school review.

B.14. Will there be time available for a teacher who has been observed to meet with the IIT review team?

If the reviewer and teacher schedule allows it, the reviewer can have a conversation with the teacher about the lesson that was observed.

B.15. When will the SED representatives be assigned?

The SED review leads are already in place and assigned. The District and School Review Team has two distinct units explicitly focused on the DTSDE review process, and they are divided into downstate and upstate review teams that will focus on conducting reviews in the districts in the geographic region to which the units have been assigned.

B.16. When are the District-led review and the SED-led reviews conducted? Are they done together or is one done before the other?

The preliminary schedule of the SED-led reviews has been provided to districts.  During the SED-led reviews, SED will usually conduct the district review component and then the school reviews will follow. There may also be times when the district component and the school reviews will be conducted concurrently or when the district review will follow the school reviews.

The District-led reviews of Focus and Priority Schools should be scheduled by the district before the end of the school calendar year.  Districts should schedule the reviews in sufficient time for the reviews to inform development of the district's DCIP and the school's SCEP for the 2013-14 school year.  Districts should communicate with all required team members, including RBE-RN and SESIS, as applicable, regarding scheduling, in order to ensure the availability of the required team members.

B.17. What about the union leadership? What is the protocol if they ask to speak directly to IIT reviewers?

If union leaders are part of any of the interviews that are scheduled to be conducted by the IIT during the school review, then the review team will have an opportunity to interact with them.

The team priority is to complete all of the events that will lead to a final report for the school.  Every effort will be made by the lead reviewer to communicate with union leadership, if requested.

B.18. What role does the Board of Education have in the process?

While the Board of Education is not formally interviewed as part of the DTSDE process, the Board of Education may need to approve the implementation of components of the DTSDE review, such as the survey administration, particularly the selection of a survey vendor, and the selection of OEEs. The Board of Education is expected to review with district staff the findings of the DTSDE and actively guide the development of the DCIP and the SCEPs that will be created to address the findings of the DTSDE.

B.19. Please clarify what information regarding interviews conducted by the IIT will be shared with the principal?

At the end of the school visit, the only information that the lead reviewer will share with the principal are the five preliminary overarching statements aligned to Tenets 2, 3, 4, 5, and 6. All findings will be quality assured before ratings and a school final report is shared and made public.  At the time of the school visit, the school leader will be left with a debriefing report for the school.  No individual interview or anecdotal notes from meetings with teachers, parents, or students will be shared with the principal.

B.20. Can the School Summary Report form be amended to include subsets (a, b and c) to maintain response clarity?

At this time, no further qualitative changes are being made to the DTSDE templates. Reviewers are encouraged in their school summary submissions to list their findings, evidence, and impact by sub-statement of practice.  

B.21. Will the HEDI ratings of a school/district be made public?

Yes, the final school and district reports conducted by an IIT, which contain the HEDI determination for each tenet, will be made public.

Back to the Top

C. Models and Exemplars

C.1. Are we going to see any working model of the process during training?

During future institutes, participants will simulate actual reviews and events to provide reviewers with an understanding of a working model of the process. 

C.2. When creating the model, did NYSED look at exemplary schools to come up with the process?

Several research based documents and resources were consulted to develop the DTSDE process. Please refer to the crosswalk for a more detailed outline of the sources used.

Back to the Top

D. Outside Educational Experts (OEE)

D.1. What is the role of the OEE?

The OEE serves as a co-lead on the NYSED- led IIT for the school review and participates in DTSDE trainings. During the school review process, as requested by the district, the OEE will provide technical assistance to the school regarding the pre-review document submission to the SED and will review all of the pre-review documents that the school submits. The OEE will conduct interviews, focus group meetings, observations and classroom visitations. The OEE will share daily notes and summarize findings with the IIT and be part of the debriefing meeting at the end of the school review. The OEE will work alongside the SED co-lead to create a final report. The OEE will review the IIT’s documentation and master school summary report and will review the first draft of the final report created by the SED co-lead prior to the submission to the CAT liaison. The OEE will collaborate with the SED lead reviewer if there is a need to produce a second draft. Ultimately the OEE and SED are working together as co-leads to produce a draft final report to provide to the CAT liaison assigned to the review.

Note: If there is disagreement between SED and OEE co-leads regarding the 1st draft or 2nd draft of the final report, the SED co-lead will e-mail the reason, evidence and Statement of Practice leading to the disagreement to the CAT liaison assigned to the review with a copy to the OEE. The CAT liaison will make the final determination and advise of next steps. 

D.2. Is the SED lead reviewer in charge of the whole process and does this person write the review? How does the OEE interact with the lead reviewer?

Please review the Principal's Guide to Understanding the 2013-2014 DTSDE Review webinar conducted on October 24, 2013.

D.3. Does the same OEE write both the district and school reports?

The OEE is not involved in the district review component and will not be involved in generating a district final report, unless there is a written request to the Office of Accountability made by the district that the OEE be part of the district review component. If an OEE participates in the district review component, the OEE will serve in a role similar to that of his or her role on the school review.

The OEE who conducts the school review as part of the IIT will collaborate with the SED co-lead to produce a school final report.

D.4. When and how will OEEs be notified that they have been approved?

NYSED has notified districts if the OEEs they recommended have been approved to participate in DTSDE reviews.  Districts are responsible for informing OEEs of the determination made by NYSED.

D.5. We are a Focus District with one Focus School. Are we required to have an OEE? 

Yes.  The OEE will serve on the NYSED-led IIT.

D.6. It sounds like many districts have had a process with a QA team. We are a "1st time" district. OEEs are not easily uncovered.  Will SED provide an OEE?

Upon request, SED will provide a list of the OEEs who have been approved to be members of IITs for the 2012-13 school year.

D.7. What if an OEE cannot make a training session due to illness or family commitment?

If an OEE cannot make a training session due to illness or a family commitment, SED should be informed.

If an OEE missed both the November and December training session then the OEE must attend in January or shadow a lead reviewer.  After the lead reviewer certifies that the OEE is prepared to join the review process, the OEE will be assigned to reviews.

Back to the Top

E. Regional Bilingual Education Resource Networks (RBE-RNs)

E.1. Will the RBE- RN staff member be assigned to Review teams per their regions or potentially outside of their regions for districts identified with ELLs?

The RBE-RN staff will be assigned by RBE-RN Region.  Only if additional support in a region is requested will an RBE-RN from another region be asked to be a member of a team visiting a school outside of the RBE-RN's region.

E.2. If a district was identified for the students with disabilities subgroup only but did not make AMAO does an RBE-RN staff member have to participate on the District Review team?

A RBE-RN is expected to participate in reviews of districts that either have been identified as Focus Districts because of the performance of ELLs or because the district has been identified as a Title III AMAO district.  If a district has failed only for one year to make AMAO and was not identified as a Focus District because of the performance of ELLs then a RBE-RN staff member is not expected to be a member of the team, but should be available to address questions from the IIT.

E.3. Are RSE-TASCs and RBE-RNs part of the district-led review process?

RSE-TASC will participate in all SED-led reviews for districts identified for the subgroup of students with disabilities.  RSE-TASC will participate in district-led reviews of schools for districts identified for the subgroup of students with disabilities as resources permit as determined by the RSE-TASC Coordinator.  

RBE-RN staff be a part of the district-led school review if the school has high percentages or high numbers of students with disabilities or English language learners who are not proficient.  In particular, in districts that have been identified for the performance of the ELL or students with disabilities subgroups, the RBE-RN coordinator or designee will participate in the district (tenet 1) review, as capacity and scheduling allows.

Back to the Top

F. Special Education School Improvement Specialist (SESIS)   

F.1. What is the role of the SESIS member on the IIT?

The SESIS will ensure that appropriate tenet review focus is directed to the subgroup of students with disabilities.  Prior to the scheduled review, the SESIS will conduct an instructional walkthrough; review and debrief the walkthrough summary data with the building principal; and provide the walkthrough summary to the district or school administrator for submission as part of the documentation included in the Pre-Review Document Review.  During the on-site review, the SESIS will conduct interviews, participate in focus group meetings and classroom visitations and participate in the various school visit events.  Overall, the role of the SESIS is to bring focus on system and instructional issues that have an impact on students with disabilities.

F.2. In the school review timeline, when will SESIS walkthrough data be completed?

The RSE-TASC instructional walkthrough review needs to be completed and summary results submitted to the district or school administrator as part of the Pre-Review Document Review prior to the on-site visits.

Back to the Top

G. Surveys

G.1. Are surveys a mandated part of the DTSDE process?

Yes, surveys of students in Grades 3-5 and 6-12, as well as surveys of teachers are mandated components of the in the DTSDE process for IIT and District-led reviews.  NYSED also encourages districts to conduct parent surveys. Once sufficient numbers of vendors are available to conduct parent surveys, SED may begin to require them as part of the process.

Districts that are conducting School Reviews (focused on SDTSDE Tenet 3) with District Oversight are encouraged, but not required, to administer surveys as part of the school review.

G.2. Who pays for the surveys?

The District will pay for the surveys. Title I 1003(a) grant funds may be used to pay for the cost of the surveys in Title I Focus and Priority Schools.

G.3. Is the State supplying the survey to the schools?

The list of approved survey vendors can be found here: http://usny.nysed.gov/rttt/teachers-leaders/approved-surveys/.  In addition to the surveys on this list, PLC Associates' Data Triangle surveys (The Community Engagement Survey, The Student Voice, and The School Performance Scan) are approved for use as part of the DTSDE review for the 2013-2014 school year. To contact PLC Associates regarding use of their surveys, please visit www.plcassociates.com external linkor email pciaburri@plcassociates.com

Only a State approved survey vendor can be used for the DTSDE review process.  If a survey vendor has submitted their survey to NYSED and is awaiting approval, a district should contact the Office of School and District Review so that further guidance can be provided.

G.4. To whom must the survey be administered?

Surveys must be administered to students in grades 3-5 and 6-12 and to teachers in Priority Schools and Focus Schools that are participating in a State led IIT school review or a District-led DTSDE review during the 2013-2014 school year.

G.5. Must all teachers and students in a school participating in a State led IIT school review or a District-led Review Using DTSDE Protocols during the 2013-14 school year be surveyed?

Districts are encouraged to survey all students and teachers.  At a minimum, sufficient numbers of students and teachers must be surveyed, as determined by the district’s selected vendor, in order to have a representative sample that will yield valid survey results.

G.6. If a district wishes to survey students in grade K-2 and/or parents, is this permissible?  May the district use 1003(a) grant funds to pay for such surveys?

Yes, 1003(a) grant funds may be used to pay for surveys of students in grades K-2 and/or of parents in a Title I school receiving a State led IIT school review or a District-led DTSDE review during the 2013-14 school year.

G.7. If a district wishes to conduct surveys of parents and/or students in a Priority or Focus School that is participating in a School Review with District oversight, may the district use 1003(a) grant funds to pay for such surveys?

Yes, as long as the school is a Title I Priority or Focus School.

G.8. If a district wishes to conduct surveys district-wide or in schools that are Priority or Focus Schools, may the district use 1003(a) grant funds to pay for such surveys.

1003(a) grant funds may only be used to conduct surveys in Title I Priority and Focus Schools. 1003(a) grant funds may be used to pay for a district-wide survey only if all of the schools in a district are Title I schools that have been identified as Focus or Priority.

G.9. When are surveys to be administered?

Surveys are to be administered (whenever possible) before the review team visits the school. The surveys are a part of the pre-documentation review that will be reviewed by the IIT to gather a better understanding of the perceptions and support that are provided by the school to teachers, students, and parents. The surveys should uncover trends and practices at the school that will lead to the development of an inquiry trail for specific events by the review team.

G.10. What if we already have a tool we are using that is driving our CEPs?

The survey tool must be State approved. For information on the State RFQ (Request for Qualifications) application and process in order to become a State approved survey vendor please visit: http://usny.nysed.gov/rttt/rfq/evalsurvey/home.html.

If a district administered an approved survey in the 2012-13 school year or will administer an approved survey in 2013-14 for APPR purposes, the district need not administer the survey twice in a school year.

G.11. Where do we get more information on the surveys and the 7Cs to inform our PD?

The 7Cs is a concept that is unique to the Tripod survey instrument created by Dr. Ron Ferguson. The only vendor that is able to administer the Tripod surveys is Cambridge Education Consultants. For information on how to contact Cambridge please go to: http://usny.nysed.gov/rttt/teachers-leaders/approved-surveys/.

The DTSDE website also has Dr. Ron Ferguson's powerpoint posted where he described the 7Cs concept and survey. The powerpoint and video of that presentation can be accessed http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/DTSDEPresentations.html.

Back to the Top

H. Report Writing

H.1. Who writes the IIT report?

The SED co-lead working in collaboration with the OEE will write the school final report. All final reports are reviewed and quality assured by the CAT liaison assigned to the review.

H.2. How many findings are required? Minimum and Maximum?

The school final report will have one finding aligned to each statement of practice. The finding will either be a strength or an area for improvement.

H.3. Are there specific words or phrases used in writing reports and report writing findings?

Yes. The reports are written using a Findings, Evidence and Impact structure:

Findings:

A conclusion reached after the examination of information from your interviews and focus group discussions during the school or district review process. 

Evidence:

  • Proof of the information used to prove or suggest the findings.
  • The sign or proof of the existence of a particular occurrence or that helps the reviewer come to a particular conclusion.

Impact:

  • The effect that the reviewer's findings supported by evidence is presently having on the school or district.  Note that the impact does not refer to the impact that will occur if the issue is not addressed, but rather the current effect on the school or district community.

I. School/District Rating

I.1. When some areas within a sub-statement of practice are observed yet all sub-statements of practice are to be rated, how do you determine an overall HEDI rating?

A reviewer can only provide Findings Evidence and Impact (FEI) statements for the events in which he or she participated, observed, and gathered evidence.  In the school summary submission to the lead reviewer, individual reviewers are to include the findings, evidence and impact statements for those sub-statements of practice. After the review, the lead reviewer and co-lead will debrief with the IIT team to discuss and come to a consensus regarding the final rating for each of the sub-statements of practice.  Post the debrief meeting, individual reviewers will submit their school summary report with the HEDI ratings to the lead reviewer and CAT liaison assigned to the review process. The lead reviewer examines and synthesizes the FEIs to create a master school summary report that will serve as the basis of the final report.  

I.2. Is each tenet and statement of practice evaluated for the review or is there a minimum requirement?

Each sub-statement of practice is evaluated as part of the NYSED IIT review.  District-led reviews must evaluate Tenet 3: Curriculum Development and Support and two additional tenets from among Tenets 2, 4, 5, and 6.

Back to the Top

J. Timeline

J.1. What is the timeline for the sequence of events?

Please refer to the school review timeline and or district review timeline for an approximation of the time for the sequence of events.

For information related to what happens after the review, please refer to the post school/district review document.

Back to the Top

K. English Language Learners (ELLs)

K.1. How are the Diagnostic Tool and the Common Core Learning Standards supporting ELLs?

The NYSED Office of Bilingual Education has worked closely with the Diagnostic Tool development team to incorporate critical information about the needs of ELLs into the review protocols. Links have been created in each tenet to the LEP/ELL PET Self Assessment Toolkit developed by NYSED.

K.2. Please clarify what “identified for ELLs” means under the waiver or AMAO. Some districts are AMAO districts but have not been identified as Focus Districts for ELLs under the waiver.

Focus Districts are identified for the performance of ELLs if the district was among the lowest performing ten percent in the State in ELA and mathematics combined or for graduation rate for the ELL subgroup and was not showing progress for this group.  Districts have been identified as AMAO districts under Title III if they have failed for two consecutive years to make AMAO 1, 2, or 3 relating to students exiting ELL status, advancing to the next ELL level, or the district making AYP for the ELL subgroup.

Back to the Top

L. DTSDE Rubric  

L.1. What do some of the sub-statements look like? How do we “behavioralize”  them? What do we see and hear as evidence and how do you assess the effectiveness of the evidence?

Statements of practice and sub-statements of practice will look different in each school and district and will be uniquely defined by the practices each school and district has put into place aligned to the needs of their communities. 

L.2. Please provide examples of how districts/schools have successfully implemented aspects of the tenets.

As both State and district reviews begin to take place, we will identify best practices and lessons learned and will present them at future DTSDE Institutes. 

L.3. What does "generic" (2.1) support mean?

Generic refers to a plan or support that is general and not specific. Please refer to the DTSDE Glossary for the definition of other DTSDE related terms.

L.4. What happens with the "grade" we get on the District/School Scoring Guide?

There is no grade provided as part of the District/School Scoring Guide. There is a rating.  Each sub-statement will receive a rating that will be aggregated to lead to an overall rating for the tenet. The rating for the tenets will inform what is an area of strength or area of improvement for the school or district. Any tenet with a rating that is at developing or below will be an area of improvement and any tenet with an effective or higher is an area of strength.

Back to the Top

M. Calibration Assurance Team (CAT)

M.1. How will we calibrate our understanding and implementation of the rubrics when each group is providing its own interpretation of the SOP and HEDI?

Calibration Assurance Team training will take place regularly throughout the school year to train CAT teams and build inter-rater reliability as well as their expertise in the rubric so that CAT members become experts who ensure that the school and district final reports are aligned to the DTSDE rubric.

M.2. Who are District CAT members?

Each district should decide who should serve as the CAT based on the needs of the district and staff capacity.  Only districts that choose to conduct district-led reviews will need a CAT.

Back to the Top

N. DTSDE Handbook

N.1. Will the document in the Handbook be shared with us in a "Word document" format?

Yes. The DTSDE handbook has been posted on line; however, only the templates and forms are in Word format. Please refer to the posted handbook at: http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/DTSDEHandbook.html.

Back to the Top

O. Schedules

O.1. When will schools be identified as needing to write Local Assistance Plans? 

The list of LAP schools is expected to be released in early spring 2013.  Plans will then need to be developed for implementation in the 2013-14 school year.

O.2. When will IIT visits begin?

School reviews began in December 2012.

O.3. Is there a school visitation schedule yet?

The preliminary schedule of visits has been shared with districts.  SED is currently working with districts to refine the schedule.

Back to the Top

P. Interfacing with other Review Protocols

P.1. How, if at all, will Title III AMAO Reviews and Improvement Plans interface with the DTSDE?

Starting in the 2013-14 school year, identified Title III districts required to develop a DCIP based on the DTSDE review process and that were identified for the ELL subgroup will not be required to create a separate AMAO Improvement or Corrective Action Plan, if all AMAO components (AMAO 1, 2, and 3) have been addressed in  the DCIP

Back to the Top

Q. District-led Review Using DTSDE Protocols and School Reviews (Focused on DTSDE Tenet 3) with District Oversight Roles and Responsibilities

Q.1. What are the roles and responsibilities for all stakeholders/positions for District-led reviews?  Who coordinates school reviews when the SED lead is not doing it?  What is the OEE role?

Each district should make their own decision on who participates and coordinates the District-led Reviews using the DTSDE Protocols and the school review (focused on Tenet 3) with District Oversight based on district needs and staffing capacity.

Q.2. Will there be training on how to conduct District-led Reviews using DTSDE Protocols and School Reviews (Focused on DTSDE Tenet 3) with District Oversight to those schools that will not be visited by an NYSED-led IIT?

If districts choose to do District-led Reviews Using DTSDE Protocols, we expect districts to follow the same steps and protocol as the SED-led reviews with the exception that the district-led reviews of schools will have the option of selecting three tenets as a focus for the review.   The district must select Tenet 3: Curriculum Development and Support and any two of the remaining tenets to be reviewed.  Districts are also expected to administer surveys as part of the District-led reviews.

NYSED will provide Webinars and other training for districts that are interested in doing School Reviews (Focused on DTSDE Tenet 3) with District Oversight.

Q.3. Is there a deadline for the District-led Reviews using DTSDE Protocols and School Reviews (Focused on DTSDE Tenet 3) With District Oversight?

Visits should be conducted so that the district can submit its final report to NYSED no later than June 30, 2013.

Q.4. How does a district do its own school visit?  What will the schedule look like?                      How does a district do a review for a school not getting a visit from the NYSED IIT?

Districts are expected to follow the same steps and protocols as the NYSED-led IIT reviews, with the exception that the district has the option of selecting three tenets to be the subject of the review. One of those tenets must be Tenet 3: Curriculum Development and Support; the district may choose the remaining two tenets from among Tenets 2, 4, 5, or 6.

The district will develop its own schedule to conduct its review of the Priority and Focus Schools that will not be reviewed by the NYSED IIT.  However, the district should use the same forms and protocols that are used by the NYSED IIT for the SED-led reviews in order to conduct the District-led Reviews Using the DTSDE Protocols. The district can use all documents referenced in the School visit section of the handbook as a guide for conducting the district-led reviews.

Q.5. Can you please confirm which district employees can/should participate on the review team? Who actually makes up the district review team?

Members of the district-led review teams should be individuals who have participated in DTSDE training. The district will want to ensure that there is a curriculum specialist on the district-led review team as an assessment of Tenet Three is a required part of the district-led reviews.  In particular, districts that have been identified for the performance of ELL or students with disabilities subgroups should have the support of the RBE-RN and RSE-TASC staff for their district-led reviews.

Q.6. We are a district/school with one preK-12 school with a student population of 325 students. Will the State come to our district? Does the Superintendent conduct the District level review and the OEE the building level review?  What is the role of the OEE in our case? 

In this case, an NYSED-led IIT will conduct both the district component of the review as well as the school review.  There will be no need for a district-led review of the school.  The OEE will participate in only the school review, unless the district requests the OEE's participation in the district component of the review.  The superintendent may serve as the district representative on the IIT visit to the school.

Q.7. What role does the BOCES have in the district-led school reviews? 

If a BOCES staff member has participated in the mandatory DTSDE training or has been otherwise certified as a reviewer, the district may have the BOCES staff member participate in the district-led reviews. In addition RBE-RN and RSE-TASC staff who are housed at BOCES and have participated in the training may participate in district-led reviews.

Q.8. Will school districts use the DTSDE as part of the self-review process once they have been identified as needing to write Local Assistance Plans ?

Local Assistance Plans (LAPs) may be required for schools that have student performance issues but have not been identified as Priority or Focus Schools.  The DTSDE report, particularly Tenet One, should inform the district's development of a LAP for any school for which one is required.

Back to the Top

R. District Comprehensive Improvement Plans (DCIP)

R.1. Can a district just use last year's plan or must the district really do another review and plan?

Districts may use last year's plan to help inform the development of a new DCIP that will be required as a result of any areas for improvement and recommendations that are uncovered by the DTSDE review process. The district should take the opportunity to reflect on activities that are already underway and discern how these activities/practices can address any new recommendations and whether the district should continue to implement or modify the DCIP.

R.2. Is there something that is submitted in response to DCIP/SCEP like a final report?

No, the DCIP/SCEP will be reviewed by NYSED, and districts will receive feedback from the reviewer.  The DCIP/SCEPs will either be approved as submitted or technical assistance will be provided for any needed revisions.  Reviewers will work with district contacts until the final DCIP/SCEP is approved. 

R.3. Is there any more guidance to complete the SCEP other than the few pages in the document itself?

Yes, webinars conducted on October 2nd and October 17th provide additional guidance.  The webinars are found at:  http://www.p12.nysed.gov/accountability/Webinars.html.

R.4. Can you clarify DCIP/SCEP plan requirements?  Should all the plan blanks be filled?  Only those pieces found in older plans such as CAP, SQR, ESCA?

It is only necessary to address the Tenets and SOPs that are applicable based upon the identified needs.  The needs assessment is based on the available reports and data for each school and district.  More information is provided in the October 17 webinar that can be found at:  http://www.p12.nysed.gov/accountability/Webinars.html.   

Back to the Top

S. Training

S.1. What kinds of provisions for training are available for District members who were closed out of this session?

There are four tiers of training: 

Tier I training is mandatory training for persons who will participate in IIT or District-led reviews.  This training will usually be provided by on-site, real time, in person Institutes, although in some instances this training requirement can be met by a person shadowing a team.

Tier II training is mandatory for OEEs and persons who will be the District lead for districts that have four or more Focus Schools.  This training will usually be provided by on-site, real time, in person Institutes, although in some instances this training requirement can be met by a person shadowing a team.

Tier III training is mandatory for OEEs and persons who will be the District lead for districts that have four or more Focus Schools but is flexible in how it is provided.  This training can be provided on-site, through webinars, or turnkey training.

Tier IV training is optional, non-mandatory training.

S.2. Will agendas be posted prior to the training along with the registration, so we can be sure not to miss an important part of the Institute that is particularly relevant?

Yes.

S.3. Do District reviewers need to attend all Calibration Assurance Team dates or choose one to attend of the two offered?

District staff who will serve as CAT reviewers are to attend both CAT training days. Districts with four or more district-led DTSDE reviews should assign a Calibration Assurance person to ensure that the visits are made with fidelity to the tool and what is stated in the report is aligned to the rubric. If there are many district-led reviews to be made in a district, additional members should be assigned to calibrate.

S.4. What level of training is required for OEEs?

OEEs are expected to participate in Tier I, Tier II, and Tier III trainings.

S.5. Who is expected to attend the in-person training sessions?

NYSED Reviewers as well as the OEEs, plus selected RBE-RN and RSE-TASC staff are expected to attend the in-person training sessions. 

S.6. When will non-SED interviewers be trained on interview protocols?

The SED and district reviewers are being trained together as part of the DTSDE Institute.  Interview protocols are one of the many topics that will be covered during training.

S.7. Are we required to have the district team participate in each in-person session for the remainder of the year?

Districts that have four or more Focus and/or Priority Schools are required to have the district leads for their teams attend each monthly DTSDE training. Districts with three or fewer Focus and/or Priority Schools are recommended to participate in the DTSDE training.

S.8. Can you please clarify who must be in attendance on Days 3 and 4 of the training sessions?

Days 3 and 4 of the DTSDE Institute are geared towards the Calibration Assurance Team (CAT) members who will ensure that all reports are written with fidelity and are aligned to the DTSDE rubric. There are a limited number of seats for the CAT trainings; hence Districts with two or more district-led DTSDE reviews should assign a Calibration Assurance person to ensure that the visits are made with fidelity to the tool and what is stated in the report is aligned to the rubric. If there are many district-led reviews to be made in a district, additional members should be assigned to calibrate.

Back to the Top

T. Other

T.1. Priority and Focus Schools need the 11-12 data to find out if they've made the 10 point PI gain that counts as the first year of progress towards removal from priority or focus status. When will we receive this information?

We expect this information to be made available to districts in late Winter/Early Spring 2013.

T.2. How do we access Ms. Gerson's Heart of Principal Teacher Practice, discussed during the Day 3 Presentation?

The powerpoint and video of Kate Gerson’s presentation can be accessed here http://www.p12.nysed.gov/accountability/diagnostic-tool-institute/DTSDEPresentations.html.

T.3. Are various offices coordinating timing of visits? I will have Title III, coordinated monitoring and this new district visit this year?   

Yes, these are all conducted by staff from the Office of Accountability, and all teams will schedule visits in coordination with each other to minimize disruption to districts.

T.4. Can we please have access to the parent involvement/engagement video?

The parent involvement/engagement video can be found at the following URL: https://www.youtube.com/watch?v=gwEPv2ob_QI external link.

 

Additional questions related to training and future DTSDE activities will be added as soon as more information is available.  Please check back for updates.

Last Updated: March 21, 2014