Preparing for a School-based Data Retreat

		Because they help participants understand the needs of their school and build a sense of teamwork and collaboration, data retreats can be powerful events for a staff. Successful data retreats are dependent on successful preparation. Preparation must occur within the groups of facilitators and participants of the data retreat. In addition, these two groups must work together at certain points along the way to accomplish key steps in the preparation process.

		Each of these steps will be described to help you anticipate what it takes to pull off a data retreat and provide guidelines for you as you work with your school to host one. But first, you must consider the key players in the preparation process: the facilitators and the school liaison. Who are these individuals and what should be expected of them?

		The Data Retreat Facilitators To facilitate means to make easy. The role of data retreat facilitators is to ease the data retreat process for participants from the beginning stages of preparation to the closing moments of the retreat. This includes helping schools prepare for the retreat, individualizing portions of the data retreat to align with the school's goals and needs, creating an environment conducive to positive change and guiding the process and pace of the event. In short, it is up to the facilitators to ensure that the data retreat meets the defined objectives and is a valuable learning experience for the participants.

		Choosing the right facilitators for a data retreat is very important. The facilitators must be able to build rapport with the participants while also possessing a variety of qualities and skills. When we speak of qualities, we have in mind characteristics or conditions that are not easily taught or changed. For example, it is difficult to teach someone to have a positive attitude and a high energy level; people tend to have them or they do not. Successful data retreat facilitators tend to demonstrate the following qualities. This is a short list and there may be other important qualities not listed here. However, at a minimum, it is advantageous for schools to consider these qualities when choosing their facilitators:

		Positive attitude High energy level Ability to motivate others Ability to think on their feet Ability to work with enthusiastic participants and "naysayers" alike

		In contrast to the qualities just described, skills can be built over time through training and experience. For example, individuals can learn time management skills to help the data retreat be an efficient process. Data retreat facilitators need to possess the following skills: Expertise in data analysis Proficiency with technology (e.g., Microsoft Powerpoint, Word and Excel) Excellent presentation skills Expertise regarding the types of data being analyzed Familiarity with the system in which the school resides Ability to plan the data retreat around the specific needs of the school and/or schools Ability to manage time Confidence in the data retreat process

		Facilitators are usually external to the school. That is, they are not members of the school staff. This allows for a degree of neutrality that is to the participants' advantage. As the data retreat unfolds, different ideas will be raised and decisions will need to be made. As an outsider, the facilitator can objectively help the participants move through these steps, monitor the group's progress and manage any conflicts. In fact, a good facilitator should be specifically trained to support this process. In addition, facilitators that have worked with other schools can bring this experience to the data retreat, giving the participants important perspective and suggesting ways other schools have addressed different challenges. With their expertise, external facilitators can step back from a problem and view it in the context of the entire educational field rather than the context of the school alone. When the facilitator is a member of the school's staff, it can be more difficult to accomplish these tasks without appearing partial to particular individuals or ideas. The number of facilitators needed for a retreat depends on several factors. The number of participants at the data retreat will have an influence, as well as the degree of their experience with the data retreat process and the quantity and variety of data to be analyzed. Most important, participants should feel that there are enough facilitators to respond to their questions and needs as they progress through the retreat.

		The School Liaison The first step in the preparation process for schools is to name a liaison for the data retreat preparation process. The school liaison provides a means of communication between the participants and the facilitators of the data retreat. This link ensures that someone consistent is working through the preparation steps with the facilitators and that there is accountability for actions that need to take place at the school. School liaisons will likely meet with the facilitators, either in person or over the telephone, several times before the data retreat to make certain that the preparation process is on track. This individual should have the following characteristics:

		Detail-oriented Comprehensive understanding of the schools goals, needs, and human resources Ability to communicate with facilitators and staff alike Accessibility Good at delegation Often, the school liaison holds a leadership position within the school, such as principal, academic head, or school reform specialist.

		Preparation Meetings For school participants, much of the preparation work for a data retreat involves ongoing organization and logistical management. This is punctuated by a series of meetings that take place to tailor the retreat to the school, communicate expectations, and follow-up on progress. Depending on the location of the school in comparison to the location of the facilitators, these meetings may occur onsite or over the telephone. Nonetheless, the structure of each will remain the same. The meetings are:

		1. Introduction Meeting: This meeting should include, at a minimum, the facilitators and the school liaison. The school may also choose to include members of the school improvement team in this meeting. The purpose of the meeting is to share background information about the school, identify the number and characteristics of participants in the data retreat, communicate the general format of the retreat, and clarify the school's guiding questions. In addition, the facilitator will familiarize the school liaison with the data retreat preparation checklist.

		2. Participants Meeting: This meeting, hosted by the school liaison, should include all participants in the data retreat. There are several purposes for this meeting. First, general information should be shared. The school liaison will want to make the participants aware of the approaching data retreat, the intended outcomes for this event, and the names and credentials of the facilitators that will be working with them. This is a good time to introduce or review the concept of continuous improvement, helping the participants see where the data retreat process fits in the "bigger picture" of school improvement. At this time, the general data retreat process can be shared, along with the questions that will guide analyses during the data retreat. This meeting provides the opportunity to solicit additional questions the participating team may have about student performance or other issues at their school. These ideas may lead to additional guiding questions that can be raised later to the data retreat facilitator.

		Next, the school liaison can move into more specific information. If participants will be divided into groups during the data retreat, such as grade level focus groups, this is a good time to advise the team of this configuration. Logistics related to the data retreat such as the location, the timeframe, and if necessary, how staffing will be handled, should be discussed. Staff should begin to consider how results from the data retreat, including new or revised goals and action plans, will be shared with other stakeholders such as students, parents, and community members. This stimulates the staff to consider change as a systemic event, rather that one that rests entirely on their shoulders.

		Finally, the data retreat preparation checklist should be shared. The liaison should use this opportunity to review the checklist, delegate tasks to appropriate staff members and agree upon timelines for their completion. For example, if one of the guiding questions relates to student attendance, each teacher may be required to bring individual student attendance records for the previous year to the retreat. The team may set a timeline for all data to be organized and in one place by one week prior to the data retreat.

		3. Follow-up Meeting: The purpose of this meeting, held between the facilitator and school liaison, is to share results from the participant's meeting, discuss the preliminary agenda for the data retreat, review progress being made on the preparation checklist, and respond to any questions. During this meeting, the facilitator and school liaison can assess the readiness of the school for the data retreat and decide if any further meetings are necessary before the retreat begins.

		Getting Organized Recall that the data analysis process begins with setting the guiding questions, which were determined during the Introduction Meeting. Next, teams must get organized. In the case of a data retreat, organization refers not only to pulling together the data, but also assembling supporting materials and supplies. This can be a big job, particularly if this is the first data retreat a school has hosted, so it should not be left to the last minute. It is not unusual for schools to find that key data, such as reports from standardized assessments, are located in several places or that portions of the data are "missing." In this case, the time it takes to prepare for the data retreat can be considered an investment in the school's future improvement efforts. The more effort put forth to pull all of the critical information together and keep it organized, the easier it will be to access and use in the future.

		For the purposes of this lesson, the steps for organizing data will be described within the context of two guiding questions that are common to most data retreats within the BIA/OIEP system: Student Achievement: What percent of our students are proficient / advanced in Reading, Language Arts, and Math? Student Attendance: What percent of our students have attendance rates of 90 percent or better? To respond to these questions, the following data sources need to be organized:

		1. Three Years of Reports from the Standardized Assessment Endorsed by Your State. Within the BIA, the current policy indicates that schools are held accountable based on the assessments being used by the states in which they reside, as long as that state's assessment program has been approved by the Department of Education. Schools whose state assessment programs are not yet approved are held accountable according to norm-referenced assessments. Because of the variety of assessments being used, the names of the reports that are needed for the data retreat differ from one school to the next. In general, the reports include the following types of data:

		Individual Student Results: There should be one of these reports for each student that participated in the assessment. This report provides information about the student's results, including the student's proficiency level, normal curve equivalent score and / or percentile rank in Reading, Language Arts, and Mathematics. In addition, it provides details about their sub-skills within each content area tested. Typically, the student's name, age, and grade are at the top of the report and an entire page is necessary to report the details of the student's performance.

		Classroom Summary Reports: There should be one of these reports for each teacher or classroom within the school. This report typically summarizes results for the entire class. The teacher's name is usually at the top, along with the grade of students reflected in the report.

		Grade Level Summary Reports: There should be one of these reports for each grade within the school. This report summarizes results for the entire grade. For smaller schools with only one teacher per grade, results from the classroom summary report and the grade level summary report will be identical. The grade level being summarized tends to be indicated at the top of the report.

		Overall School/District Report: There should be one of these reports for the school and/or district. This report is sometimes called an "administrative summary" and provides very general information.

		It is best to have at least three years of data for the data retreat so that a performance trend can be established. With only one year of data, it is impossible to place the outcomes into context. There is no telling whether the results are better, worse or even with those of the previous year. Similarly, with only two years of data, the context provided is limited. It is possible for performance to look as if it has improved since the previous year, but if the previous year was a particularly poor year, there may not be reason to celebrate. Three years of data is the minimum to identify trends in performance. The more historical data we have, the longer the trend we can analyze and the more conclusive we can be about our school's trajectory of improvement.

		As a precaution, it is a good idea to make copies of the data reports that will be used at the retreat. This ensures that the originals can remain safe and that the school will always have at least one copy in case the report is lost in the course of the data retreat. It also gives participants the freedom to mark up or highlight the reports as they use them during the retreat.

		2. Previous Year's Attendance Data for Each Individual Student. Schools handle attendance data in different ways. Some record the information in a database, others keep this information in teacher record book. At its most basic, attendance data should reflect how many days each student was present for the academic year. However, this outcome is complicated by students who enrolled after the beginning of the school year for one reason or another. These students did not have the option to be in school every day of the academic year. It is up to the school to decide how they will handle the data for these students. Some options and the pros and cons associated with them are:

		Analyze results only for students who were enrolled for the entire academic year. Pro: Keeps the analysis simple, ensuring that the same criteria has been used for each student. Con: Eliminates certain students from this set of analyses and may therefore bias the results.

		Analyze results for students according to the total number of days enrolled. For students enrolled for the entire year, the total number of days possible for them to attend is typically 180. Pro: All students can be included in the analysis. Results will represent the total population of enrolled students throughout the academic year. Con: Different criteria are used for students to arrive at a common result. That is, the resulting attendance rate has been derived based on a various enrollment situations, making it difficult to compare results from one student to those of another. This can also bias results. Although students may have attended the majority of the days they were enrolled, if their parents did not enroll them in any school until 30 days into the school year, that child has potentially missed significant learning opportunities.

		Disaggregate results for students. First, this involves analyzing results for students according to the total number of days enrolled. This is the option we just discussed. Next, the results can be disaggregated for better understanding. For example, the school may wonder if attendance varies between students that are enrolled for the entire academic year, those that are enrolled late and those that move into the school from another school. Pro: This approach allows for the best of both worlds. The result for the school can represent all students, regardless of their enrollment situation. Yet when disaggregated, the results can illuminate potential differences in attendance patterns between groups of students with different experiences. Con: This approach requires additional work and therefore additional time. It is only with this extra investment that a more refined result can be determined.

		It is helpful if the attendance data are organized by grade and by classroom to allow for other types of disaggregation. In addition, if more than one year of results are available for the attendance data, the data can be analyzed to identify trends in attendance over time, similar to what is done with achievement data.

		3. Student Demographic Data. To comply with the NCLB requirements, it is necessary to know which students are identified as disabled and Limited English Proficient (LEP). Knowing this information makes data disaggregation possible. Other demographic variables, such as gender and residential status, may be of interest to the school and should be collected as well.

		There are other things to organize besides the data, such as supporting materials and supplies. Supporting materials are documents that can be referenced during the course of the data retreat. They provide contextual information that may assist the participants as they interpret the results of their data analyses. They may also suggest changes that can be made at the school to improve results. Examples of supporting materials that align to the two guiding questions at hand include:

		Teacher's guides to the standardized assessment. Many standardized assessments have accompanying materials, such as teacher's guides, which describe the content tested at each grade level and provide sample questions. This tool can be valuable for helping teachers attach meaning to the different skill sets described on the assessment's data reports. For example, if the assessment report shows results for how students performed on the reading sub-skill called "multiple meanings," participants can look to the teacher's guide to better understand what this sub-skill represents and the type of questions that tap into it on the assessment. This can lead to rich discussions related to curriculum (e.g., do we teach this skill?) and instruction (e.g., I teach this skill, but have my students been exposed to this type of question?). From here, decisions can be made that will help students perform better in this area in the future.

		Copy of the school's attendance policy. The concept of attendance is deceptively complex and schools often discover this when they begin to analyze their data. What does it mean to be "present" or "absent"? Is this being interpreted in the same fashion from teacher to teacher within a school? Sometimes we take for granted that everyone is using the same criteria to determine attendance, but this is not always the case. There are some gray areas in terms of attendance. For example:

		If a student is present for the morning roll-call, but is checked-out of school early by a parent If a student arrives to school late due to a doctor appointment In cases like these, schools sometimes find that teachers are not collecting consistent data on attendance because they are either unfamiliar with the attendance policy or are interpreting the policy differently. When this happens, the validity of the data come into question. Having the policy on hand for reference can help clarify the situation and can help ensure that data are collected based on consistent criteria in the future. Copy of CSRP / CIMP. Each of these should be "working" documents that are referenced and updated on a regular basis. One important time to do so is in the planning stage of the data retreat. As an added benefit, as participants work through these plans, they become more familiar with the school's goals and progress that has been made toward them.

		The remainder of organizational steps for a data retreat includes pulling together various supplies, such as easels, easel pads, markers and highlighters and calculators. There is some expense entailed by the school the first time these are ordered, but those supplies that are non-consumable can be used again and again. The cost can be considered an investment for future data-driven events.

		Conclusion What occurs before a data retreat, the preparation, is critical to its success. The data retreat facilitators and the school liaison are key to successful preparation. Data retreat facilitators should possess a variety of qualities and skills. The required skills are areas in which an individual can be trained, but the qualities tend to reflect individual traits and are not easily taught. Most important for a facilitator is to ensure that school improvement teams proceed smoothly through the process and meet their objectives during the data retreat. School liaisons must communicate with both the facilitators and the participants in the data retreat and will work with the staff to manage the preparation steps onsite. Most of these steps involve organization of data, supporting materials, and supplies.

		When a school is well prepared for a data retreat, its staff has made an investment in data-driven decision-making that will pay off over time.

Using Data for Continuous Improvement