Importance of Data-Driven Decision-Making for School Improvement Planning

		Introduction Over the past ten years or so, "data-driven decision-making" has become a catchphrase among educators. We have heard that it is important to analyze our educational data. With the start of the No Child Left Behind legislation, this practice is no longer a luxury. This law requires that, at a minimum, we lend importance to the yearly assessment data generated by our students.

		Knowing how our students are performing right now, how they have done in the past, and how they need to improve in the future are important pieces of information that should guide the everyday practices within our schools. So what is the meaning of this catchphrase?

		Data-driven decision-making means just what it says. It is the process of making decisions based on relevant and timely information. Some wonder if data-driven decision-making replaces professional judgment. It does not. Decisions always rely on the good judgment of the decision-maker. But if our judgment is supported by personal opinions or hunches, our decisions can be misguided. Without objective information, we take the risk of making decisions that do not respond to the current situation. Data help inform our professional judgment and allow us to make better decisions.

		Everyday Decision-Making We make data-driven decisions every day. As we drive, we check the speedometer to monitor our speed. The speedometer provides data. Based on what it says, we may decide to accelerate, slow down, or maintain our current speed. But before we decide, we will want to take in additional data.

		What is the posted speed limit? What are the weather conditions? Is there a police officer in sight? We will use our best judgment to consider all of these data. Our judgment is important here. It may be influenced by our past experiences and our current state of mind.

		Have we received any speeding tickets in the past? Are we running late? Are there children in the car? After considering all of these factors, we will make a decision. Certainly the decision we make will be a more informed one than if we did not have the data provided by the speedometer.

		Similarly, a thermometer tells us if we are running a fever or not. The receipt from an automated teller machine lets us know the current balance in our bank account. And the weather report lets us know if it will be sunny, rainy, or snowy the next day. Each of these sources of data lead to informed decisions: I will take some aspirin to lower my fever I need to slow my spending until my next paycheck I will buy an umbrella tonight so I am prepared for tomorrow. These data sources have become so integrated into our lives that we take them for granted. We use them very informally and without much thought. But if they were suddenly unavailable to us, our ability to make the day-to-day decisions that affect our well-being would be impaired.

		The Business of Data-Driven Decision-Making Most professions rely on data to guide their decision-making. But the philosophy behind this practice has changed over the years. Starting in the early 1800's, American industry focused on efficient production rather than suitable products, or in other words, quantity over quality. Mass inspection techniques were used to catch faulty products before they left the factory, and workers that produced too many faulty items were dismissed. In spite of the high stakes at hand, workers were discouraged from suggesting ways to become more effective or efficient. In the 1920's, Walter A. Shewhart of Bell Telephone Laboratories introduced a technique that put the focus back on quality and the workers themselves. Statistical quality control (SQC) helped managers and workers graph their work habits and visualize how these practices influenced the quality of their products. Continuously reviewing these data, they made informed decisions about when they should take action and change the manufacturing process and when they should allow ongoing processes to operate undisturbed. The result was more standardized production methods and better quality products.

		After World War II, Japan's industrial infrastructure was devastated and American W. Edwards Deming, a protègè of Shewhart, was invited to help the Japanese rebuild. Deming taught Japanese workers to keep data on their own work and thus monitor the quality of the items they sent down the production line. Like his mentor, Deming believed that if workers could manage their own work processes, the quality of their output would improve and the manufacturing process would cost less. By 1980, Japan was known for quality products and had risen to a position of power in the global economy. The United States, no longer secure at the top of the economic heap, began to take notice.

		By 1990, the entire U.S. government and much of U.S. industry officially adopted the philosophy of total quality management (TQM). TQM is defined as the application of human resources and analytical methods to the continuous improvement of quality of goods and services.

		Deming firmly believed that to provide quality consumer products and services, it was necessary to study and improve the processes within organizational systems, and hence make a commitment to continuous improvement. The Continuous Improvement Process is best represented by the "Plan, Do, Study, Act" cycle.

		The Plan step involves analyzing where the organization is, setting goals for where it needs to go, and strategizing how the workers can help it get there. During the Do step, workers implement the plan and collect data on the process. The Study step involves reflecting on results. And during the Act step, results are shared and decisions are made to "stay the course" or modify the initial plan. Because the improvement cycle never ends, the Plan stage is then revisited to allow for adjustments that will guide and improve the next phase of implementation. Relevant data, group reflection, and a willingness to change are all critical elements of the continuous improvement process.

		Decision-Making in Education Data flow through the education system. Of the many sources of data in schools, each has the power to tell a story about our students, our teaching practices, and our school. For example, we have information about student performance on assessments, their attendance patterns, their involvement in school activities, and their feelings about school. Student performance provides the mirror from which we can reflect about teaching practices. It tells us what is working and what is not. Parents and community members provide valuable data as well. The degree to which they attend our activities, their involvement in school programs, and their thoughts about the school all suggest strengths and weaknesses about our school.

		Yet collecting, organizing, and analyzing all of these data, then putting the pieces together to create a picture of school performance is an overwhelming task for many educators. They find themselves unprepared for such work. Most teachers and administrators do not receive formal training on how to use data to guide school change. As a result, much of the valuable and telling information available to educators is not analyzed or shared among the various stakeholders in schools.

		In the absence of meaningful information, educators default to decisions based on intuition, teaching philosophy, or personal experiences. School improvement plans become a writing chore that has little to do with the actual needs of the school and its students. And the elements for continuous improvement are absent. The danger in this situation is that schools will change due to factors that have nothing to do with student needs. In this situation, the whims of a new administrator, a rising educational fad, teacher preferences, or community uproars will have an influence on school programs and practices. And year after year people will scratch their heads wondering why student performance does not improve.

		Making the Change The current practice of decision-making in education has been compared to that of medicine during the Colonial period (Carnine, 2000). In this era, medical decisions were driven by fads, expertise was based on subjective judgments, and standardized responses to common illnesses were nonexistent. The medical field was forced to mature as a profession due to external pressures by insurance companies and the Food and Drug Administration (FDA). It was only in 1962 that the FDA began requiring that drugs be proven effective and safe before they could be prescribed. Many physicians strongly resisted the imposition of scientific claims and felt threatened by the introduction of precise measurement tools. Yet today, physicians routinely rely on the research literature and on Outcome data, both of which shape and develop their professional judgment.

		Today, education is at the same turning point faced by Japanese business after the war, American business in the early 1980´s, and medicine in the early 1960´s. External factors are creating pressure to change and are encouraging the profession to mature. Mature professions have the capacity to monitor and evaluate the way they are structured, the way they operate, and their accomplishments and make necessary changes. Because schools are being held accountable based on student achievement data, they must become more data-driven.

		Data-Driven Schools Data-driven schools do exist. But currently they are few and far between. The challenge is to understand the practices these schools have adopted, and then share them so that other schools may experience the same positive changes. There are some common characteristics of data-driven schools. Data-driven schools: prioritize their data, make data analysis a shared experience, are creative with their time, link data-driven decision-making to a continuous school improvement process, and have a different power structure.

		Data-driven schools prioritize their data. With all of the data available in schools, it is important not to become overwhelmed. Too much information is difficult to analyze and make sense of. On the other hand, if schools do not collect enough data, they run the risk of focusing too heavily on one or two data sources that may not tell the whole story of their school. Because of this data-driven schools strike a balance between what data are important to them, what data others require, and what data help round out the picture of their school. Data that do not add value to their analyses or data that duplicate other information may be eliminated from the picture. When there is a gap in the information needed, data-driven schools investigate valid and cost-effective ways to collect these data.

		Data-driven schools make data analysis a shared experience. Some schools have one or two staff members that are skilled in data analysis. These individuals may have a background in research or evaluation, may have taken special training, or may simply have an interest in this area. The temptation is to channel all data and analysis chores to these individuals, allowing them to take over this task for the school. Schools that are data-driven identify those with special analytical skills or passion and find ways to help them lead and teach others. But they ensure that all stakeholders become involved in the process.

		Teachers, parents, and students themselves have much to offer schools as they press to improve. They can only help if data are shared and their ideas are welcomed. With this in mind it is important for data-driven schools to create opportunities for stakeholders to come together, share data, and make informed decisions about their work. These opportunities can come in a variety of formats. For example, a data-rich school may require that:

		Teachers and students meet briefly each week to review that student's attendance, performance on various assessments, and general well-being.

		Teachers within each grade level meet bi-weekly to share results on student assessments, discuss successful or challenging lessons, and troubleshoot student behavior problems.

		The entire staff, including administrators, teachers, and support staff, meet monthly to assess progress according to school improvement goals, celebrate successes, problem-solve challenges, and revise goals.

		The parents and community come together twice a year to learn about the school's goals, the progress it has made, the challenges it faces, and how they can help. Although their schedules may differ, all data-driven schools have regular, ongoing opportunities for decision-makers to come together, share data, reflect upon results, and make informed decisions. Creating a process for continuous improvement is a priority and scheduling supports this.

		Data-driven schools are creative with their time. Because time is a major challenge for all schools, data-driven schools get creative about it. They may change the format and structure of ongoing meetings. They may cancel regular meetings or eliminate committees that do not align with school improvement goals. Or they may tinker with scheduling during the academic day to create opportunities for reflection. Consistent with Deming's theory, schools that are data-driven become more efficient in everything they do. Time becomes less of an issue as they progress.

		Data-driven schools link data-driven decision-making to a continuous school improvement process. Data-driven schools have a purpose behind what they do. That purpose is improvement. No matter what their current level of functioning, data-driven schools know they can always do better. When they use data to reflect on their practices, they do it with the aim to identify their strengths as well as their opportunities for improvement. Data-driven schools follow the Plan, Do, Study, Act, cycle.

		Plan. Data-driven schools develop improvement plans that include input from all the major stakeholders in the school. Goals and strategies align with findings from the school's analysis of its data as well as the resources that are available. Accountability within the plan is distributed among stakeholders. Those that have a stake in the school's performance also have responsibility for following and implementing the strategies within the improvement plan.

		Do. After improvement plans are created, they are used, much like a roadmap, to guide the direction of change. Roadmaps are modified over time; new roads are added to access new areas, roads are widened to improve the flow of traffic, bridges are built to get from one place to another more quickly, and some roads are eliminated altogether. Such is the case with school improvement plans. Data-driven schools anticipate the need for modification in their plans by collecting data as they implement. These data will guide them later when they evaluate their progress and press toward the future.

		Study. Data-driven schools use the data and the observations they have made as they have implemented their improvement plan and reflect on their progress. They identify what has worked well, what goals have been accomplished, where the trouble spots lie, and begin to brainstorm about how to modify the plan.

		Act. Results are continuously shared in data-driven schools, and stakeholders have input on the school's next steps. Data-driven schools make informed decisions about how to proceed in the future. They modify their plan as needed, and move through the Plan, Do, Study, Act, cycle again.

		Data-driven schools have a different power structure. In data-driven schools, those with data in their hands are empowered. They have the ability to speak with authority about their needs and make specific requests. This includes students, teachers, support staff, and parents. The traditional top-down approach to management is turned on its head. Everyone in the system has a say in what happens, as long as they use data to back up their claims. When schools become data-driven, change is often stimulated from the bottom-up. Students may be the first to recognize problems, or their performance may provide the first sign that something is working or is not working.

		Data-driven administrators anticipate and embrace this phenomenon. They are confident enough in themselves to become facilitators, not dictators, of change. They are comforted by the fact that leadership is distributed throughout the school. Still, they understand that they are ultimately accountable for the performance of their school and its students. Data-driven leaders do everything in their power to create the optimal environment for positive change.

		With the range of data available in schools, a common framework can help structure our thinking before we attempt to organize and analyze it all. First, we will consider the broad characteristics of the data. Then, we will show how the data can be organized into three main categories.

		Data Characteristics A distinction can be made between data that are qualitative and those that are quantitative. Both of these forms of data have strengths and limitations.

		Qualitative data are not measured. Their values vary in kind but not degree. For example, teachers may be "certified", "not-certified", or "provisionally certified" to teach in their grade and/or subject area. We can label these categories as one, two, or three, yet these values are simply codes and have no quantitative interpretation. Therefore qualitative data relate to the characteristics of our students, teachers, and even the instructional processes we have in place, but they do not represent measurements. Other examples of qualitative data include: perceptions of performance student reflection and feelings parental input

		Quantitative data can be counted or measured. For example, student performance can be measured by various assessments, and may be reported by scaled scores, percentile ranks, or number correct. Student attendance can be reported by the number or percent of days present, or the average daily attendance rate. In these cases, there is an instrument to measure achievement and a standard used to count attendance. Other examples of quantitative data include: number of words read correctly number of problems solved correctly percent of students participating in extracurricular activities average number of parents attending school improvement meetings

		Quantitative data provide important objective information and can be analyzed relatively easily. Quantitative data also offer a degree of rigor to decision-making, particularly when they are derived from standardized tools that are valid and reliable. But numbers alone cannot tell the complete story about student, teacher, and school performance. Qualitative data help fill in the gaps. For instance, where quantitative data may show that the majority of parents in our school are not participating in school improvement meetings, qualitative data may suggest why. A parent focus group, series of parent interviews, or a parent survey may provide some of the reasons that parents are reluctant to participant. Quantitative and qualitative data are complementary sources of information that can guide the improvement process. Both tell a story about the school. Both inform instruction. And both provide evidence.

		Data Categories School data can be broadly clustered into three domains: Outcome, Demographic, and Process. Data from each of these areas provide direction as we make decisions. Used together, outcome, demographic, and process data provide a detailed understanding of performance.

		Outcome data measure the extent to which students demonstrate particular knowledge, skills, and achieve goals. Outcome data measure the impact of instruction on achievement. Examples of outcome data include: State assessments Local assessments Report cards Classroom quizzes Homework Portfolios Awards Student and teacher observations Graduation rate

		As educators, we have no direct control over outcome data. We cannot guarantee that a student will perform at the level that we would expect and desire. Delivering quality educational programming and targeted instruction are our indirect tools for influencing student outcomes. When trying to improve the school as a whole, there may be other outcomes that are of concern. For example, we may try to raise community, parent, and student perceptions of school quality. A survey could be used to measure our progress on this. We may also try to increase attendance, decrease administrator and teacher turnover, increase the number of certified teachers, or raise the number of students enrolling in advanced mathematics courses. All of these outcomes can be measured by simple counts and/or percentages.

		Demographic data characterize the student and his or her family and community. Group memberships and experiences, attitudes, and perceptions can be categorized and analyzed to identify any relationships they have to the manner and rate in which students learn. Demographic data can also be used to help understand other school outcomes, such as participation rates in particular classes or extracurricular activities, parental involvement, and community perceptions about the school. Examples of demographic data include:

		Grade Gender Parental education level Socio-economic status Ethnicity Mobility Attendance rate Enrollment in Gifted, Special Ed, and Bilingual Programs Extracurricular enrollment Discipline record We must never use demographic data as an excuse for the status quo. The power of using demographic data is in helping to understand our outcomes in greater detail and allowing us to target our strategies to meet the needs of all students.

		Process Data describe the components and practices that comprise the instructional program at the classroom, school, and district levels. In essence, process data describe what we do as educators: the strategies we implement, the materials we choose, our behaviors as teachers and administrators. Recall Deming's belief that when workers could manage their own work processes, the quality of their outcomes would improve. And it worked.

		Process data are the only data that teachers and administrators can control. When we implement the optimal instructional processes that meet the needs of our students, the quality of our outcomes, namely student achievement, will improve. Thus, when strategizing how to improve outcomes with different types (demographics) of students, we need to focus on our processes, such as: Standards and expectations Curriculum Assessment Alignment Feedback to students and parents Instructional time Instructional strategies Differentiated instruction Classroom management Materials Equipment School culture

		Put simply, outcome data tell us results. Demographic data help us understand the results in greater detail. And process data tell us how our work has an impact on the results and suggest how we can improve. Each of these sources of data provides an avenue to understanding the complexities of our schools. Used in tandem, we will find a rich well of hypotheses, ideas, and focused strategies for improvement.

		Summary In this lesson you have learned about data-driven decision-making, its importance in school improvement efforts, characteristics of data-driven schools, and the types of data that data-driven schools use. Now more than ever, schools are being expected to continually improve their performance according to a variety of standards and to document that improvement. Data-driven decision-making is a powerful tool for schools to achieve an ongoing effective school reform program. Data-driven schools prioritize data to ensure that information overload doesn't occur while at the same time making sure that necessary data is available. Data-driven schools make data analysis a shared experience across the range of stakeholders in the educational process. Data-driven schools are creative with their time to ensure efficiency in their work and planning. Data-driven schools link data-driven decision-making to continuous school improvement. Data driven schools create a power structure that distributes decision-making across the institution. Data-driven schools use Outcome, Demographic, and Process data appropriately to reach decisions that in an ongoing and continuous process such as the Plan, Do, Study, Act cycle.

Introduction to Data-Driven School Improvement