Basic Assessment Literacy: Focus on Standardized Assessments

		Introduction As the issue of accountability in education has come to the forefront, so has the issue of educational measurement. To begin to understand the degree of success the educational process has on student outcomes there must be ways of assessing these outcomes that are valid and consistent.

		Standardized assessments, also known as standardized tests, large-scale assessments, and "high-stakes" tests, fill this need. Assessments that are standardized are administered, scored, analyzed, and reported in a uniform manner.

		For purposes of this lesson, standardized assessments will refer specifically to those that are typically administered annually and are mandated by federal, state, or local educational policy. The items, or questions, on these assessments are typically aligned to state or national learning goals.

		The use of standardized assessments in education has been in practice since the 1960's and was spurred by our economic and political rivalry with the Soviet Union. After the Soviets beat the United States to space in 1957, politicians advocated for increased testing to assess student learning.

		In the 1970's, concerns were raised that the achievement levels of U.S. students were falling behind those of students from other countries. As a result, a new wave of testing began, this time assessing basic skills to ensure that all students had at least the minimum level of competency needed to compete in society.

		By the 1980's, minimum competency tests fell out of favor. One of the reasons was the perception that the "minimum" level of knowledge and skills assessed on these tests inadvertently lowered standards for teaching. A Nation at Risk, a well-publicized report released in 1983 by the National Commission on Education, pressed for assessments based on higher standards and linked to higher "stakes," or consequences.

		Policymakers for education took note. Today, every state but Iowa has developed learning standards and all states but Iowa and Nebraska have mandated assessments to evaluate their impact. The number of standardized assessments has broadened significantly and the focus on their results has become more acute. The No Child Left Behind Act of 2001 exemplifies the contemporary importance of standardized assessments. It requires that each state administer annual tests for children in grades three through eight in Reading and Math beginning in the 2005-2006 academic year. The rationale for this policy, as stated by the U.S. Department of Education, is that these tests:

		Measure performance to gauge whether children are meeting standards Identify students that need help, so they are not "left behind" in their learning Identify how well schools are helping students learn With all eyes on standardized assessments, both now and in the future, it is critical that educators understand how they fit into the overall assessment picture, their characteristics, and the types of information they provide.

		The Big Picture of Assessment It is important to note that standardized assessments published by states or large testing companies are necessary, but not sufficient, components of a comprehensive assessment program. All forms of assessment have strengths and limitations that relate to the type of information they offer and their ultimate utility.

		The strength of standardized assessments relates to their breadth in information and utility. Information from standardized assessments can be used to help schools improve their curricula, to suggest the effectiveness of different instructional processes, and to measure the knowledge and skills of students. Because they are administered to large groups of students, the results from these assessments can be used to monitor the educational system as a whole. And when the same standardized assessment is administered year after year, the resulting data provide an important measure of improvement over time.

		This form of assessment also has its limitations. Although the data from these assessments can provide some specifics about individual student strengths and weaknesses, educators must keep in mind they reflect the student's performance during a given period of time and under specific conditions. The student's performance on the assessment may not reflect his or her true capacity if the conditions for testing were less than optimal.

		In addition, standardized assessments are typically given once every year, often in the months of March, April, or May. While the information they provide is valuable, it is too infrequent to guide the teaching process in a continuous way. In this sense, standardized assessments are a form of summative evaluation. Summative evaluations are performed as an activity or program is completed, such as the academic school year. The outcome data they provide help decision-makers consider the program's overall merit based on what was accomplished, the positive or negative consequences, and the final results. For example, standardized assessment data may suggest whether a new method for teaching reading has been effective by comparing outcomes in reading from one year to the next.

		Finally, some educators question the cultural appropriateness of standardized assessments. Native Americans may feel that some of the vocabulary and concepts reflected within are unfamiliar to their students or do not reflect the reality of their culture. The challenge in this situation is to recognize the contributions standardized assessments bring to a comprehensive assessment program while recognizing their limitations.

		A comprehensive assessment program will supplement information offered by standardized assessments with other tools throughout the academic year. It should also compensate for any limitations posed by different assessment tools. Periodic grade level, subject area, or curriculum-based assessments can provide ongoing data that help educators monitor the education process as it occurs. And, classroom assessments give teachers and students immediate, targeted feedback regarding what has been learned. These may be designed with greater cultural sensitivity than is offered by assessments designed for a broad national population.

		They can be also be used for formative evaluation. Formative evaluations occur as an activity or program is being implemented. The data provided by formative evaluations suggest whether progress is being made and allow for adjustments and improvements to be made along the way. For example, classroom assessments may suggest to a teacher whether a particular reading lesson has been effective by indicating how many students mastered the targeted skill.

		When considering the value of summative versus formative assessments, each contributes to the continuous improvement process. The rate of feedback they offer simply differs and leads to different kinds of decisions. This lesson's focus on standardized assessments as important tools for all educators is not intended to diminish the importance of formative assessments, the need for multiple measures of student performance, or the need for a comprehensive assessment program.

		Myths and Realities about Testing When it comes to standardized assessments, there is no shortage of opinions regarding their value and utility. Standardized assessments have their detractors and advocates, and at times the discussions between these groups can become quite passionate. The U.S. Department of Education acknowledged and responded to some of these common objections on their web pages devoted to No Child Left Behind. Some are paraphrased below:

		Objections to standardized assessments include the following: Testing suppresses teaching and learning. Testing narrows the curriculum and rewards test-taking skills. Annual testing places too much emphasis on a single exam. Testing can cause anxiety in some students. Testing hurts the poor and people of color. Testing will increase dropout rates and create physical and emotional illness in children.

		Rationales for standardized assessments include the following: Learning must be assessed. Teachers can use the information gained in testing to better understand the needs of students. Standardized tests are aligned to rigorous learning standards. If standards are rigorous, learning will be as well. Annual testing provides important information about teaching and learning.

		Accountability systems, based on testing results, make it impossible to ignore achievement gaps. Children drop out not because of testing, but because of frustration when they cannot read, write, or learn. Testing helps identify students with learning needs. Testing is part of life; children need to be equipped to deal with it.

		There is a middle ground. Standardized assessments will neither make nor break the process of education reform. They are simply tools we can use to inform our work, allowing us to make ongoing improvements to the educational process. Like any tool, they must be used appropriately to provide optimal results.

		Types of Standardized Assessments There are two major types of standardized assessments: norm-referenced tests (NRTs) and criterion-referenced tests (CRTs). At face value, these assessments may look similar. They are both given once a year in a standardized format, have similar types of questions, and provide a broad array of data. It is the manner in which the data are manipulated that differs between norm- and criterion-referenced assessments. How student performance is described depends on which of these assessments is given.

		Norm-referenced tests are developed based on concepts and skills commonly taught throughout the country and are designed to differentiate between students. Examples of norm-referenced tests include: Stanford Achievement Test-Ninth Version (SAT 9) Iowa Test of Basic Skills (ITBS) Terra Nova California Achievement Test-Fifth Edition (CAT 5) Developers of norm-referenced tests create test items and experts review them to ensure the items are valid, or measure what is intended. Groups of students are given early versions of these tests to narrow down the number of items that are included. Items that are answered correctly by most of the students are thrown out, as are items that most students miss. This improves the chance that student performance will vary on the test.

		When students take a norm-referenced test, their performances are compared with those of a specified reference group, typically a representative national sample of students in the same grade. The median score on the test for each sample group, called the norm, is designed to reflect the performance of the "average" student in the nation.

		When a student takes a norm-referenced test, the resulting data describe how his or her performance compares with peers throughout the country. Statistical terms such as percentile, grade equivalent score, scaled score, and normal curve equivalent are common indicators. Data from norm-referenced tests tell less about what a student knows or is able to do and more about how his or her performance "stacks up" to others. A common analogy states that norm-referenced scores tell you which students are farther up the mountain, but not how far anyone has climbed. Therefore, ranking well compared to peers does not always guarantee that the skills being assessed have been mastered. For example, if everyone taking the norm-referenced test does poorly, a student doing better than most still may not know the material well.

		This exposes a limitation of norm-referenced tests. There will always be students that rank "below" or "above" average on norm-referenced tests but these rankings are not precise indicators of student knowledge. In addition, because norm-referenced tests are developed according to national curricular trends, their alignment to state standards is often close, but not assured.

		For these reasons, most states have designed, or are in the process of designing criterion-referenced tests. Items on criterion-referenced tests are developed to reflect specific educational objectives, typically derived from established state standards. Data from criterion-referenced tests describe which learning objectives students have mastered.

		Students may be placed into performance categories, such as "partially proficient," "proficient," and "advanced" based on predetermined cutoff scores, but they are not ranked against other students. On criterion-referenced tests, it is possible for all students to be "proficient" or even "advanced" if they all demonstrate the required knowledge and skills.

		Educators often find value in combining information from norm- and criterion-referenced tests to obtain an in-depth understanding of student and school performance. They may use a norm-referenced test to understand how the performance of their students' ranks compares with students throughout the country and a criterion-referenced test to evaluate which students are meeting their state's learning standards.

		The Use of Standardized Assessments With all the discussion about summative versus formative assessments, and norm-referenced versus criterion-referenced tests, the bottom line is this: no assessment will add value to the business of educating children unless it is used. Recall the continuous improvement process represented by the "Plan, Do, Study, Act" cycle. It is during the Study step that we reflect on the results of our work and analyze information necessary to guide future decisions. Standardized assessments should enter into those analyses every year soon after a school receives its results.

		To begin the process of analyzing results, schools should develop guiding questions that will focus the types of analyses to be done. It is best to start with broad questions that correspond with overall school goals, then continue with more focused questions depending on the trends that emerge in the data. The current year should be examined in detail, and then changes in the results over time should be determined by comparing current results to those of previous years.

		For example, one of the five goals within the BIA/OIEP system is that "70% of students will be proficient or advanced in Reading and Math." First, a school would want to focus on the current year's results to establish whether this goal has been met. A guiding question might be, "What percentage of our students are performing at proficient or higher levels in Reading, Language Arts and Math?" In this example, a school may include Language Arts data in their analyses due to the overlap between this subject area and Reading. Using guidelines established by BIA/OIEP to define proficiency levels, the school would quantify the number of students falling into the proficient and advanced categories and determine what percent of the student population they represent. These data can be broken out, or disaggregated, by demographic variables such as grade, gender, and residential versus non-residential students, to identify any differences in performance between groups.

		A guiding question might be, "Are both boys and girls meeting Reading, Language Arts, and Math goals at the same rate?" The goal should be to minimize any differences between student demographic groups while ensuring that all meet high standards.

		The next guiding question relates to assessing change over time. It might ask, "Has the performance of our students changed over time?" To determine trends in the data (i.e., are we improving, are we getting worse, or are things about the same) it helps to have a minimum of three years of information. Assuming the same standardized assessment has been given for the past three years, the data can show improvement by grade levels, and can also be used to gauge the improvement of student cohorts.

		After looking at broad data trends, schools can become more focused in their analyses. Their next guiding question might ask, "How do our students perform on the knowledge, skills, and concepts tested in the broad content domains?" If the standardized assessment is aligned to the state standards, this question allows educators to start relating student performance to specific learning objectives.

		Criterion-referenced tests that are mandated by states are designed to align to their standards. Their reports often indicate the percent of students that have reached mastery on the various standards. Norm-referenced tests are aligned to common standards throughout the country, but as standards do not vary significantly from state to state, they tend to provide reasonable alignment. Their reports also indicate student attainment toward various objectives. For example, a norm-referenced test may report the percent of students that performed in the "average," "below average," or "above average" range compared to other students.

		These analyses provide a comprehensive picture of school performance that can then be discussed with stakeholders, compared with other sources of data, and contribute to decision-making about school improvement goals. If a school desired, they could continue with more refined analyses, such as tracking individual student gains from year to year, performing item-analyses, and relating student performance to attendance and other factors. When using standardized assessments, it is important to keep the following cautions in mind:

		Beware of making high-stakes decisions or broad-sweeping judgments when the number of students tested is small. When in doubt, seek additional data to confirm the trends.

		When using a norm-referenced test, check the alignment of the objectives within each subject area to the state standards. This will help decision-makers understand the degree of weight they should lend to the assessment results.

		Understand the statistical terminology used within your standardized assessment reports. Know what the terms mean and how they should be used. Some statistics are not appropriate for certain analyses. An excellent reference is provided in the resource section for this lesson.

		Standardized assessment data, and all assessment data, should help educators understand strengths and weaknesses in the schools academic programs, teaching processes, and in individual student knowledge and skills. However it does not tell educators what to do with this information. Follow-up data analyses with further investigation into the curriculum, teaching practices, teacher preparation, professional development, student attendance, student behavior, and other issues to help identify directions for improvement.

		Summary Standardized assessments play an important part in a comprehensive assessment system. The emphases of standardized assessments have varied over the years in relation to varying public concerns about the educational system. Today, the collective focus on these forms of summative evaluation for schools is even stronger due to the No Child Left Behind Act of 2001. It is important that educators are aware of the types of standardized assessments used in their schools, the consequences linked to them, and how they can be best used within the continuous improvement process.

Introduction to Data-Driven School Improvement