Policy makers, administrators and educators at all levels need sound data with which to make decisions. Quality data enable decisions to be made with greater accuracy. But the growing need for student achievement in the current standards-based environment has placed increasing demands on all of those involved with K to 12 education to obtain sound data with which not only to more fully understand their school systems but also to improve the quality of education.
While there seems to be no lack of school-based data, what seems to be missing is data that are generally agreed upon to be sound or of good quality. But discussions of data quality and their use with models usually brings to mind the adage of “garbage-in, garbage-out” because the quality of decisions made, the outcomes, is directly related to the quality of data used, the inputs. According to Kowalski, Lasley, and Mahoney (2008) “unless those entering the data use exactly the same metrics, measures, or procedures, the realities of one school can be quite different from the realities of another, even though the data may look similar. Because all fifty states use different tests, measures, and approaches, a real absence of common metrics exists and that void makes it difficult for educators, even psychometrically astute ones, to make good judgments based on published data.”
The key to school improvement is improved data-relevance. Therefore, it is crucial to understand how data are collected, aggregated, analyzed, reported and used. But this may sound easier than it is because data come in many forms, at many levels, and is often unconnected or is connected from the individual district, school, classroom, teacher and student. There are also different methods for data collection, aggregation, analysis and reporting.
DNA is defined as the location where your body’s cells store coded information and the pairing of the DNA bases in the middle of the two strands or helices helps to keep the coded “data” intact. Because data, like DNA, are so intertwined in the formulation of educational policy such as decision-making for funding formulas, the double helix that forms the structure of DNA might be the best way to depict our five-level model. The DNA diagram below depicts how its coded “data” or information flows vertically, up and down its two congruent helices, as well as horizontally, across the base pairs.
In our model the vertical dimension relate each level to the one above and below it. The vertical dimension represents the ways in which the data bubble up and down between and among the various levels. Using DNA’s double helix to exemplify our model, individual student test score data move up from the student in the classroom for collection at the school level before being aggregated at the district level and reported to the state Department of Education where the data are analyzed and transmitted to the federal level for nation-wide use. The federal data, in turn, are disseminated back to the individual states for their use and, in turn, the states provide the information to their school districts for policy making purposes among other things. The districts share the information with the schools within the district so that improved curriculum, operational and program student-centric decisions can be made.
Just as the “base pairs” of DNA intersect the two “sugar phosphate backbones” or its two helices, our model’s horizontal dimensions form the important intersections with its two helices or data flow “backbones.” Our model’s horizontal dimensions include the relationships, comparisons and uses within a level such as:
- Federal level: Comparisons of the American educational system with those of other nations.
- State level: Comparisons of school systems between and among states.
- District level: Comparisons of local education authorities (LEA) or school districts with one another especially ones that share similar characteristics.
- School level: Comparisons of different schools either within a state or across a number of states.
- Student level: Comparisons of students according to various factors such as socio-economic status, race, gender, subject matter, and grade level.
The dimensions and intersections of our model resemble those of DNA as data flow vertically, up and down the two congruent helices, as well as horizontally, across the levels as shown below for our model:
|
School |
|
Student |
|
State |
|
District |
|
Federal |
This study poses a five-level model for data building and data use that is intended not only to help gather the right types and “levels” of information but also to put the information where it is needed most and best used. It examines the five key levels of education-based decisions as highlighted above, identifying the availability and limitations of data at those levels as well as how the data analysis might affect education at that level and throughout the system. While decision-making depends on data, it is important to explain the limitations at each level and what might be done (for good or for bad) by creating more information at that level.
Level 1: Federal Data and Decision Making:
The United States is beginning to create a national system of schools, with national accountability and nationally as well as internationally comparative data. This is further necessitating more national standards, alignment of curriculum across the states, and new reliable data on how America’s schools are performing. All nations of the world have information on their schools and many provide comparative studies.
Federal level data are actually an aggregation of state level data such as data collected according to the No Child Left Behind (NCLB) Act, the National Assessment of Education Progress (NAEP) often referred to as “The Nation’s Report Card” and other state-level achievement tests. “The Nation’s Report Card” is an aggregation and an analysis of the NAEP test results and the NCLB Act requires the NAEP testing of all students nation-wide in the near future. NAEP provides a measure of how students in grades 4, 8 and 12 nation-wide are performing in mathematics, science, and language arts.
The National Center for Education Statistics (NCES) holds a wealth of information on schools and student performance nation-wide particularly student demographic data and school district financial data. The NCES also provides analyses of its data in such publications as the Education Statistics Quarterly, the annual Conditions of Education report, the Nation’s Report Card, the Digest of Education Statistics, and reports on selected current educational issues.
Level 2: State Data and Decision Making:
States, through their departments of education, collect, aggregate and report data to the federal level as well as other levels through measures such as the NCLB, NAEP, and in New Jersey, the New Jersey Assessment of Skills and Knowledge (NJASK.) The NJASK is a state assessment of public school student achievement in grades three to seven which is administered by the New Jersey Department of Education. The NJASK is defined by the New Jersey Core Curriculum Content Standards (CCCS) in language arts, mathematics and science that was implemented to help meet the requirements of NCLB. The NJASK test is given for up to two hours per day covering a three to five day time frame. The questions are either multiple choice or ones requiring a written response.
The New Jersey CCCS provide local school districts with benchmarks for student achievement of the skills the State of New Jersey expects its public school students to acquire during their K to 12 education in nine content areas. These benchmarks set the levels which students should attain in the following areas:
- Visual and performing arts
- Health and physical education
- Language arts literacy
- Mathematics
- Science
- Social studies
- World languages
- Technology
- Career education, consumer, family, and life skills
The CCCS are “outcome statements” that form the basis of “strands” and “Cumulative Progress Indicators” (CPI). Strands are defined as tools to help teachers identify content and skills. Each strand is composed of a number of CPIs. The CPIs provide the specific content and skills to be taught at the appropriate grade levels.
Level 3: District Data and Decision Making:
In all states except Hawaii, the Local Education Authority (LEA) or school district is the major decision-making setting. The overwhelming majority of districts elect boards of education who in turn hire the superintendent as well as other staff and operate the school system within the LEA or district. Hence, data gathered, analyzed and acted upon at the district level are critical to the system of control and accountability.
Districts play a central role in collecting data as well as in using data to improve student achievement. While nearly all districts nation-wide generate some sort of district “Report Card,” districts in New Jersey are key to the process of collecting, aggregating, reporting and using data through such measures as the:
- Grade Eight Proficiency Assessment (GEPA)
- High School Proficiency Assessment (HSPA)
- Advanced Placement (AP) program and tests
- New Jersey Quality Single Accountability Continuum (NJQSAC)
A variety of tests are used to assess public school student achievement as well as to help improve public education through data collection in New Jersey school districts. The GEPA is a standardized test administered to all New Jersey eighth graders on several subjects and is very similar to the HSPA. As such, the GEPA is often referred to as the “preparation test” for the HSPA. The HSPA is a standardized test administered during a four day period to all New Jersey high school students in their eleventh grade or junior year on language arts literacy and mathematics. Public school students must pass the HSPA exams to graduate from high school in New Jersey. The Advanced Placement (AP) program provides high school students with a way in which to earn college level credit depending how well they perform on the subject matter exams given for the AP level courses they attend.
The system for monitoring and evaluating New Jersey’s public school districts is the New Jersey Quality Single Accountability Continuum (NJQSAC) which is often referred to as the “QSAC.” QSAC replaced the Quality Annual Assurance Report (QAAR) beginning with the 2006-07 school year. As a result it shifted the focus from primarily compliance to district, individual school and student improvement. The QSAC combines a wide range of state monitoring requirements with those of the federal government into a “single” system of monitoring and evaluating school districts. All New Jersey school districts must perform an annual self-assessment according to five key components:
- Instruction
- Personnel
- Financial management
- Operations
- Governance
The QSAC addressed the problem of a large number of significantly different and often conflicting state and federal monitoring and evaluating requirements. The QSAC simplified the monitoring of district performance by forging one set of standards for all school districts as well as enabling districts to make their own adjustments more readily. It also enables more informed school district comparisons through the use of a “continuum” on which all districts are rated.
Level 4: School Data and Decision Making:
The school is the primary working unit for education and as such it is also the primary decision-making unit. Many educators, central office staff and policy makers tend to believe that those closest to the classroom because of their daily access to students and their performance have a more in depth understanding of school-centric and student-centric data. Therefore, those at the school level may be better positioned to make more informed decisions concerning educational programs and services than those at other levels especially at the state and federal Departments of Education. Examples of school level measures include school “Report Cards” and Annual Yearly Progress (AYP).
Level 5: Student Data and Decision Making:
Ultimately, the level of decision-making and analysis is the student: the child is taught, supported, tested and reviewed in many ways. Data are collected on students according to many factors including but not limited to subject matter, grade level, socio-economic status, race, gender, Limited English Proficiency (LEP), Advanced Placement as well as special education and Individual Education Plans (IEP).
_______________________________
References
Kowaski, T. J., Lasley II, T. J., & Mahoney, J. W. (2008), Data-Driven Decisions and School Leadership: Best Practices for School Improvement, Boston: Allyn & Bacon.
State of New Jersey, Department of Education, web site. United States Library of Medicine, DNA Double Helix diagram.
