Part of the NCA Commission on Accreditation and School Improvement Journal of School Improvement, Volume 4, Issue 1, Spring 2003
Review of the Literature

David Bitter, Jay Heath, Karen Enos


About the Authors: Dr. David Bitter is a member of the professional staff at NCA CASI and serves as the Assistant Executive Director for the Central States Team and the accreditation functions of the Commission. He can be reached at dbitter@ncacasi.org.

Dr. Jay Heath is a Professor of Educational Administration at the University of South Dakota and is the NCA CASI State Director for South Dakota. He can be reached at jheath@usd.edu.

Karen Enos is a doctoral student and graduate assistant in Educational Administration at the University of South Dakota. She can be reached at kenos@usd.edu.

 
Previous Article | Next Article | Contents, This Issue | Feedback | JSI Home | NCA Home
 


Adkison, S. and Tchudi, S. (2001). Reading the Data: Making Supportable Claims from Classroom Assessment. English Journal, (September), 43-50.

"Are we assessing ourselves to death?" is the question posed by the authors. The focus on assessment is traced back to E. L. Thorndike and the scientific education movement. The proponents of this movement in the 1920s postulated that anything important can be measured and anything that cannot be measured is not important. While Adkison and Tchudi agree that we are assessing ourselves to death, they offer a different perspective on assessment, provide some guidance in the use of assessments, and present a case for interpreting assessment through the perspective of the Toulmin model of logic. They begin by offering three arguments for evaluation pedagogy:

  • Assessment need not be a killer of innovation if it is consistently linked to teaching.
  • Alternative assessments provide richer data than standardized tests.
  • Students taught and assessed with a theory-based, multiple measures pedagogy will do as well as, if not better, on standardized tests than students who have been taught to reach the lowest possible acceptable score on blanket assessments.

Using three stories about students, the Toulmin model is illustrated throughout the article. The model is designed to account for factors resulting from both the range of the understanding and convictions of various audiences and the complexity of the situation.

While portions of the article are complex and somewhat theoretical, several important points are made that should be considered by educators. The first point focuses on the use of data, indicating that while various audiences may be able to agree on what the assessment data are (69th percentile), what the data mean to each audience is influenced by the general patterns of thinking of the audience, the information that backs their way of thinking (from research to tradition), and the assumptions that the understandings are founded upon. Educators clearly have different understandings, research knowledge, and traditions concerning assessment than those of politicians, the media, business people, or parents. A second point illustrates the importance of presenting the reasons, or arguments, that support one's interpretation when presenting data to a different audience. A third point of the article underlines the importance of using multiple methods of assessment and building skills in understanding and interpreting data.

NCA CASI has encouraged, promoted, and required multiple means of assessment that draw upon both standardized and locally developed measures. The authors appear to support this concept. An important lesson for educators presented in this article is that if we want the public to understand and interpret assessments in the same way we do and understand how tests are currently misused and incorrectly interpreted, then we must help them understand assessment from the educational perspective.

Licata, J. W., and Harper, G. W. (2001). Organizational Health and Robust School Vision. Educational Administration Quarterly, (37, 1) 5-26.

Schools engaged in genuine school improvement have long acknowledged the importance of a vision, the "picture" of the preferred future for the school. Effective leaders know that they need a clear vision to guide their personal decision-making and their school. An abundance of research has also focused on the importance of creating a positive organizational health as an important aspect of leadership. Licata and Harper's research investigates the relationship between a robust school vision and the organizational health of the educational institution. Their first contention postulates that the robust vision, one that is high in dramatic content and illustrates the discrepancy between the present and the future, will arouse teachers to a higher level of action than a bland or vague vision. The second contention proposes that teachers in a healthy organization are more likely to engage in the risk-taking needed to achieve a vision and that the simple promotion of a vision in a less then healthy environment would not result in the implementation of the vision.

A robust school vision was described as interesting, action-packed, powerful, and challenging. For purposes of the research, characteristics of healthy school organizations included the following:

  • Teachers frequently observe the principal and his/her colleagues working with parents, community, and the central office to develop support and resources.
  • Open and collegial leadership by the building administration.
  • A principal who encourages teachers to try new methods.
  • Teachers who provide assistance to students in order to meet high expectations.
  • Open teacher communications with one another and a willingness to explore new ideas.
  • A high level of trust between individuals and groups in the organization.

The study included 38 middle schools and reflected the perceptions of 554 teachers. Vision robustness was measured using the Robustness Semantic Differential and the health of the organization was measured with the Organizational Health Inventory. The investigators found a number of correlational relationships among the sub-scales of the instruments. The correlation between the organizational health and the robustness of the vision was both positive and significant.

While the reader must remember that correlation does not equal causation, the statistical relationship found between these two factors provides a support for something that good leaders have understood for many generations. Effective leadership, and effective schools for that matter, results from a complex inter-relationship of many components, constructs, and capacities. Leaders might note that the researchers didn't find any influence by socio-economic status of the school populations on the relationship between vision and organizational health.

Achilles, C. M., Finn, J. D., and Pate-Bain, H. (2002). Measuring Class Size: Let Me Count the Ways. Educational Leadership, (February), 24-26.

This brief article explores the number of confusing ways class size is determined and the importance of understanding how it has been defined in interpreting the results of research related to class size. The most important point made by the authors is that class size is not the same as student-teacher or student-staff ratios. Pupil-teacher ratios include all of the support and specialized staff that serve students in a school. While a student-teacher ratio might be 17:1, it is still quite possible that the average class size in a building might be 30 students. The authors point to research that indicates that the typical difference between pupil-teacher ratios and class size in US schools is about 10. If the pupil-teacher ratio is 17:1, one could expect to find average class size to be 27!

The article draws particular attention to class size research that most educators are familiar with including the STAR study. The noted studies all focus on class size not pupil teacher ratios. It is smaller class size that makes a difference not lower pupil-teacher ratios. The conditions for smaller class size to have a substantial effect are noted in the article. Class size reduction efforts must occur early, they must have a duration of at least three years, and their efforts must be intense (students must be in small classes every day, all day).

Class size continues to be in the center of the educational reform debate. Current economic challenges may diminish this focus, but if educators and administrators hope to sort out the plethora of research on class size and hope to help board members, politicians, and parents understand what it means, it is important for them to understand how class size is determined in the research before considering the results of the research. This article does an excellent job of communicating the differences.

Patterson, J., Breeding, K., Puetz, M., Thimmesch, M., Torres, B., Vogel, R., Welty, M., and Woodward, K. (2002). Faculty Meetings: From Dull to Dynamic. Principal Leadership Middle Level Edition. (October), 36-40.

School leaders, regardless of the level or role, all know the challenge to make meetings productive and worthwhile. The authors of this article offer an organized set of suggestions for the structuring of faculty meetings based on elements from identified sources backed with a limited research project. The article offers a useful outline with brief but illustrative examples. According to the authors, there are four major components of dynamic and successful faculty meetings with a number of supporting areas for each component. These components and supporting areas include the following:

Design With Purpose

  • Set beginning and ending times and follow them
  • Establish a reasonable length (30 minutes is the most effective)
  • Prepare and distribute an agenda
  • Limit discussion time
  • Solicit input from staff
  • Begin on a positive note

Prioritize the Agenda

  • Include staff development, school improvement, and communication
  • Plan for large and small group discussion
  • Provide for variety
  • Discuss issues and problems
  • Share routine communications by memo or some other means
  • Hold meetings only when necessary

Engage Attendees

  • Follow the agenda
  • Don't waste time, get to the point
  • Involve everyone
  • Provide balanced speaking opportunities
  • Discourage side conversations
  • Be energetic
  • Have fun or provide some activities that are fun

Of important note to NCA school leaders is that only three activities are highlighted as typically appropriate topics for a staff meeting. School improvement is one of the three! If trivial and routine communications are conducted through other means and faculty meetings are held only for important topics, the inclusion of school improvement activities in these meetings will certainly highlight the importance of the process. In a well-developed and effective school improvement plan, staff development is aligned with the interventions and activities in the plan. One might reasonably conclude then that, between the staff development and school improvement topics, the school improvement process should drive the majority of all faculty meetings.

Brown, F. (2001). Site-Based Management: Is It Still Central to the School Reform Movement? School Business Affairs, (April), 5-10.

The question posed by Brown is certainly important to educators as they struggle with new waves of school reform, federal and state mandates such as No Child Left Behind, shrinking budgets, and media criticism. Site-based management has its roots in the 1960s and became very popular as a school reform movement in the 1980s. At its core are the principles of expanding leadership and decision-making authority to the staff, parents, and community of the local school. According to the author, five states currently mandate site-based management and other states and schools require certain aspects of this management form.

Has site-based management [SBM] improved schools? Brown contends, at least based on improving student achievement, that the evidence from the limited number of studies isn't very positive. In his own words, "Most studies have found no relationship between SBM and students' academic achievement." Why then is it such a popular reform effort? Brown offers some insight into this question.

  • SBM creates the impression that a dramatic change has taken place.
  • It is a "low" threat strategy for teachers, administrators, unions, and community representatives.
  • It is visible to the community and painless to the system.

In a further analysis, Brown indicates that SBM has failed for a number of reasons and probably hasn't been implemented as it should be in many cases. Problems cited included political power base structures, the lack of desire on the part of teachers to be responsible for some aspects of school operation such as teacher evaluation, traditions in a system, ingrained organizational structures and vested interests, the lack of training and resources, and a lack of trust.

While NCA CASI certainly supports shared decision making and responsibility through the involvement of administrators, teachers, parents, and students in the school improvement process, Brown's information provides an excellent illustration of a principle that is a core belief in the NCA process. Any school improvement plan must focus on the achievement of students and must involve what students will engage in and be able to demonstrate as a result of the efforts. It is quite possible that SBM hasn't brought about a change in achievement because it focuses on how we manage the school rather than the work the students do, how they do that work, and what educators might do differently to help them learn.

Longstreet, W. S. (2002). Profit and Loss in the Classroom: Will the Business Model Bankrupt Education? Social Education, (66, 7) 450-452.

Longstreet openly attacks the imposition of what she refers to as the business model (bottom-line profit or loss) through high stakes testing. The article traces the testing movement through the latter part of the twentieth century and concludes this historical recount with a very insightful statement: "The use of the business model for evaluation of the quality of education has become deeply embedded in the public's thinking" (p. 450). The author equates the business model with a "simplistic" approach to complex situations and problems and makes an impassioned plea for those outside of education to recognize that neither the goals nor the outcomes of education are simplistic. Indeed, she points out that different groups purport different goals for education.

Through individual illustrations, research, and personal opinion, the author presents a number of risks that are being ignored or are in danger of being forgotten if high stakes testing continues to be seen as the only way to improve education. Areas of risk presented in the article include lower performance levels in the arts; a threat to the spirit of inquiry in the sciences; the inability to deal with complex, controversial, or ethical challenges; and a removal of any focus on citizenship and social understanding.

Longstreet offers a number of credible reasons why we shouldn't depend on high stake tests as the measure of education or educational reform; however, she doesn't address what is probably the most significant statement in the article concerning testing: this type of measure "has become deeply embedded in the public's thinking." The arguments in the article must be presented to the public in a manner that offers a different way to understand assessment and includes alternative ways to satisfy their desire for evaluation and results. Perhaps such a role of adult education fits in well with the NCA CASI Transition Model of continuous education and transition.

Brent, B. O. (2001). School Volunteers: Hidden Benefits and Hidden Costs. School Business Affairs, (April), 14-18.

Reformers have called for the use of volunteers in schools for many years. Teachers and administrators have sought volunteers to help in a number of ways and may well be forced to expand such efforts as economic conditions challenge schools to provide more services with less staff and less resources. Brent sought to explore the following four areas in the reporting of this study:

  • Who volunteers in the schools?
  • What types of activities do volunteers perform?
  • Do volunteers benefit schools?
  • What are the costs of volunteer use?

The results of the study indicated that the volunteers were overwhelming females and they fell within the age span of 36 to 55, although a number of volunteers were senior citizens. Volunteers worked an average of 2 hours a day twice a week and typically volunteered for 23 weeks. Most volunteers supported classroom activities or tutored individual students, mostly in the areas of reading, writing, and math, with reading receiving the most attention and math the least. Volunteerism decreased steadily with grade level, possibly due to social concerns of children and the higher complexity of the subject matter in the classroom. While the research indicated that volunteers did impact the climate of the school, Brent did not find any relationship between the number of volunteers in a school and its performance on state administered reading and math tests. The volunteers may impact the achievement of individual students, but they did not appear to have an impact on the overall academic performance of the school. Volunteers did report much higher levels of respect for teachers and a better understanding of education, resulting in better school-community relations. The principals in the study reported an average of 32 hours spent administering volunteer activities.

Brent concluded that the cost of volunteers, mostly in time for administration, is worth the effort and results in better school-community relationships, improved achievement for individual students, and contributes to a positive building climate. The analysis of the results of the study indicated that poor schools had access to fewer volunteers than wealthier schools.

The advantages Brent identified concerning school-community relationships and school climate might be relevant to the use of volunteers (parents, students, and community members) in the NCA CASI school improvement process. One could reasonably expect that parents and community members involved in the school through the school improvement process as members of the steering committee, goal committees, or a profile committee would produce results similar to those same individuals volunteering in a classroom. It appears that the "cost" is worth the results.

Previous Article | Next Article | Contents, This Issue | Feedback | JSI Home | NCA Home


All material on this site © 2000-08 NCA Commission on Accreditation and School Improvement unless otherwise noted.
Questions may be directed to the Webmaster (webmaster@ncacasi.org).