|Douglas J. Lyons |
Measuring What We Value: Assessing School Effectiveness
(published in the 2012-2013 Independent School Guide; Moffly Media)
At the 2007 Annual
Conference hosted by NAIS (the National Association of Independent Schools),
Jim Collins, author of the best-selling book Good to Great, issued a
challenge to the gathering of 3,000 independent school educators. Citing wisdom
derived from his analysis of over 1,400 businesses, Collins suggested that
independent schools replicate a practice found in the world’s best-run
companies. These are all companies that are deliberate and intentional in
defining what they value most in their products or services— and they are
adept at measuring what they value.
We are living in a
new age; it is an era of amazing opportunities made possible by technology, but
also a time of anxiety about the future. The current generation of parents who
are considering an independent school education for their children think and
decide like consumers. They are interested in acquiring data on the benefits
and outcomes (the “Value-Added”) of the independent school experience.
Responding to these applicant families is a new challenge but one that is
welcomed by the independent school community.
Any assessment of
school quality in the twenty-first century should consider data that is derived from three sources:
- DIRECT EVIDENCE Direct
evidence in education refers to student achievement that is measurable
and able to be referenced or benchmarked across a larger, perhaps
national sample of students. Standardized tests serve this purpose.
Although many independent schools use standardized tests, generally for
ad-missions and/or faculty planning purposes, the tests represent only
one of multiple measures of student achievement and are considered
merely baseline information.
Despite their increasing use in
public schools, mandated by the No Child Left Behind legislation, there
are serious limitations to the value of these tests. Scantron bubble
answer sheets graded by machines are capable of assessing performance in
essentially low-level cognitive skills. Although performance on these
tests is highly predictive of success in similar school-based tasks, it
is of questionable value in predicting success outside of a scholastic
environment. These tests do not measure twenty-first century skills
(analytical thinking, creativity, communication and collaboration).
- INDIRECT EVIDENCE
The Obama administration has awarded $370 million to a small group of
test design companies. The expectation is that, beginning in 2014,
there will be a new generation of standardized tests; tests (scored in
part by humans!) that will be able to assess authentic and important
capacities. Until these tests are available, indirect evidence will
continue to be the more powerful data source.
evidence operates under the assumption that if certain student
behaviors, institutional practices and statistics exist, a high degree
of learning and a high degree of student engagement are predictable.
This data has proven to be reliable. For example, although parents often
read and carefully review school curriculum guides that describe the
content of courses and programs, it is far more valuable to know how
students will be required to demonstrate learning. Two students may take
the same course in two different schools, but be held to very different
The following questions are appropriate to ask:
How often are students required to write? What types of writing are
assigned? Do students confer with their teachers during the drafting
stage? How do teachers respond to writing—do students simply receive a
grade or do they receive a full critical review?
» How (and how often) are oral communication skills assessed? What
types of communication tasks are required (debates, individual and
group presentations to the class, public speaking recitations or
competitions, interviews conducted outside the school, theatrical
often and in what ways are students required to complete tasks in
collaboration with other students? How do teachers manage and supervise
often are students engaged in activities that enable them to identify a
personal interest or a real-world need and design a project that
represents a potentially valuable contribution to others?
» How is creativity fostered and/or celebrated?
Other examples of indirect evidence include:
The percentage of students who participate in extra-curricular
activities (a high percentage suggests an “all-hands-on-deck” school—a
place of engaged, attached, motivated students, learning valuable social
and leadership skills)
» Student retention rates (omitting geographical relocations and financial hardship losses)
The percentage of families participating in Annual Giving (a high
percentage indicates a high level of confidence and gratitude)
“Occasional Teacher Absentee” statistics. This statistic measures
short-term public school teacher absence rates and is publicly reported
by most states, including the State of Connecticut. A high rate (which I
would define as anything over 5 days, on average) should be a concern
to parents and school leaders. Independent schools are known for very
low rates of OTA and are not required to report their data.
- APPLIED EVIDENCE Applied evidence gauges the impact of the school experience “after the fact”.
my 42-year career as teacher, coach, superintendent of schools and
independent school head, I have derived more broadly meaningful
information from detailed alumni surveys than from any other source.
Alumni, especially graduates from earlier decades, can speak of school
influence, years after the experience, when institutional effects have
had time to fully develop.
parents should ask about the results of alumni and or parent exit
surveys. Choosing a child’s precollegiate school or schools is a
time-consuming task. It is also one of the most consequential decisions
parents make in the care of their children. The early school experience
establishes attitudes and habits of thinking. It is rare that a student
discovers a love of learning after she has been admitted to Yale.