Guest Letter by Mr. Ed Johnson, Advocate for Education, Atlanta, GA
Creative Commons “I Come In Peace” by JDevaun is licensed under CC BY-ND 2.0
Ed Johnson wrote a letter in response to the Atlanta Public Schools Equity Audit which was prepared by researchers at Georgia State University to look at differences in the characteristics across schools in the APS district. As you will see in Ed Johnson’s letter, he uses a form of thinking that looks at the APS as a whole, and not as separate schools, and applies the work of W. Edwards Deming, Russell Ackoff, and Peter Bernard to investigate equity in the context of systems thinking.
This is an important letter written by a person who for years has explored how to improve education in the Atlanta Public Schools. It is hoped that the new Atlanta Public Schools superintendent will seek his advice, and in so doing challenge the “turn around” and “urban” mentality that dominates educational reform.
June 26, 2014
Well, of course, Atlanta Public Schools’ equity audit would find differences. Differences always exist. No two of anything are exactly the same. So the discerning question always is, what do differences mean?
In its upfront Executive Summary, the APS Equity Audit Report proclaims:
Equity audits are a relatively new tool for school systems and there are large variations in their thresholds for determining whether or not characteristics are substantially different across schools. Simple percentage difference cutoffs or using standard error calculations to generate confidence intervals of means both avoid complex questions of whether or not differences across schools are practically meaningful. This report finds substantial variations across schools on numerous characteristics, but leaves questions of whether and how to address these differences to the broad group of stakeholders concerned with educational outcomes for the students of APS.
On the one hand, the APS Equity Audit Report responsibly cautions against using “[s]imple percentage difference cutoffs or using standard error calculations to generate confidence intervals of means both [of which] avoid complex questions of whether or not differences across schools are practically meaningful,” and that is fortunate. Such figures are usually presented in business-style financial reports that often prompt reacting to and holding people “accountable” for past performance while typically providing no rational basis for predicting performance and learning into the future.
Equity from the Standpoint of Random Variation v Non-Random Variation
On the other hand, without question, although it “finds substantial variations across schools on numerous characteristics,” the APS Equity Audit Report clearly forgoes addressing “whether or not differences across schools are practically meaningful,” and that is unfortunate.
It other words, the APS Equity Audit Report does not address the very important question of what do differences mean. Do differences with respect to a particular characteristic mean something or mean nothing? To answer the question requires detecting and distinguishing differences that arise from random variation and differences that arise from non-random variation.
Random Variation Means…
Detection of differences that arise from random variation would indicate differences that mean nothing, that are not “practically meaningful.” Such differences would be due to common, ever-present systemic causes, any or all of which may be known, knowable, and unknowable.
Non-Random Variation Means…
On the other hand, detection of differences that arise from non-random variation would indicate differences that mean something, that are “practically meaningful.” In this latter case, for better or worse, such differences would be due to special causes powerful enough to dominate and stand apart from all differences due to common causes. Special causes may occur continually, irregularly, or temporarily and are generally known or knowable.
So, there are differences due to common causes that may be referred to simply as “common cause variation.” And there are differences due to special causes that may be referred to simply as “special cause variation.” Hence, there exist two kinds of variation.
Signals and Noise in the Data
Now, considering any characteristic’s data in the APS Equity Audit Report, can something be done with those data to detect and distinguish the two kinds of variation the data may contain? Asked differently, is there a way to filter the data to separate “signals” the data may contain from the “noise” the data do contain?
To do so is important so as to:
- avoid responding to a signal as if it were noise and
- avoid responding to noise as if it were a signal
To fail at either 1) or 2) is to drive up costs and generate excessive waste, unnecessarily.
Is there a Way to Detect Signals from Noise?
Indeed there is a way to detect and distinguish common cause variation and special cause variation. And it is a way even some elementary school children have learned to use in the process of continually improving their own learning. The way is to make a “process behavior chart” from the data. (The process behavior chart is much like an EKG (electrocardiogram) made to tell a story about the behavior of a patient’s heart.)
Now, from actually having made a process behavior chart for a fair number of characteristics the APS Equity Audit Report covers, none revealed any special cause variation, save a few where Forest Hill Academy was detected to represent a special cause matter, which is to be expected of APS’ alternative school.
The Inexperienced Teacher Category
For example, the process behavior chart in Figure 1 below takes a district-level look at the characteristic “Inexperienced Teacher (Less than 3 years),” in the category “Teacher Experience by Academically Disadvantaged Students” (APS Equity Audit Report, pages 179-183). The APS Equity Audit Report explains this characteristic means the proportion of students’ time spent with a teacher that has less than three years experience, and that the proportion can be expressed as a percent by multiplying by 100, which the process behavior chart in Figure 1 does.
Figure 1: District-level Teacher Experience by Academically Disadvantaged Students, Inexperienced Teacher (Less than 3 years) ©Ed Johnson
The process behavior chart in Figure 1 detects only differences due to common causes, or common cause variation, or noise. All the variation ranges around the center-line average of 26 percent (26.26%) and between the lower control limit, at zero percent (0.00%), and the upper control limit, at 55 percent (54.55%). No variation exceeds the upper control limit. This means academically disadvantaged students that have a teacher with less than three years experience is a systemic matter among all APS elementary schools and not a matter for any individual schools. It would be top administration’s mistake, and abdication of their leadership responsibility, to single out Centennial Place, or Hutchinson, or M. Agnes Jones so as to hold any people there “accountable” as a special matter.
Again, Figure 1 is district-level, with all APS elementary schools taken as a system. But what about APS Region-level, with each Region taken as a system? What might process behavior charts say about how the North, East, South, and West regions of APS compare on the example characteristic being considered here?
Consider Figure 2, below. The figure comprises four process behavior charts, one for the East Region, North Region, South Region, and West Region of APS. Figure 2 makes it easy to compare the APS Regions holistically and rather straightforwardly and much at a glance. Like the district-level process behavior chart in Figure 1, each Region-level process behavior chart in Figure 2 detects no evidence of special cause variation; all differences are due to common cause variation, to noise. No differences are “practically meaningful.”
It is also quite easy to see in Figure 2 that common cause variation appears the least “spread out” around the North Region center-line average compared to the spread of variation around the other Regions’ center-line averages. Even so, if extended to the right, the North Region lower and upper control limits would cover all West Region schools as well as all South Region schools. And if extended to the left, North Region lower and upper control limits would cover all East Region schools, save Centennial Place. Thus Figure 2, like Figure 1, says differences among all APS elementary schools with respect to the example characteristic are systemic, and equitable.
Moreover, it is also quite easy to see from Figure 2 that each APS Region’s center-line average compares favorably to the district-level center-line average in Figure 1. In Figure 1, the district-level center-line average is 26.26%; in Figure 2, the four Regions’ center-lines average to 26.37%. The difference is a mere 0.11%, or roughly one-tenth of one percent.
The observations made from Figure 2 support and now extend the observation made from Figure 1. Now it can be said that academically disadvantaged students that have a teacher with less than three years experience is a systemic matter for all APS elementary schools, and is not a matter for any individual school or APS Region.
Figure 2: Region-level Teacher Experience by Academically Disadvantaged Students, Inexperienced Teacher (Less than 3 years) ©Ed Johnson
Implication for Administrators, Especially Those at the Top
In addition, and much like already concluded, it would be APS top administration’s mistake, and abdication of their leadership responsibility, to single out any school or Region so as to hold any people there “accountable” as a special matter. Leadership from the top, from both the school board and the superintendency, is required. Only they can be held “accountable” in any rational way. And no manner of “accountability” pushed down from the top can substitute for the requisite leadership needed to foster collaboration with and among affected stakeholders, as a system.
Now, let’s be clear on this point: Both Figure 1 and Figure 2 present process behavior charts that evidence only equity; neither evidences inequity.
Where is the Inequity?
So, if inequity exists, then where does it exist?
Well, actually, knowing where the inequity exists comes through the story the process behavior charts in Figures 1 and 2 tell. The charts tell the story that the teacher characteristic “Inexperienced Teacher (Less than 3 years)” has been optimized among APS elementary schools only about that singular teacher characteristic. It is a story with telltale signs of strictly systematic analytical thinking operating to the exclusion of systemic synthetical thinking. It is a story with telltale signs of believing that the whole is the sum of its parts, and that the whole can do its best only if each part does it individual best, that each part “executes with fidelity.” It is a story where teachers that have less than three years experience have been assigned quite equitably throughout APS elementary schools and to academically disadvantaged students.
And that is the rub, the genesis of the inequity, though it may seem counterintuitive.
Standardized test results have for more than a decade shown APS to be, in effect, “two systems in one,” White-Black, with Black greatly lagging. More recently, standardized test results have begun to show APS’ devolution into becoming “three systems in one,” White-Hispanic-Black, with Black still lagging.
Therefore, the inequity comes not from placing less experienced and unremarkable teachers with especially “Black” students in the APS West Region and South Region. Again, the process behavior charts in Figure 1 and Figure 2 say equity exists among all APS elementary schools with respect to the teacher characteristic “Inexperienced Teacher (Less than 3 years).” Rather, the inequity comes from “Black” students being without greatly experienced and remarkable teachers! For example, five-weeks trained personnel by Teach for America placed with “White” students would constitute equity. However, five-weeks trained personnel by Teach for America placed with “Black” students would constitute inequity. Why? Simply because none can possibly be a greatly experienced and remarkable teacher.
Why the Inequity?
Now, why might this inequity exist? What might be its root?
Consider that the Atlanta Board of Education Policy Manual offers understanding. Specifically, Policy Number BBBC, titled “Board Member Development Opportunities,” states, in part:
The Atlanta Board of Education places a high priority on the importance of a planned and continuing program of professional development of its members. … The board considers participation in the following activities consistent with the professional development of its members: Conferences, workshops, conventions, and training and information sessions held by the state and national school boards associations and other conferences sponsored by local, state, and national educational organizations. … The list shall include, but need not be limited to, the following organizations:
- National School Boards Association
- Georgia School Boards Association
- Council of Great City Schools
- National Alliance of Black School Educators
This policy has inequity built-in. How? First, it restricts “professional development” (PD), which goes policy-wise undefined, to school board procedural matters vis-à-vis the school board associations listed. Then it more narrowly restricts PD to thinking and treating APS as an “urban” school district in need of “urban school reform” or “transformation” vis-à-vis Council of Great City Schools and similar other organizations. Then more narrowly still, the policy restricts PD to a “racialist ideology” (Fredrick Douglass) vis-à-vis National Alliance of Black School Educators.
Regressive Policy
The policy is regressive, and acts much like a funnel to direct APS into associations with persons and organizations committed to disrupting public education as a common good or who have not the wisdom to understand and value public education as a common good. The aim is the transformation of public education in especially “urban” school districts into a profit-making, free-market commodity all the while opportunistically and unashamedly co-opting Civil Rights struggles. This inequity built into school board policy and steeped in urbanism effectively keeps APS stuck in stasis and incapable of learning to continually improve, unlike the global community that is continually learning to improve.
A consequence of such inequity rooted in Atlanta Board of Education policy is the thinking that “it takes a black educator to educate a black child” made a prominent operational aspect of APS culture, and with it APS never going beyond urbanism’s boundary to seek greatly experienced and remarkable teachers to place with especially “Black” students! The inequity is such a deep, self-imposed operational aspect of APS culture that it goes virtually unspoken and unchallenged among stakeholders until it becomes convenient to use to insinuate, excoriate, or defend against allegations of maltreatment or oppression, or to conduct an equity audit.
Fortunately or unfortunately – take your pick – wisdom teaches that the problem is in here, with us, not out there, with them. But then, wisdom comes from learning, not from achievement and certainly not from merely performing.
So, isn’t it time for Atlanta Public Schools to leapfrog City of Atlanta’s modern day “Atlanta Compromise” and turn to embracing humanness more so than “race?” Isn’t it clear by now that especially “Black” children’s quality of education depend on doing so?
Kind regards,
Ed Johnson
Advocate for Quality in Public Education
O: (404) 691-9656 | C: (404) 505-8176 | edwjohnson@aol.com
“The foundation of every state is the education of its youth.”
Diogenes of Sinope (c. 412 – c. 323 BCE)
0 Comments