A Response to Some of the Points of: When Academic Disagreement Becomes Harassment and Persecution

Wayne Bishop[1] (with R. James Milgram[2])

An error filled complaint was recently posted on the Stanford web-site of Professor of Mathematics Education Jo Boaler, [B1]. It begins

"Honest academic debate lies at the core of good scholarship. But what happens when, under the guise of academic freedom, people distort the truth in order to promote their position and discredit someone's evidence? I have suffered serious intellectual persecution for a number of years and decided it is now time to reveal the details."

The irony of this claim of violation of honest academic debate - absolutely essential to academia - is overwhelming. Herein are addressed some of the more obvious points.

The paper she finds to be inaccurate and an inappropriate violation of academic freedom is available at R. James Milgram’s website, [BCM]. It is entitled A close examination of Jo Boaler's Railside Report, by W. Bishop, P. Clopton, R.J. Milgram.

Paul Clopton is a professional statistician with statistics degree from Stanford. R. James Milgram is a professor of mathematics at Stanford. Wayne Bishop (hereafter, I) is a professor of mathematics at California State University, LA.

When Prof. Boaler decided to leave Stanford for her native England, the authors elected not to publish [BCM] (in spite of peer review approval and acceptance by a highly reputable education journal) partly to spare her and Stanford unnecessary embarrassment, partly for other reasons.

Jo Boaler, in her complaint, [B1], goes on to introduce herself as follows:

"I am a Stanfod University professor and researcher of mathematics education. My research focuses on the most effective learning environments for students learning mathematics and has won awards in both England and the United States.""

My contention is that Boaler's "most effective learning environments" are not the environments that I consider to be most effective, nor that careful, data-based research has supported. However, that is not the point of this response nor [BCM].

She then says

"The particular area of my research that Milgram and Bishop have tried to discredit is focused upon equity and the ways that the mathematics achievement of all students in the US may be raised."

This conviction of Prof. Boaler is completely misguided in at least two ways. For one, we have not tried to discredit her research in any special way. When I have seen data-based examples of mathematics education that appear to be incredible, I have tried to pursue the data to their original sources to see if my interpretation of the larger situation agrees with the conclusions of the authors.

I have done this a number of times; i.e, there is nothing special about Prof. Boaler's work in this regard. For another, we do not believe that there are essential gender and/or racial differences in mathematics learning styles that impact educational achievement equity. Would it be great to have some identified subpopulations perform dramatically better than they currently collectively do? Of course. Motherhood and apple pie.

Boaler then proceeds to make the incredible claim that

"Bishop has used explicitly racist language when discussing issues of equity, claiming that teachers and other 'experts' believe that 'little pickaninnies just don't learn math like we do.'[PG]. His accusations towards educators are offensive and serve as important background to the attacks upon my research in which he and Milgram have engaged."

Although this is true, this is not language that I use in public or in private except in jest or, as in this case, in derision. Of course, this quote is taken completely out of context. The full quote is

"Too many teachers, ACSA (American Council of School Administrators?) and `experts' with the standard 'developmentally appropriate' form of racism. You know, `little pickaninnies just don't learn math like we do'."

and inasmuch as Boaler is a professional researcher I would expect her to be aware of this.

She chose to post this 2002 article in regard to my being commissioned to examine the mathematics pedagogy and curriculum of the Pittsburgh Public Schools (along with three others) that came out on a Monday but completely ignored the scathing indictment of that perspective by syndicated columnist Tony Norman that came out in the same Pittsburgh newspaper the following day (appropriately chastising my naivety, not my racism)[5]: She then continues in her third bullet:

"Between 1999 and 2003 Bishop posted on mathematics education websites that I had invented the schools in my studies. He asserted that 'The schools exist only in her mind'."

I do not recall making this rather harsh assessment but it does sound like me, so is probably correct. I did try to identify the schools in question and found out that some other skeptics, with much better connections that mine, had also tried but had failed. In that book, [B3], she compared two schools with the pseudonyms Amber Hill and Phoenix Park --

  1. where the latter illustrated her preferred methods, popular in mathematics education circles,
  2. and the former illustrated "traditional methods."

It took little extrapolation beyond the words to conclude that Phoenix Park -- the equivalent of her favorite school, "Railside," in her new work, [BS], and following much the same teaching and curricular model -- was likely to have failed. As was the case at Railside, it was likely that it would have also been forced to abandon her ideas of mathematics educational superiority and return to a more conventional approach, or perhaps close. (It turns out that the last is what happened to her English school.)

Boaler's fourth bullet item in her article is

"In 2003 Bishop discussed Schools of Education in the US and suggested to readers that they 'nuke 'em all dammit.' This, alongside his personal attacks on my work, prompted Stanford police department to travel to LA to speak to Wayne Bishop."

There is some truth in these assertions -- the quote is accurate -- but the context is completely suppressed as usual. As to the claim "travel to LA," what actually happened was that the head of the Stanford police department placed a single call to me from Stanford to clarify what was described to me as a frantic report that she had made during their off-hours (evening or weekend, maybe both; I have forgotten).

I assume the Stanford police logs would confirm that. In spite of the seriousness of terrorism threats, we shared a good laugh over the source of my exaggeration, something said in a high-profile panel discussion a few month earlier by Reid Lyon -- who is one of the best reading researchers in the world and who was, at the time, second in command to federal Secretary of Education Rod Page. That panel discussion was in fact chaired by Secretary Page, and on that occasion, Lyon was quoted as saying

"You know, if there was any piece of legislation that I could pass, it would be to blow up the colleges of education.' [3]

Boaler's fifth bullet is

"In 2005 preliminary results of my NSF research were published, again showing that students who are more actively involved in mathematics achieve at higher levels."

The "again" refers to her earlier work reported in [B3]. Her preliminary results for Railside received widespread interest including a standing room only invited address to the NCTM annual conference in Anaheim, CA, and an article in Education Week where I was invited to offer my thoughts on her work, [4]:

About all I could do was offer my skepticism:

"... said such disparities on the different exams may be cause for skepticism. He also said the results of the study could be meaningless, given that the schools were unidentified, leaving independent researchers with no means to cross-check the testing data."

However, I did suggest to the Ed Week article's author, Debbie Viadero, that she ask Prof. Boaler to give her the names of the actual schools so that she could confirm the school data herself, albeit confidentially. Debbie said that she already had asked and had been refused.

In my usual gentle manner, I repeated a suggestion that I have often given to education writers: If there are no names and contact information for schools, there should be no article. There was an article, of course, and in the widely read Education Week.

Up to that point, I did not know that Boaler had a website where she posted her current work but, seeing the reports that were getting so much attention, I vowed to see what I could do, as I was convinced that while she was not lying about her posted data, only that -- perhaps accidently -- her careful selection of which part of the data to present was causing it to be misleading. To that end, on February 19, I emailed Prof. Boaler requesting the names of the schools so that I could check the state's assessments myself, but I got no response. A week later, February 25, I sent a reminder and received a scathing - even unprofessional - refusal:

"I have documented the different lies and insults you have written about me and I have decided not to engage you in discussions of my study as you are not somebody who is interested to learn about the results, only to dismiss it in any way you can.

I have absolutely NO choice about not naming the schools, the anonymity of schools in studies is an ethical standard in educational research that is not mine to dispute."

We will see later that, in fact, she was doing almost exactly the reverse of what federal law, (FERPA), which only applies to student identifiers and records, the main federal regulations on human subjects, [HHS], and Stanford's faculty requirements for openness in research, [Handbook], required of her.

We return to our article, [BCM]. As we had assumed, she was not lying about her carefully-selected data, so it did not take long for the three of us to uniquely identify each of her three schools. What we did was compare her posted data with the entire state database. We had more than a little suspicion about two of the three, (as she knew full well), but the most important one, Railside, was a complete surprise.

This search turned out to be particularly serendipitous for me personally. Prof. Boaler possibly perceived me as spending lots of time and energy trying to destroy the credibility of her widely-accepted research. So, for the second time, she had tried to involve Stanford in discrediting me; this time, much more seriously. She had persuaded the head of the Stanford legal staff, instead of the campus police, to write a letter to the head of the legal staff at my university listing accusations in regard to my "unprofessional activities" i.e., her "documented different lies and insults you have written about me." Our head of legal had dutifully passed the complaint on to my campus president who, in turn, passed it along to our campus ethics committee for a recommendation as to whether or not my actions merited some kind of censure. After I appeared for the first meeting before the committee, where I perceived a definite chill toward me in support of the accusations (that I still had never seen), I requested and was given a copy of those accusations.

Very shortly after that meeting, the three of us involved in [BCM] had the names of the schools positively identified and, as we suspected, reality was very different from the perception -- both that of the highest socioeconomic school, "Greendale," and the lowest, "Railside."

The state has a global measure of a school’s academic performance, the API, (Academic Performance Index). At the time, it was a measure of only mathematics and English language competence although I believe more is now used. The year 2003 was her primary year, the full data-set for Railside at that time was extremely interesting as their performance had steadily declined so that Railside had become a 1-1 school with the first 1 indicating the lowest decile among all California high schools and the second 1 indicating the lowest decile among "comparable schools":
YearRailside Hilltop Greendale
1999 3-66-79-5
2000 3-2 6-7 9-4
2001 2-1 7-5 8-2
2002 2-3 7-8 8-1
2003 * 6-3 9-3
2004 1-1 7-4 8-4

The "Railside" pattern is obvious even with the 2003 data missing because the school appears to have tried the age-old ruse of not testing enough lowest performing students that year to try to improve its average performance. To defend against that, a sufficient percentage of the schools' students must be tested or the state withholds assessment.

The 2004 year was even worse than that indicated above. Searching the database after restricting to 1-1 schools, it was not quite the absolute bottom but almost. The few schools with worse API performance were all very specialized and understandable - a special school for the deaf and 5 or 6 continuation schools. In other words, it could be argued that by the API measure, "Railside" was the worst performing all-purpose high school in the entire state of California. Algebra performance? These were measured by the CST, (California Standards Tests), and this is the 2004 grade-by-grade performance that is recorded as percent of the students who took the exam who fall into one of five categories, Advanced, Proficient, Basic, Below Basic, and Far Below Basic:
CourseNumber Adv ProfBasicBelowFar Below
Algebra 1 22901033 488
Geometry15412 27 56 14
Algebra 2 57 001837 46
Algebra 28204 17 36 41
This records the distribution of the scores for the ninth graders in Algebra 1 in 2004, tenth graders in Geometry in 2004, the eleventh graders in Algebra 2, while the fourth line records the combined scores for the ninth graders that took the Algebra 2 exam in 2001, the 10th graders that took it in 2002, and the 11th graders that took it in 2003. Since the entering class in 2000-2001 consisted of about 390 students we see that 21%, or roughly 1/5th of the class of 2004 ever took the Algebra 2 exam. Since the classes at Railside were somewhat non-standard with each year apparently involving some algebra and some geometry, this would tend to indicate that the likely subgroup of the students taking the Algebra 2 exam thought of algebra as their strongest subject the year they took that exam, and this, in total comprised one fifth of the class.

Of this group, slightly more than three-quarters were Below Basic and more than half of those Far Below. At a minimum, Proficient in Algebra 2 is needed as a prerequisite for an adequate pre-calculus course, and only 4% of the 1/5 of the class -- less than 1% of the 12th graders at Railside who had been in the first year course there in 2000-2001 -- tested ready for pre-calculus much less calculus.

Thus it should come as no surprise that over 60% of the Railside students entering the California State University system in 2004 had to take remedial mathematics.

Yet, in [BS] we find that over 40% of the students at Railside had taken calculus or pre-calculus before graduating. Actually, in the original pre-print, [BS], we find, as quoted by Debra Viadero, Education Weekly (where I became involved): "41 percent of the Railside students had taken calculus by the end of 12th grade."[V] Further, Boaler presented a paper at the 2004 ICME in Copenhagen, [B2], where she gave a verbatim quote from ([BS], page 5), but from the Introduction on page 1 she says

"The study has produced a number of interesting findings, the most significant being the incredible success of one of the schools. At Railside school ...... and to consider the reasons they enabled many more students to be successful than is typically the case in urban American high schools."

Summarizing, perhaps we could say, referring to the Railside students that "They don't have to hurry quite as much, just enroll in a calculus course even though demonstrably not ready to do so."

In any case,

I prepared a point by point response to her charges and submitted it to the committee for their consideration prior to our next meeting. Our next meeting was never held. The committee sent a recommendation to the president recommending no censure was warranted and the president sent me verification of that and assurance that no reference would be added to my permanent file.

Boaler continues [B1] with her eighth bullet:

"Milgram and Bishop attempted aggressively to identify my research subjects -- schools and students that had been promised confidentiality for their protection, consistent with fundamental research study principles."

She continues this line of argument with her tenth bullet

"Milgram and Bishop's 'paper' contravenes federal law that protects the human subjects of research as it identifies schools, teachers and students. Its identification of individual students breaches the Family Educational Rights and Privacy Act (FERPA). The 'paper' has never been peer reviewed, and no journal has accepted it for publication."

This statement is unique in that each claim is incorrect. Perhaps the most interesting misconception is about what FERPA says. In fact, FERPA is strictly concerned with the privacy of student identifiers and student records. Since we entirely expected that the student identifiers and individual records would be redacted from any material sent to us, there is no FERPA issue. Additionally, the main federal rules on human experimentation are contained in [HHS], particularly § 46.101, and they do provide a considerable number of restrictions. But paragraph (b) of § 46.101 specifically excludes studies like [BS] from the [HHS] protections. Likewise, many if not most U.S. universities' policies on openness of research -- in particular Stanford's -- preclude hiding data that does not identify specific human treatment subjects. Even teachers involved in helping with the data collection, and the schools where the data originated, have no expectation of privacy.

On the other hand, even if Boaler were correct, she would be the guilty party since it was the data in the 2005 preprint of [BS] that allowed anyone to positively identify the schools. Thus she was, perhaps inadvertently, obeying the letter of the law if not it's spirit.

With the above as preliminary information, we can now look at the rest of Boaler's eighth bullet in [B1]:

"Yet Bishop contacted numerous school district officials, including principals, and pressured them to disclose whether they were subjects of my study. Among other tactics, he threatened to take legal action against them. Two of the people concerned contacted Stanford University and sent details of Bishop's communication with them. In letters to Stanford they stated that Bishop had been 'unprofessional, demanding, condescending, dishonest' and 'verbally aggressive'."

I have no recording of what I said but it is not my custom to be unprofessional. Perhaps irritation with non-cooperation was showing in my voice. I did tell those principals who were not cooperative that I would be filing (and would have filed) California’s version of FOIA, as I have done for other schools and other states with considerable success. Public information is supposed to be available to the public. I do have copies of the follow-up letters to those principals reiterating my requests and would be happy to provide them on request.

One statement she has particularly wrong is claiming that I was "pressuring them to disclose whether they were subjects" in her study. At the time of contact, the schools had been positively identified; I was only seeking public data (e.g., AP Calculus grade distributions) that were not available on the state’s API and STAR database, but were public documents. [Shortly after this, we -- the authors of [BCM] -- did file formal FOIA requests to the three schools for their AP data, and we soon had that material in hand, so it is duly recorded in [BCM].]


[BCM] Wayne Bishop, Paul Clopton, R. J. Milgram, A close examination of Jo Boaler's Railside Report, ftp://math.stanford.edu/pub/papers/milgram/combined-evaluations-version3.pdf

[B1] Jo Boaler, "When Academic Disagreement Becomes Harassment and Persecution," www.stanford.edu.~joboaler

[B2] Jo Boaler, "Promoting Equity in Mathematics Classrooms - Important Practices and their impact on student learning", (full version, text of a ‘regular lecture’ given at ICME, 2004, Copenhagen.

[B3] J. Boaler, Experiencing School Mathematics: Teaching styles, sex and setting, Open University Press, Buckingham - Philadelphia, 1997

[BR] D. Briars, L. Resnick, CRESST Technical Report 528, (2000)

[BS] Jo Boaler, Megan Staples, Transforming Students’ Lives through an Equitable Mathematics Approach: The Case of Railside School, pre-print 3/2/2005, and Teachers College Record, 110(3) (2008), 608-645.

[HHS] Subpart A -- Basic HHS Policy for Protection of Human Research Subjects 45 C.F.R. § 46.101 (1991), paragraph (b). www.hhs.gov.ohrp/humansubjects/guidance/45cfr46.html

[Handbook] rph.stanford.edu/2-6.html

[PG] http://old.post-gazette.com/neigh_city/20021021mathcity2p2.asp

[V] "Study: teacher-designed math curriculum is effective," Education Week; 2/16/2005, Vol 24 issue 23, p5 Viadero Article


[1] W.Bishop, Professor of Mathematics, California State University, LA, Email: wbishop@calstatela.edu

[2] Due to a horrendous bicycle accident, Prof. Bishop must use voice recognition software to access his computer. This makes editing his writings extremely difficult. In this instance Prof. Milgram helped with this process.

[3] November 18, 2002 - Forum on Rigorous Evidence - Transcript, P. 84-85 The Key to Progress in Education? Lessons from Medicine, Welfare and Other Fields

[4] http://www.edweek.org/ew/articles/2005/02/16/23math.h24.html?r=684511131

[5] http://old.post-gazette.com/columnists/20021022tony1022p5.asp