In the last four posts, I have written articles that call out the National Council on Teacher Quality on the basis of their built-in bias against teacher preparation institutions, and their confrontational style of what they call research.
In the last post I used research by Michael Carolan, who investigated when does science become junk. In one study, drawing from the Lexis-Nexis database, ten years (1995–2005) of newspaper articles containing the term ‘junk science’ in the headline were systematically analysed and coded according to how the term “junk science” was used. From this analysis and coding, 11 definitions of junk science were identified. Michael Carolan’s research is significant here because it offers a method to find how we can distinguish between junk science and science reports, especially if we think the report is published by non-refereed sources. It’s vital to be able to answer this question, especially in the present age where the media gives equal time to every idea regardless of the idea’s validity.
To rate the NCTQ study, I used the 11 definitions as criteria to decide whether the study fits the definitions of junk science, or whether it might be considered a valid scientific investigation. In future posts, I’ll report evaluations of other studies, especially two studies on science standards by the Fordham Institute.
Using the criteria listed in Figure 1, I examined the NCTQ Report on Teacher Prep. For the criteria, the goal was find out if that particular definition could be applied to the NCTQ report. If it did, then the criteria was marked YES. To support the decision, examples from the NCTQ report are included in the last column of Figure 1.
| Junk Science Definitions | Junk Science? | Score | Comments |
| Bad policy based upon | Yes | 1 | Assumes American education is declining; Believes the marketplace is the engine for change |
| Experts with agendas | Yes | 1 | Assume that one of causes of America’s decline is teacher preparation institutions. NCTQ wants teacher education to be a training ground not an environment of learning |
| False data | No | 0 | Data reported are ratings, which are dubious. We could rate this “yes” |
| No data or unsubstantiated claims | Yes | 1 | Data unclear; claim teacher education is “chaotic” with no data; |
| Failure to cite references | Yes | 1 | No review of the literature; cherry picked three references; |
| Using non-certified experts | Yes | 1 | There is no evidence that experts in the field of teacher education participated |
| Poor methodology | Yes | 1 | Data was limited to college bulletins and syllabi. Very few syllabi received to make decisions. |
| Too much uncertainty to arrive at conclusions drawn | Yes | 1 | Universities did not cooperate thus creating great uncertainty in any information gathered by NCTQ |
| Revealing only that data which supports findings | Yes | 1 | The data is in the form of ratings |
| Non-peer reviewed claims | Yes | 1 | Not subjected to review by experts in the field of teacher education |
| Totals | 10 yes; 1 no | 10 | The NCTQ report on Teacher Prep scored 10 on a scale of 0 -11. The report is clearly junk science |
Caution
The NCTQ study scored 10 out of 11 on the Junk Science scale, and as result earned four “CAUTIONS, as indicated in Figure 2. To earn 3 or 4 cautions is clear evidence based on the JS Score Method, that the study under consideration is pure junk. The results should be considered fraudulent, invalid, unreliable, and pure ideology.
Figure 2. Junk Science Rating System Using JS (Junk Science) Score. The higher the score, the more caution should be exercised using the results and conclusions. Hit the delete button or drag the file to the waste basket if the study has a rating of more than 2 Cautions.
I invite you rate the NCTQ report on teacher prep using the JS Score Method, and then compare it to a study done by Linda Darling-Hammond, Powerful Teacher Education, Lessons from Exemplary Programs. The NCTQ study compared to the Darling-Hammond study is a classic example of junk science compared to science.
Do you think NCTQ study is junk science, or is it a credible study of teacher prep?
0 Comments