In yesterday’s post, I described Science Debate 2008, and efforts to engage the two major candidates for President to answer 14 important questions about science.  The one question that focused on science education was as follows:

A comparison of 15-year-olds in 30 wealthy nations found that average science scores among U.S. students ranked 17th, while average U.S. math scores ranked 24th.  What role do you think the federal government should play in preparing K-12 students for the science and technology driven 21st Century?

Each candidate discussed this question (you can read each candidates’ response by following this link and then scroll down to question #4).  You can decide for yourself which candidate provided the kind of answer that will improve science education in the nation’s schools.   To give you a flavor for what they said, here are two excerpts from their answers.  One of the candidates said that:

All American citizens need high quality STEM education that inspires them to know more about the world around them, engages them in exploring challenging questions, and involves them in high quality intellectual work. (please note: I added the link to the STEM education coalition)

The other candidate said that:

America’s ability to compete in the global market is dependent on the availability of a skilled workforce.  Less than 20 percent of our undergraduate students obtaining degrees in math or science, and the number of computer science majors have fallen by half over the last eight years.  America must address these trends in education and training if it hopes to compete successfully.  

Note that the question on science education was couched in the language of international assessments, e.g. “A comparison of 15-year-olds in 30 wealthy nations found that average science scores among U.S. students ranked 17th, while average U.S. math scores ranked 24th.”  What does this mean?  Is the sky falling?

No. Large-scale international assessments of student achievement receive a great deal of attention when the results become public.  Normally, the results comparing countries are reported in a fashion similar to standings in professional sports, where 1 is at the top, which is typically Singapore, followed by lower scoring countries, and as suggested in the question, placing the U.S.A. 17th out of 30.  

There is a real problem in using results to compare one country to another.  As some researchers have pointed out, the scores reported are averages for the country of the students who took the test.  Often the the differences between average scores from country to another are not significant, BUT politicians, educators and the public see that if their country is not NUMBER ONE, “the sky is falling.”

And its not just a concern expressed by U.S. politicians.  Svein Sjoberg of the University of Norway reports (in a study–Real Life Challenges: Mission Impossible) that results on the PISA test (Programme for International Student Assessment) of students in Norway provided “war-like headings” in most of Norway’s newspapers.  In fact the commissioner of education of Norway was quoted as saying, “Norway is a school loser, and now it is well documented.” 

So, when U.S. students score 17th on an international test, policy makers make the claim that science education in the U.S. is in free-fall, and needs to uplifted.  Remember, that the score used on these tests is an average.  There are more than 15,000 independent school systems in the U.S. and to use an average score on a science test (typically comprised of 40 – 60 questions) does not describe the qualities or inequalities inherent in the U.S.A.’s schools.  David Berliner (in a research study entitled Our Schools vs. Theirs: Averages That Hide The True Extremes)points out that the TIMSS (Trends in International Mathematics and Science Study) data for the U.S.A., when analyzed by socioeconomic levels, shows great disparities and inequalities.  For example, schools in the most affluent neighborhoods do well on these tests, but schools in poorer neighborhoods do not.  And Berliner points out that scores on international tests will not change unless the inequalities in the schools are fixed.

Results on these international achievement tests, taken out of context, might not be the best way to assess how well science is taught in any country’s schools.

How can school science be assessed that will help us close the gap between schools?  What suggestions do you have?  Do you think that these international test results are valid ways of assessing school science learning?