Sunday, January 12, 2014

Stop the press! Don't believe the hype-steria! Or, Let's think logically about PISA and math.

There has been much talk about education since the release of the PISA 2013 results in December of last year. In Canada, instead of celebrating our successes, there has been a lot of concern over our “dropping” in the rankings. The same consternation has been occurring in Finland, where no longer is Suomi number one (except in junior hockey).

It is with trepidation that I read that Canada’s ranking in the PISA 2012 had dropped as compared to its placing in previous rounds of the test (2009, 2006, 2003, 200). I worried that people would look at our drop in the rankings and think that we need to, as Nhung Tran-Davies writes in a Globe and Mail opinion piece, “stem the systematic breakdown of our children’s education.”

Tran-Davies opens her piece with:
Although I am neither a teacher nor a mathematician, as a mother of three young children who are entering the primary education system, I am calling our ministry of education in Alberta to task with a petition to urgently stem the systematic breakdown of our children’s education.
Personally, from someone in the system who thinks about this all the time, Canada is doing amazingly well. We are a geographically enormous nation, with a diverse set of learners due to our multicultural population, while being a country without a centralized education system, and yet we are still among the top performers in the world. Historically, PISA 2012 has us lower than previous PISA tests, but we are not in free fall. We need to understand how the numbers (and thus rankings) are calculated before we call for massive reforms in the way that we have been teaching and how we quantify, according to this one measure, success. This is why headlines like “B.C.’s educational reforms are running into resistance“ from The Globe and Mail on January 9, 2013 are concerning to me. In it, reporter Andrea Woo references two people and their concerns as it relates directly to math. 
But critics say the proposed curriculum is filled with “edu-speak” and stress the importance of basic, common-sense approaches. 
Tara Houle, a mother of two in North Saanich, says she has seen firsthand the confusion so-called discovery-based teaching techniques can lead to.

Malgorzata Dubiel, a senior lecturer at Simon Fraser University’s Department of Mathematics, said creativity in teaching is valuable, but must not come at the expense of basic math skills, clear instructions and practice. As well, curriculum and teaching at one grade level must feed into those of the next level.(
I don’t discount either Houle’s nor  Dubiel’s experience, nor their concerns, but nowhere does it say in BC curriculum that we should be abandoning “the basics.” And the most concerning aspect of that article, and many others like it, is that it is again referencing opinions of people outside the system. Nowhere else in the article is there any reference to widespread resistance to the new draft curriculum changes in BC. In this particular instance, there are two people who are concerned about math. I hope that there every parent is involved in their child’s education, asking questions and, above all, finding out what works best for their child, because, unlike some math questions, there is no one answer for how every child will learn.

I also fear that the hysteria that we are seeing, in particular with this supposed math crisis, is more about opinion-based click-baiting headlines than actual research-based analysis. With more blog-style news sites competing for page views with older news publications, it seems that anything that has to do with concern over the well-being of our youth will drive click-throughs. I am not here to say that our educational system is perfect, but the hysteria over our results in a test that is, for simplicity’s sake, bell-curved, is very concerning to me as a teacher. Our strongest allies in education are our parents. If our parents perceive that the education system is in disarray because if disingenuous online articles, then that perception drives political agendas. The perception that our education system is in crisis is not true nor good. Of course we should always be looking to improve. Of course we should always want better for our children, regardless of how well we are doing. Of course I would ask for better working conditions for teachers in BC, but that is only relative to our previous working conditions in this province and to the current working conditions of colleagues in Canada, not those of colleagues around the world, because, frankly, in that comparison, we are doing very well. As is our education system. 

There have been some great articles written by people I believe to be global thinkers (both in worldy-terms and in big-picture terms). Both have provided a rebuttal to the ideas that many commentators in media have been projecting as truth (and who, by and large, are outside the system, who are writing for page-views because we do need to keep funding all forms of thoughtful journalism). The aspect of PISA that I feel most people are ignorant of is summed up beautifully by Pasi Sahlberg: 
First, because PISA is a standardized test, its data are analyzed so that the average performance score each year in mathematics, science and reading literacy is about 500 with a standard deviation of 100. The statistical nature of PISA test means, in practice, that if your 15-year-olds in Sweden maintain their level of knowledge and skills in these subjects compared to the past but others improve their learning (or teachers’ and students’ skills of PISA-type test-taking), your position in rankings will, as a result, decline. In the 2012 PISA study, the Asian countries did better than before, which made it harder for the others to look better compared to the average. In this way, PISA is like a marathon run: Your time of two hours and 20 minutes might make you one of the fastest five today, but you would drop from the top ten five years from now.
And in reference to Finland’s decline, Sahlberg write this, which can also be applied to Canada’s “decline”:
Part of the stagnation and decline of educational performance measured in PISA can be explained by the nature of the metric of this assessment. A quick look at the new PISA results immediately reveals the notion that Asian countries – they now hold all top positions in mathematics, reading and science – have improved their performance from the previous study. Shanghai’s 613 points in mathematics is higher than any ‘country’ has ever scored in PISA. It is almost 100 points higher than Finland’s result that leaves Finnish students two school years behind their Chinese peers. In other words, when others do better it makes you look worse. We should keep in mind that many, if not all, of the Asian education systems get their stellar results in PISA with high social and financial cost for young people and their parents. American, Swedish and English schools are actually more efficient than Korean or Chinese schools because students there get higher PISA scores per time studying than do the Asian students.
I wanted to share this because it is important for us to realize that it is possible for Canadian kids to be doing just as well this year as in years past! Even though we may have stayed equal,  our mean score and ranking could have dropped because the PISA average has gone up (due to the higher scoring countries scoring even higher). As an example, the top score in PISA 2013 was Shanghai-China with 613 and the lowest was Peru at 368. In 2009, it was Shanghai-China with 600 and Kyrgyzstan with 331. In 2006, it was Taiwan at 549 and Kyrgyzstan at 311.

Further to this, Chris Kennedy, the superintendent of the West Vancouver School District in British Columbia, shares his perspective on the PISA results as it relates to BC in particular:
So, while acknowledging the limits of using the nation “rankings”,  let me share some of the insights I have gleaned from my first look at the results and some stories you may have not seen:
1)  British Columbia was the highest performing English-speaking jurisdiction in the world
British Columbia is not only the highest performing province in Canada, but ahead of all other English-speaking participating nations including Australia, United States, United Kingdom, and New Zealand (to name a few).  If you look at countries in general, Canada would be first in this category.  
2)  British Columbia was the highest performing multicultural jurisdiction in the world 
One characteristic that other countries at the top of the charts do not share with British Columbia and Canada is its diversity.  In language and cultural diversity, BC and Canada stand out as the highest performing on the assessments.
3)  British Columbia was the highest-performing province in Canada in science and reading and second to Quebec in Math
British Columbia has typically been among the strongest performing provinces in each area (typically with Alberta, Ontario and Quebec).  The most recent results show BC was first in science, ahead of Alberta and Ontario.  Reading on, the same three provinces performed at the top level in Canada, and again, all near the top of the International charts. In math, Quebec led the way with BC, Alberta and Ontario following.  It is worth noting, of those who completed the digital math assessment, BC was the highest performing province (more on digital below).
4)  There was both excellence and equity in British Columbia’s results
The difference between the high and low achievers in BC (those between the 90th and 10th percentile) is lower than in all of Canada, and the OECD, in all three disciplines. The gap is also lower than that in Finland (often cited for its high level of achievement and equity) in both Reading and Science.
 5)  British Columbia’s results have been steady for the last decade
In absolute terms, since 2006, British Columbia’s results have been fairly steady. It is true that in Mathematics in particular, in relative terms BC (and all of Canada) has declined — in part due to more countries participating, and also because of the improvements in several Asian countries.
My take away is this: please don’t over-react to over-zealous headline writers trying to make sure that they keep their website at the top of their own rankings. It is their job to make parents look; it is a parent’s job to make sure they read. And 15 year old Canadians are still doing quite well in that.

Post a Comment