Progress Over Time- Part 1- Using Research- How Robust is the Effect?

In this series of blog posts, I will share with you memos that I am issuing to all teaching staff at my school, Wyvern College, to update them on a whole-school ‘Progress Over Time’ teaching & learning development initiative. The teachers at Wyvern are a talented, highly-committed and special group of people. I’m sure you will agree, the work they are producing in exploring how to effectively implement retrieval practice, spacing and interleaving is quite inspirational…

Part 1- Using Research- How Robust is the Effect?

Desirable Difficulties- “Conditions that create difficulty for the learner, slowing the rate of apparent learning, but actually lead to better long-term retention and transfer of knowledge”- R. A. Bjork

Examples of Desirable Difficulties: R………….. Effect, S………….. Effect, I………….. Effect

After the Progress Over Time INSET session last October, a number of colleagues commented to me that the content resonated with their intuitions gained from their many years of experience in the classroom. ‘Rapid and sustained’ progress always felt oxymoronic, they said; in learning as in life, there are no shortcuts. ‘Rapid or sustained’ would be closer to the mark… Progress that is sustained rarely grows from learning techniques which emphasise rapid, superficial acquisition of knowledge or skills.

When learning about Desirable Difficulties myself for the first time, I certainly remember feeling as though these ideas gave me a vocabulary and conceptual framework to articulate what my teacher instincts were telling me; to learn something to meaningful depth and longevity, students need study schedules that look far beyond just delivering them the information just once…

The same is true in our own professional learning, of course. When we identify adaptations to our practice that we want to embed, it takes considerable sustained, conscious effort and deliberate practice to ingrain the change to the point of automaticity.

When Alan asked me to lead the Progress Over Time thrust on behalf of the college, I agreed under the condition that we scheduled the training such that it modelled the ideas it advocated. A one-off INSET session and a few posters around college would be somewhat hypocritical! The twilight INSET had a follow-up delayed quiz, and there’s another one coming later in the year! Furthermore, I insisted on time for colleagues to digest the ideas and reflect upon any implications for their own practice. For Curriculum Leaders, we scheduled follow-up meetings to harvest and share best practice from across the college and also to explore ways in which the leadership of the college could empower and support CLs with follow-up amendments they wanted to make in their own departments. A good number of these meetings have now happened, the rest will be scheduled soon, and it now seems timely to share with all teaching staff a ‘progress update’ on Progress Over Time…

Firstly, I would like to effuse about three things: the tremendous amount of good practice with regards to Progress Over Time already within the college; the professionalism and expertise of CLs in reflecting on the implications of Desirable Difficulties etc within their own subject curricula; and also the open-mindedness of so many staff across the college to think about and act on these ideas. Given the talent, and competence already in the college, along with the considerable thought already having been given to how we plan for and deliver Progress Over Time, I very much see my role as a ‘facilitator and sharer’, rather than just a ‘pollinator’… This series of memos is a means through which to share what I have been privileged to learn from many of you thus far.

In this first memo, I would like to focus on reflections more broadly about applying Desirable Difficulties within 11-16 teaching and what I’ve learned about trying to implement lab-based research in school environments. Future memos will then focus on updates about practical Wyvern examples regarding implications for individual teacher practice, social factors involved in implementing Desirable Difficulties, curriculum design implications and also updates from the Y9 Learning Leaders project.

One of the most important questions I have learned that you should ask anyone advocating ideas from educational research for use in classroom teaching is, “how robust is the effect and what evidence do you have with regards to this?” You see, much of the research out there was conducted with undergraduate students in laboratory-based environments. Undergraduates have, by their very position, demonstrated they are effective learners and they don’t represent the spread of academic attainment that we have within our school populations. They are also typically developmentally more mature.

In the vast majority of laboratory-based studies, the experiments are specifically designed to maximise the impact of the phenomena they are trying to observe. Any factors that could interfere with or reduce the impact size of the results are eliminated from the experiment by design.

If only we were in such a privileged position! The takeaway point is that many of the effects that are observed in the research-world are ephemeral and/or ethereal, fragile things that can be hard to replicate in a reliable, predictable way in real-world classrooms. Perhaps there is no better example in recent times than Dweck’s Growth Mindset…

To my knowledge, there has not been a single longitudinal study run in a school environment which has shown an implementation of Growth Mindset which has led to better student academic outcomes. Furthermore, according to some sources, the theory itself is in somewhat of a ‘replication crisis’ in academia. Allegedly, few research teams outside of Dweck’s own lab can replicate the findings, even under laboratory conditions.

The point, I believe, is that whilst people don’t doubt that Growth Mindset is real, at this stage it is too fragile, perhaps too little understood, to be a finding that we are able to implement in real-world 11-16 classrooms in reliable ways with predictable impact. Martin K and myself learned this the hard way in our cluster research study a couple of years ago!

Replicability and predictability under a broad range of contexts (robustness) is the first criterion one should have in mind before we invest time and energy into bringing any research finding into our classrooms. Show us the evidence…!

There is still much work to be done with Desirable Difficulties, but one of the reasons I feel confident in making it the foundation for our whole-school Progress Over Time thrust is precisely the replicability and predictability of the effects. The academic research into spacing, interleaving and retrieval is decades-old and well-understood. As Nick Soderstrom PhD, a researcher and graduate from the Bjork lab recently said in a conversation with me, “it’s very hard to design a study where you don’t observe the retrieval, spacing or interleaving effects.” These effects have also been shown not just in knowledge domains, but also physical skill domains too. Also, whilst I think there is a long way to go to build a research base of using these effects to their maximum in real-world 11-16 classrooms, researchers such as D Rohrer, J Karpicke, H Roediger, M McDaniel, S Carpenter and more have all published school-based studies showing impact.

Having set out these thoughts, I would like to finish this first memo by acknowledging from my work with CLs so far just how diverse the knowledge structures are across our subjects. It is remarkable to learn about how different the contexts and constraints are within which we all work. Skills-focus versus knowledge-focus, practical versus abstract, project-based versus cumulative knowledge-based, our subjects are certainly very different! I am delighted to share that in our Progress Over Time work so far we have been able to find some strategies and ideas that are useful across this great divide!

Next, we’ll take a look at what we’ve realised so far…

Desirable Difficulties- “Conditions that create difficulty for the learner, slowing the rate of apparent learning, but actually lead to better long-term retention and transfer of knowledge”- R. A. Bjork

Examples of Desirable Difficulties: Retrieval Effect, Spacing Effect, Interleaving Effect

first seen http://www.greatmathsteachingideas.com