Tag Archives: literature

Waves on Pinnau in Mölln. By Uwe Glessmer

You learn better when you explain to yourself

I just read a really interesting article on explaining to yourself as a mechanism for learning by Tania Lombrozo. We have talked about peer instruction being valuable because explaining to others helps both the “others” and the explainer, and it’s really common to hear student tutors say that they only understood something really well when they had to explain it to students they were tutoring. In fact, many people I know use putting someone in the position of having to explain something to make themselves (or their students, if they appoint them as tutors) understand better, and studies show it works. But explaining to yourself?

The author describes research on how, why and when explaining leads to new learning. You should go check out the original blog post, too, but here is what I am taking away from it: When you explain, you are looking for general pattern.

The author cites research that shows that explaining to yourself is not the best strategy for all kind of learning outcomes — only for those that are related to the causal effects you were explaining to yourself. For other details, it might be a better strategy to just observe, or describe what you are seeing.

How is this relevant for our teaching? There are several ways.

Explaining to themselves is a strategy we can recommend to our students. I remember studying for my oral examinations at Vordiplom (now equivalent to Bachelor) level. I used to come up with questions and try and answer them late at night when I couldn’t go to sleep (Why is the Atlantic ocean more salty than the Pacific ocean? This kind of stuff). Those were questions that I didn’t know the correct answer of at the time (and some of my questions there might not be an answer) and it definitely helped me when I was then asked what geometry of sound receivers I would use if I were to build an array for SOFAR floats, and it made me feel safer going into the exam, knowing that I had answered all questions that I could come up with previously as well as I could.

And of course you can just tell students that they will have to teach about a topic, since anticipating having to teach already leads to improved learning. Then you can reflect later on how thinking they would have to teach led them to use different learning strategies, and whether they might want to use those in the future even when they were not expecting having to teach.

I even see a similar effect with having a blog. Now, when I take pictures of water somewhere, I observe pretty carefully, anticipating that I will write about what I saw and that someone might ask questions about it. That definitely makes me put a little extra effort into observing and thinking about what might be going on there!

Check out the original blog post on explaining to yourself as a mechanism for learning by Dr. Lombrozo — there is a really nice example in there that I definitely want to use in future workshops to make that exact point. You will enjoy it, too!

Mirjam Glessmer and Timo Lüth leading a workshop for university instructors

You learn better when you think that you will have to teach

Have you ever worked as student tutor? Then you’ve probably felt like you understood the content of the course you tutored a million times better after tutoring it. Or at least that’s what I hear over and over again: People feel like they understood a topic. Then they prepare to teach it, and realise how much more there was to understand and that they actually understood it.

And there is research that shows that you don’t actually need to teach in order to get the deeper understanding, it is enough to anticipate that you will teach: “Expecting to teach enhances learning and organization of knowledge in free recall of text passages” by Nestojko, Bui, Kornell & Bjork (2014).

In that article, two groups of participants are given texts that they are to study. One group is told that they will be tested on the text, the other one that they will have to teach someone else who then will be tested. After all participants study the text, they are then all tested (and nobody gets to teach). But it turns out that even expecting to teach had similar benefits to what we see in student tutors who actually taught: Participants expecting to teach have a better recall of the text they had to study, can answer more questions about it and especially questions regarding main points.

So what does that mean for teaching? As the authors say: “Instilling an expectation to teach […] seems to be a simple, inexpensive intervention with the potential to increase learning efficiency at home and in the classroom.” And we should definitely use that to our advantage! :-)

How to know for sure whether a teaching intervention actually improved things

How do we measure whether teaching interventions really do what they are supposed to be doing? (Spoiler alert: In this post, I won’t actually give a definite answer to that question, I am only talking about a paper I read that I found very helpful, and reflecting on a couple of ideas I am currently pondering. So continue reading, but don’t expect me to answer this question for you! :-))

As I’ve talked about before, we are currently working on a project where undergraduate mathematics and mechanics teaching are linked via online practice problems. Now that we are implementing this project, it would be very nice to have some sort of “proof” of its effectiveness.

My (personal) problem with control group studies
Control group studies are likely the most common way to “scientifically” determine whether a teaching intervention had the desired effect. This has rubbed me the wrong way for some time — if I am so convinced that I am improving things, how can I keep my new and improved course from half of the students that I am working to serve? Could I really live with myself if we, for example, measured that half of the students in the control group dropped out within the first three or four weeks of our undergraduate mathematics course, while of the experimental group, only much fewer students dropped out, and much later in the semester? On the other hand, if our intervention had such a large effect, shouldn’t we be measuring it (at least once) in a classical control group study, so we know for sure what its effect is, in order to convince stakeholders at our and other universities that our intervention should be adopted everywhere? If the intervention really improves this much, everybody should see the most compelling evidence so that everybody starts adopting the intervention, right?

A helpful article
Looking for answers to the questions above, I asked Nicki for help, and she pointed me to a presentation by Nick Tilley (2000), that I found really eye-opening and helpful for framing those questions differently, and starting to find answers. The presentation is about evaluation in a social sciences context, but easily transferable to education research.

In this presentation, Tilley first places the proposed method of “realistic evaluation” in the larger context of philosophy of science. For example Popper (1945) suggests using small-scale interventions to deal with specific problems instead of large interventions that address everything at once, and points to the opportunities to investigate the extent to which the theories (on which those small-scale interventions were built) can be tested and improved. Similarly, Campbell (1999) talks about “reforms as experiments”. So the “realistic evaluation” paradigm has been around for a while, partly in conflict with how we do science “conventionally”.

Reality is too complex for control group studies
Then, Tilley talks about classical methods, specifically control group experiments, and argues that — in contrast to what is portrayed in washing detergent ads, for example — studys are typically too complex to directly transfer results between different contexts. In contrast to what science typically does, we are also not investigating a law of nature, where the goal is to understand a mechanism causing a regularity in a given context. Rather, we are investigating how we can cause a change in a regularity. This means we are asking the question “what works for whom in what circumstances?”. With our intervention, we might be introducing different mechanisms, triggering a change in balance of several mechanisms, and hence change the regularities under investigation (which, btw, is our goal!) — all by changing the context.

The approach for evaluations of interventions should therefore, according to Tilley, be “Context Mechanism Outcome Configurations” (CMOC), which describe the interactions between context, mechanism and outcome. In order to create such a description, one needs to clearly describe the mechanisms (“what is it about a measure which may lead it to have a particular outcome pattern in a given context?”), context (“what conditions are needed for a measure to trigger mechanisms to produce particular outcome patterns?”), outcome pattern (“what are the practical effects produced by causal mechanisms being triggered in a given context?” and this finally leads to CMOCs (“How are changes in regularity (outcomes) produced by measures introduced to modify the context and balance of mechanisms triggered?”).

Impact of CCTV on car crimes — a perfect example for control group studies?
Tilley gives a great example for how this works. Investigating how CCTV affects rates of car crimes seems to be easily measured by a classical control group setup. Just install the cameras and compare their crime rates with those of parking spaces without cameras! However, once you start thinking about mechanisms through which the CCTV cameras could influence crime rates, there are lots of different possible mechanisms. There are eight named explicitly in the presentation, for example offenders could be caught thanks to CCTV and go to jail, hence crime rates would sink. Or, criminals might not choose to commit crimes, because the risk of being caught increased due to CCTV, which would again result in lower crime rates. Or people using the car park might feel more secure in using it and therefore start using it more, making it busier at previously less busy times, making car theft more difficult and risky, leading to sinking crime rates.

But then, we also need to think about context, and how car parks and car park crimes potentially differ. For example, crime rate can be the same whether there are a few very active criminals, or many not as busy ones. So catching the similar number of offenders might have a different effect, depending on context. Or the pattern of usage of car parks might depend on working hours of people working close by. So if the dominant CCTV mechanism would be to increase confidence in usage, this would not really help because the busy hours are dedicated by people’s schedules, not how safe they feel. If this would lead to higher usage, however, more cars being around might mean more car crimes because there are more opportunities, yet still a decreased crime rate per use. Another context would be that thieves might just look for new targets outside of the one car park that is now equipped with CCTV, thereby just displacing the problem elsewhere. And there are a couple more contexts mentioned in the presentation.

Long story short: Even for a relatively simple problem (“how does CCTV affect car crime rate?”), there is a wide range of mechanisms and contexts which will all have some sort of influence. Just investigating one car park with CCTV and a second one without will likely not lead to results that help solve the car crime issue once and for all everywhere. First, theories of what exactly the mechanisms and contexts are for a given situation need to be developed, and then other methods of investigation are needed to figure out what exactly is important in any given situation. Do people leave their purses sitting out visibly in the same way everywhere? How are CCTV cameras positioned relative to the cars being stolen? Are usage pattern the same in two car parks? All of this and more needs to be addressed to sort out which of the context-mechanism theories above might be dominant at any given car park.

Back to mathematics learning and our teaching intervention
Let’s get back to my initial question that, btw, is a lot more complex than the example given in the Tilley-presentation. How can we know whether our teaching intervention is actually improving anything?

Mechanisms at play
First, let’s think about possible mechanisms at play here. “What is it about a measure which may lead it to have a particular outcome pattern in a given context?” Without claiming that this is a comprehensive list, here are a couple of ideas:
a) students might realize that they need mathematics to work on mechanics problems, increasing their motivation to learn mathematics
b) students might have more opportunity to receive feedback than before (because now the feedback is automated), and more feedback might lead to better learning
c) students might appreciate the effort made by the instructors, feel more valued and taken seriously, and therefore be more motivated to put in effort
d) students might prefer the online setting over classical settings and therefore practice more
e) students might have more opportunity to practice because of the flexibility in space and time given by the online setting, leading to more learning
f) students might want to earn the bonus points they receive for working on the practice problems
g) students might find it easier to learn mathematics and mechanics because they are presented in a clearer structure than before

Now contexts. “What conditions are needed for a measure to trigger mechanisms to produce particular outcome patterns?” Are all students and all student difficulties with mathematics the same? (Again, this is just a spontaneous brain storm, this list is nowhere near comprehensive!)
– if students’ motivation to learn mathematics increased because they see that they will need it for other subjects (a), this might lead to them only learning those topics where we manage to convey that they really really need them, and neglecting all the topics that might be equally important but where we, for whatever reasons, just didn’t give as convincing an example
– if students really value feedback this highly (b), this might work really well, or there might be better ways to give personalised feedback
– if students react to feeling more valued by the instructor (c), this might only work for the students who directly experienced a before/after when the intervention was first introduced. As soon as the intervention has become old news, future cohorts won’t show the same reaction any more. It might also only work in a context where students typically don’t feel as valued so that this intervention sticks out
– if students prefer the online setting over classical settings generally (d), or appreciate the flexibility (e), this might work for us while we are one of the few courses offering such an online setting. But once other courses start using similar settings, we might be competing with others, and students might spend less time with us and our practice problems again
– if students mainly work for the bonus points (f), their learning might not be as sustainable as if they were intrinsically motivated. And as soon as there are no more bonus points to be gained, they might stop using any opportunity for practice just for practice’s sake
– providing students a structure (g) might make them depend on it, harming their future learning (see my post on this Teufelskreis).

Outcome pattern
Next, we look at outcome patterns: “what are the practical effects produced by causal mechanisms being triggered in a given context?”. So which of the mechanisms identified above (and possibly others) seem to be at play in our case, and how do they balance each other? For this, we clearly need a different method than “just” measuring the learning gain in an experimental group and compare it to a control group. We need a way to identify the mechanisms at play in our case, and those that are not. We then need to figure out the balance of those mechanisms. Is the increased interest in mathematics more important than students potentially being put off by the online setting? Or is the online setting so appealing that it compensates for the lack of interest in mathematics? Can we show students that we care about them without rolling out new interventions every semester, and will that motivate them to work with us? Do we really need to show the practical application of every tiny piece of mathematics in order for students to want to learn it, or can we make them trust us that we are only teaching what they will need, even if they aren’t yet able to see what they will need it for?

This is where I am currently at. Any ideas of how to proceed?

And finally, we have reached the CMOCs (“How are changes in regularity (outcomes) produced by measures introduced to modify the context and balance of mechanisms triggered?”). Assuming we have identified the outcome patterns, we would need to figure out how to change those outcome patterns, either by changing the context, or by changing the balance of mechanisms being triggered.

After reading this article and applying the concept to my project (and I only read the article today, so my thoughts will hopefully evolve some over the next couple of weeks!), I feel that the control group study that everybody seems to expect from us is not as valid as most people might think. As I said above, I don’t have a good answer yet for what we should do instead. But I found it very eye-opening to think about evaluations in this way and am confident that we will figure it out eventually! Luckily we have only run a small-scale pilot at this point, and there is still some time before we start rolling out the full intervention.

What do you think? How should we proceed?

How to learn most efficiently when participating in a MOOC

How to learn most efficiently when participating in a MOOC? Yes, I’ll admit, that title promises quite a lot. But there is a new article by Yong and Lim (2016) called “Observing the Testing Effect using Coursera Video-Recorded Lectures” that tells us a lot about how (not) to learn. We have talked about the testing effect before: repeated testing leads to better results on examinations that repeated studying does. And it is confirmed again in this study.

Why am I so excited about this? Because both video-based studying and testing are becoming more and more common these days, and both are sometimes made out to be really bad ideas.

We find video-based learning in most aspects of our lives now (at least if we are talking about lives similar to mine ;-)) — I always follow one or two Coursera courses at the time, and I love watching TED talks. Most softwares I use have video tutorials, and in fact I talked about how I liked the video tutorials of the Monash simple climate model interface only on Tuesday. And whenever I get stuck with a task, I watch video tutorials on youtube to get me going again. And of course many of the lectures at my university are being recorded and many students rely on re-watching them when studying for exams. And, of course, there is the One Planet — One Ocean MOOC that I am involved in preparing. So obviously I see value in video lectures. Even though many people believe that re-watching a lecture does not provide the same experience as seeing it “live”, I don’t think that matters much for lectures in which there is not a lot interaction between lecturer and audience. If you can make yourself use them wisely, I think video lectures are a great substitute for lectures you — for whatever reasons — can’t watch live.

But this is also the biggest issue I have with video lectures: they can easily seduce us into believing that we are learning, when we in fact are not. For example, when I say that I am “following” those Coursera MOOCs, what that means is that I have videos playing while I do something else (like writing emails or cleaning my apartment), i.e. I am not listening carefully, and I never ever do the tests and quizzes they provide. Yet, I still feel like I am learning something. I might or might not* be, but in any case I am not using those resources as effectively as I could be, and in fact most people aren’t.

And testing, I get it: Educators typically don’t like designing tests, because it is really hard. And most students don’t like taking tests, again because it is really hard, so tests have a really bad reputation all around. Especially repeated testing and e-assessment (like we are developing for mathematics and mechanics) people really love to hate!

But this is where the Yong & Lim (2016) study comes in. They showed a short (<3min) Coursera lecture to their participants. Depending on the group, during study time, they showed the clip either once and then tested three times, showed it three times and tested once, or showed it four times. Initial recall right after the study period is best for the group that watched the same clip four times, but it turns out that both groups that test during studying perform significantly better on a test a week after the study period: testing as part of studying (and in contrast to just repeatedly watching a clip) helped anchor the new knowledge significantly better.

From this is it clear that we should definitely be taking advantage of the tests provided with video lectures! Or if there are no tests available, like with TED talks**, instead of watching a lecture over and over again, test ourselves on it: Can I remember the main points? What were the reasons for x or the steps in y? Why did she say z?

And, more importantly, as educators we should take these results to heart, too.  If testing is this important, we need to provide good tests to students, and we need to encourage them to use them to practice.

One scary fact to end this post with: Of the 30 idea units presented in the videos of the study, the best group retained on average only about half until a week after watching those videos. And the worst group only one-third. I didn’t see those videos so I can’t speak about how well they were made and whether the tests addressed all of those 30 idea units, but I wouldn’t bet on students remembering more of the videos I want them to learn from. Which really gives me something to think about.

*watching those videos and feeling good about doing something productive might actually just give me the illusion of competence

**or if we feel that the tests are really bad, which does happen

Yong, P., & Lim, S. (2016). Observing the Testing Effect using Coursera Video-Recorded Lectures: A Preliminary Study Frontiers in Psychology, 6 DOI: 10.3389/fpsyg.2015.02064

How your behavior as an instructor influences how your students behave during peer instruction phases

It probably doesn’t come as a surprise to you that how you behave as an instructor influences how your students work during peer instruction phases. But do you know what you can do to make sure that student discussions are reaching the level of critical thinking that you want? I.e., how do you construct classroom norms? There is a paper by Turpen and Finkelstein (2010) that investigates just that.

In their study, they focus on three factors of classroom culture: faculty-student collaboration, student-student collaboration and sense-making vs answer-making. For this, they use Mazur-like sequence of Peer Instruction (PI) (except that they usually omit the first silent phase) and compare their observations of instructor behavior with student observations.
On the continuum between low and high faculty-student collaboration, there are a couple of behaviors in which mainly those instructors engage who have a high collaboration with students: leaving the stage during PI phases to walk around and listen to or engage in student discussions, answering student questions, and hear student explanations publicly (often several explanations from different students). Here students have many opportunities to discuss with the instructor, and the correct response is often withheld until the students have reached a consensus. Unsurprisingly, in classes where instructors are on the high end of faculty-student collaborations, students talk to the instructor more often, have lower thresholds of asking questions, and feel more comfortable discussing with the instructor.
Looking at student-student collaboration, there are again instructor practices that appear helpful. For example, low-stakes grading does provoke competitive behavior the same way high-stakes grading would.
When using clickers, collaboration is more prevalent when discussion phases are sufficiently long, when collaboration is explicitly encouraged (“talk to your neighbor!”), and when the instructor often models scientific discourse. Modeling scientific discourse (“can you explain your assumption?”) is more effective when the instructor talks to student groups during peer instruction and they have the chance to practice the behavior rather than being one out of several hundred students listening passively, but even modeling the behavior you want in front of the class is better than not doing it.
Sense-making (in contrast to answer-making) can be encouraged by the instructor through practices like explicitly putting emphasis on sense-making, reasoning, discussion, rather than just picking an answer, which means that ample time for discussions needs to be given.
Another practice is providing explanations for correct answers (also in the lecture notes) rather than just which answer was correct.
I find it really interesting to see that the observations made by researchers on concrete teaching practices can be related to what students perceive the classroom norms in a particular course are. This means that you can explicitly employ those behaviors to influence the norms in your own classroom and create a climate where there is more interaction both between the students and yourself, and among the students. So next time you are frustrated about how students aren’t asking questions even though they obviously haven’t understood a concept, or about how they just pick a random answer without sufficiently thinking about the reasons, maybe try to encourage the behavior you want by explicitly stating what you want (and why) and by modeling it yourself?

Turpen, C., & Finkelstein, N. (2010). The construction of different classroom norms during Peer Instruction: Students perceive differences Physical Review Special Topics – Physics Education Research, 6 (2) DOI: 10.1103/PhysRevSTPER.6.020123

How your students might be hurting themselves by skipping classes

Mandatory attendance is seldomly done in german higher education. The system relies on a series of examinations, and whoever passes those get the degree, no matter how much or how little time they have spent inside university buildings*. At the same time, there is a push for mandatory attendance because people feel that only if they force students to be physically present in class, they can make sure students learn what they are supposed to be learning, because they feel students can pass examinations with good grades without ever having set foot in a class, thereby missing out on a lot of learning they should have done**.

And then Ib (Hi Ib! :-)) recently asked me about an article on the importance of student attendance by Schulmeister (2015, “Abwesenheit von Lehrveranstaltungen. Ein nur scheinbar triviales Problem“). In that meta study, about 300 articles from many different countries are brought together to ponder the question of mandatory attendance.

The motivation is that one of the German Federal States recently changed its laws and now prohibits making attendance at university compulsory. The two main reasons are that attendance (and more generally, learning) is seen as the personal responsibility of students, and that students may depend on working to fund their studies. However, Schulmeister argues, many studies have shown that even though personal responsibility for outcomes is a huge motivator, there is no way to “force” someone to take on personal responsibility. And for the need to work to finance being at university, recent studies show that most students don’t work out of the necessity to earn money, but because they would like to have a higher income to be able to splurge on more things. Hence those two arguments don’t seem to carry a lot of weight. But what are the reasons for students not attending class?

There are a couple of “external factors” that affect student attendance. Students who live further away have higher attendance rates than those who live close by, maybe because they aren’t as tempted to have a quick nap at home in the middle of the day and then never come back to university. The weather also plays a role: the worse the weather, the lower student attendance. On the other hand students miss class more often due to vacations during summer. And attendance even depends on the day of the week!

But there are also other reasons for students to decide to stay away from class. Being tired or expecting the class to be boring are often mentioned, and most reasons appear trivial. Some students — interestingly, typically those with low grades — mention that they stay away because the teaching is bad. And studies find that students are convinced that it doesn’t matter for their learning outcome whether they attended class or not.

In fact, students often claim that they can compensate for not being in class by studying at home. And that might be the case if someone missed a single meeting for important reasons. However once people miss a couple of classes, on average they don’t compensate for it by studying more at home. On the contrary – students who miss a lot of classes often don’t even use the resources provided in learning management systems or by their peers. And even when they do, it cannot compensate for the missed attendance. Attendance is an important predictor of student success.

A big part of the discussion is whether personal freedom of students is limited if they were to be forced to attend classes. Some say that students are grown-ups, so it should be up to them to decide. On the other hand, studies show that those students who miss more classes hurt themselves by earning lower grades. Studies also show that the more classes someone attends, the higher their learning outcomes and the lower the risk to fail classes or drop out of university. So might it even be the responsibility of teachers to ensure students don’t hurt themselves, even if it meant limiting their personal freedom?

So what does all of this mean for us?

First, students need to be aware that they are, in fact, hurting themselves by staying away from classes. There are enough studies that have shown this, no matter what they might believe. And there are further studies that show that being aware of this alone already leads to increased attendance.

Second, we need to be aware that making attendance mandatory will make weaker students perform better (and the weaker students are also those who miss more classes in the first place).

Third, if we want mandatory attendance, policies that punish for missing class are more successful than those rewarding attendance (in most studies – not all). This seems to contradict the classical “dangle a carrot rather than threaten with a whip“.

But in the end, the best way to ensure high learning outcomes is probably the middle ground between mandatory attendance and complete laissez-faire. A compromise might be to monitor student attendance and use extended absence as a reason to warn students about the dangers of missing classes, and to provide mentoring and education on how learning works. And to keep negotiating with our students how much freedom they want and need and how much we are willing to provide to keep them from harming themselves.

What is your take on student attendance? Should they decide for themselves whether  or not they want to attend, or should attendance be mandatory?

And Ib, what else would you like to know about this study? :-)

*Of course there are courses where attendance is or can be made compulsory, for example certain lab courses or student cruises. And even without mandatory attendance there are courses where you have to submit work continuously throughout the semester, making attendance compulsory for logistical reasons. But those are not the norm.

**To which I would reply — well, if your examination actually tested everything you want students to know and be able to do after your class, you would make sure that only those students pass that actually mastered everything. And then it would not matter how and where they learned it! Not relying on your examinations to “filter out” students who have not learned “enough” means that your examinations failed, not necessarily your teaching…

Rolf Schulmeister (2015). Abwesenheit von Lehrveranstaltungen. Ein nur scheinbar triviales Problem Studien zur Anwesenheit in Lehrveranstaltungen

Why you should shuffle practice problems rather than blocking them

We like to get into the flow when practicing something, and we like to have our students concentrate on one particular type of problem at a time until they have mastered it, before moving on to the next. But is that really the best way of learning? Spoiler alert: It is not!

In a 2014 study, Rohrer, Dedrick and Burgess show the benefits of interleaved mathematics practice for problems that are not superficially similar. If problems are superficially similar, it makes intuitive sense that one needs to – at least at some point – practice several types together, because clearly distinguishing different kinds of problems and choosing the appropriate approach to solving it is not easy since the problems themselves look so similar. But for problems that look already very different one might think that blocking similar problems and practicing on them until they are mastered, and then moving on to the next type of problem might be a good choice, since one can really concentrate on each type individually and make sure one masters it.

However, this is not what the data shows. Mean test scores in their study (on an unannounced test two weeks after a nine-week practice period) were twice as high for students who had practiced interleaved problems than for those who had been objected to blocked study. Why is that the case?

There are many possible reasons.

One not even connected to interleaving or blocking is that the spacing effect comes into play: just by learning about a topic spaced in chunks over a longer period of time, the learning gain will be higher.

But interleaving itself will help students learn to distinguish between different kinds of problems. If all problems students encounter in any given lesson or homework assignment are of the same kind, they cannot learn to distinguish this kind of problem from other kinds. Being able to distinguish different kinds of problems, however, is obviously necessary to pick the appropriate strategy to solving a problem, which in itself is obviously necessary to actually solving the problem.

So why can’t student learn this in blocked practice? For one, they don’t even need to look for distinguishing features of a given problem if they know that they will find its solution by applying the exact same strategy they used on the problems before, which will also work for the problems after. So they might get a lot of practice executing a strategy, but likely will not learn under which circumstances using this strategy is appropriate. And the strategy might even just be held in short-term memory for the duration of practice and never make it into long term memory since it isn’t used again and again. So shuffling of types of problems is really important to let students both distinguish different types of problems, and associate the correct strategy to solving each type.

If you are still not convinced, there is another study by Rohrer and Taylor (2007) that shows part of what you might be expecting: That practice performance of “blockers” (i.e. students who practice in blocks rather than mixed) is substantially higher than that of “mixers”. Yet, in a later test on all topics, mixers very clearly outperformed blockers here, too.

So what does that mean for our teaching? Shuffle practice problems and help students learn how to discriminate between different kinds of problems and associate the right approach to solving each kind!

Rohrer, D., & Taylor, K. (2007). The shuffling of mathematics problems improves learning Instructional Science, 35 (6), 481-498 DOI: 10.1007/s11251-007-9015-8

Rohrer D., Dedrick R.F., & Burgess K. (2014). The benefit of interleaved mathematics practice is not limited to superficially similar kinds of problems. Psychonomic bulletin & review, 21 (5), 1323-30 PMID: 24578089

Does multitasking hurt learning? Show ’em!

I am reading the “Faculty Focus” mailing list, and a side-note in one of their recent posts, “Why policies fail to promote better learning decisions” by Lolita Paff, really struck a chord with me.

The article is about how to modify policies (like no screens! compulsory attendance! etc) to help students understand why behaving in a way the policies tries to enforce is actually beneficial to them and their learning. She refers to the article “The effect of multitasking on the grade performance of business students” by Ellis, Daniels, Jauregui (2010), where they show the effect of multitasking by splitting a class in two, and allowing one half to text while the other half has to switch off their phones. It turns out that the half that wasn’t multitasking performed significantly better on a test later.

So far, so not surprising. But what Paff suggests is really simple: Rather than telling your class about how multitasking is harming their learning, or even talking explicitly about the Ellis et al. paper, re-do this experiment with your class! In times of clickers in most (many? some?) classrooms and online-testing as abundant as it is, doing this for a class period, then testing, then showing the results is really not a big deal any more. And how much more impressive for your students to see how one half of the class performs significantly better than the other than just hearing that multitasking might not be such a good idea? I would certainly like to give this a try next time I’m teaching a class where I feel that students are multitasking too much.

P.S.: Maybe you shouldn’t split your class front vs back to get those results or other factors might come into play ;-)

Yvonne Ellis, Bobbie Daniels, & Andres Jauregui (2010). The effect of multitasking on the grade performance of business students Research in Higher Education Journal

And even more on motivation

Last week we talked about motivation quite a bit: First about why do students engage in academic tasks?, then about how motivation is proportional to the expectation of achieving a goal. Today I want to bring it all together a bit more, by presenting two other theories (both also described in the Torres-Ayala and Herman (2012) paper, which — should you not have read it yet — I strongly recommend you look into!).

The self-determination theory describes three components of motivation: Autonomy (i.e. being able to determine what you learn, when you learn it and how you learn it), competence (feeling like what you are learning is giving you (more) options to achieve what you want to achieve) and relatedness (feeling connected to a group).


Self-determination theory

Those are all components that you, the instructor, do have some influence on. For example a feeling of autonomy can be fostered as easily as giving students the choice of three problem sets and asking them to choose the one they want to work on. Or to let them choose the group they want to work with rather than prescribing groups yourself. Or even only letting them determine the order in which you talk about homework questions.

A feeling of competence is a little more difficult for you to influence, but can be achieved by giving problem sets that gradually become more difficult, instead of giving them really challenging problems right away.

And a feeling of relatedness can be achieved for example by letting students choose who they want to work with, and by making sure you observe the group processes and intervene when necessary.

So far, so good.

There is a fourth theory in the paper, that I also drew little pictures for, but which when preparing for my upcoming workshop for TU Dresden, I chose to drop for now. After all, there is only so much theory one can take in at a time, and I know that what the participants of the workshop come for are methods, methods, methods. Which I might actually give them!

Anyway, I still want to look at the expectancy-value theory here.

Expectancy-value theory basically connects motivational beliefs with achievement behavior.

If you believe you can achieve your goal (expectancy) and reaching that goal is important to you (value), this will modify your behavior. For example, you will likely choose to practice more, and on harder problems than people who don’t have the same beliefs. You will likely be more persistent in pursuing your goal. The quality of your effort will be higher, your cognitive engagement will be higher, and your actual performance will also be better.


Expectancy-value theory

There are a lot of studies that link student beliefs with their behavior, and I find this super interesting. I’ll definitely get back to reading and writing about this very soon!

Ana T. Torres-Ayala, & Geoffrey L. Herman (2012). Motivating Learners: A Primer for Engineering Teaching Assistants American Society for Engineering Education

Motivation proportional to the expectation of achieving a goal?

In the last post I talked about a paper on “Motivating Learners: A Primer for Engineering Teaching Assistants” by Torres-Ayala and Herman (2012). Today, I want to present a different motivation theory, also described in that paper:

Attribution theory

Attribution theory basically says that motivation is proportional to the expectation of achieving a goal. Three different factors come into play: externality, stability and controllability. So there are basically four different mindsets students can have:

The most desirable one is one that places an emphasis on effort. Students believe that their chance for success is something internal and unstable, which means that since it is determined within themselves and is not fixed, it can be changed. So they know that if they work harder (or work differently), they can be successful. Since their fate is in their own hands, it is easy to be motivated to do your best.

Other students focus on their ability. This is not desirable, because while they still perceive their chance for success as something that is determined within themselves, they also think that they cannot influence whether they are successful or not. They typically feel like they are not smart enough (or — almost as bad — that they are so smart that everything has to go their way, no matter how much effort they put into it).

A third group of students focusses on task difficulty. Task difficulty is obviously determined externally and is stable – students are likely to feel like the exam was too difficult anyway and they had no chance of controlling whether or not they would be successful.

And then lastly, students that feel that their success depends on luck. Luck is also external, and it is unstable. They don’t know whether they will be lucky or not, but in any case they feel like there is no point putting in an effort.


My illustration of attribution theory of motivation

How does knowing about attribution theory help us improve our teaching?

When we know that students are basically only motivated when they feel like they have a direct influence on whether or not they will be successful, we should try and create an environment where learners do feel like that. That means fostering a growth mindset, i.e. not focussing on student abilities, but making sure they realize that they can learn whatever they chose if they put in the effort. It also means making sure that students can rely on the environment being exactly like you said it would be, meaning that if you say you won’t call on people which didn’t raise their hands, you can absolutely not do it. And it also means that students cannot get the impression that you favor some over the others, or that your mood and your grades depend on the weather. Lastly, it means that the task difficulty has to be appropriate. Some challenge is good, but if students don’t have a chance to succeed, they will not continue trying indefinitely (in fact, most quit a lot faster than expected). And who can blame them when their chances of success aren’t more or less proportional to the amount of effort they put in?

Ana T. Torres-Ayala, & Geoffrey L. Herman (2012). Motivating Learners: A Primer for Engineering Teaching Assistants American Society for Engineering Education