Tag Archives: literature

Outreach is about more than about the perfect presentation (or even the perfect hands-on tank experiment!)

Leave a reply
In most of my blog posts on outreach I focus on how to run the _perfect_ experiment. And while I still think that’s awesome, I recently read an article by Johanna Varner (“Scientific Outreach: Toward Effective Public Engagement with Biological Science”, 2014) that made a lot of points that I have definitely not stressed enough on my blog, and probably not even considered enough.
Outreach is often modeled on scientific communication and intuition. Of course, since that is what we’ve learned over the years and gotten good at, and what we are most comfortable with. But when we are trying to engage the “general public”, those are mostly people who have a very different background from us. Speaking of backgrounds — there is a problem with the concept of “the general public”, as there is no _one_ general public. The general public is very very diverse, and it is important to consider each audience individually. And there is the next thing: “Audience” then often implies that a scientist talks and “the general public” listens, which is not the best model. One-way communication that we often use in outreach, more often than not using simplified, sensationalized stories, is just not effective. For retention of facts as well as for building enthusiasm and for engaging in deep thinking, the public needs to be actively engaged, not talked to.
To also consider is that the reliability of a source is not judged by how many PhDs a speaker has, but by how well it supports the listener’s preconceptions. Any new information is interpreted in such a way that it supports existing ideas. And even if ideas could be “objectively transferred”: new knowledge does not change attitudes or behaviour. And even the intention to act is a poor predictor of future behaviour!
So what can we do?
The article provides a structure for planning outreach activities which is basically backward design: Start with what you want people to learn, then think about what you would take as evidence that they actually learned it, and then plan the activity. Check out the article if you are not familiar with the concept, it’s a really nice introduction. And it is always important to remember that effectiveness of any activity depends on an explicit definition of the goals.
Then, there are a couple of design elements we can use. All of those come from the article originally, but I give my own interpretation and examples.
  • Use “trusted resources” to help us share our message. Instead of doing our outreach activity as a self-organized event, use local churches, artists, any institution or person whom the community trusts to invite you and set the stage for you, this will make it much more likely that people will not only listen to, but actually consider taking on your message.
  • Know your audience. This is super difficult! But since you will want to create personal relevance for your audience (since personal relevance is essential for engagement), you need to know about what your audience’s knowledge, attitudes, values are. And it goes without saying that every outreach activity needs to be tailored to each audience specifically.
  • Establish common ground with your audience, this makes your message more likely to be accepted. Don’t be the scientist who nobody can relate to, be the person who lives in the same neighbourhood, who supports the same sports team, who likes the same kind of music, whatever is applicable in your case.
  • Use appropriate language! Don’t alienate by speaking to science-y, and also beware that words carry a very different meaning in science than in everyday language sometimes (And if you have never seen those tables that tell you that the term “alcohol”, vor example, means “booze” to the general public, when you use it to mean “solvent”, definitely check out examples of such tables here or here!)
  • Get into dialogue instead of just “preaching” in a one-way manner. Ask for questions and feedback, offer to follow-up by email, engage with the people there!
  • Frame your science in a storyline. It makes it much easier to follow and to digest as well as to remember.

    wasserflaschen copy

    Click to enlarge

  • Use “vivid hooks”, i.e. present your research question as an actual question or puzzle to solve, ask people to brainstorm hypotheses, show them the real data, let them get actively involved! Experiential learning and personal experience influence attitudes and beliefs strongly. This might be easiest if you had animals to show, but even just a good question works. Sometimes it’s actually surprising to see what works: The other day I had a blog post showing an empty bottle and one filled with water and asked whether people knew which one was which. And I got so many private messages with people’s answers, asking me to confirm they were correct! I had never thought that this particular blog post would raise such interest.
  • Emphasize benefits of action rather than risks of inaction. Fear appeals can backfire, since they lead to feelings of helplessness, which then lead to denial, apathy, resignation. And all of those prevent engagement.
  • Provide action resources. Enthusiasm and active engagement don’t stay up for very long after you are done with your outreach experiment if you don’t do something to keep them up. Therefore, provide action resources! Let people know when your next event will be, or the schedule of public events at your institution. Hand out take-home activities. Provide online resources or lists of other people’s online resources. Make sure that those who would like to stay engaged have a very low threshold to do so!

And now, go read the original research where all of these ideas came from:

Varner (2014) “Scientific Outreach: Toward Effective Public Engagement with Biological Science”
Waves on Pinnau in Mölln. By Uwe Glessmer

You learn better when you explain to yourself

Leave a reply

I just read a really interesting article on explaining to yourself as a mechanism for learning by Tania Lombrozo. We have talked about peer instruction being valuable because explaining to others helps both the “others” and the explainer, and it’s really common to hear student tutors say that they only understood something really well when they had to explain it to students they were tutoring. In fact, many people I know use putting someone in the position of having to explain something to make themselves (or their students, if they appoint them as tutors) understand better, and studies show it works. But explaining to yourself?

The author describes research on how, why and when explaining leads to new learning. You should go check out the original blog post, too, but here is what I am taking away from it: When you explain, you are looking for general pattern.

The author cites research that shows that explaining to yourself is not the best strategy for all kind of learning outcomes — only for those that are related to the causal effects you were explaining to yourself. For other details, it might be a better strategy to just observe, or describe what you are seeing.

How is this relevant for our teaching? There are several ways.

Explaining to themselves is a strategy we can recommend to our students. I remember studying for my oral examinations at Vordiplom (now equivalent to Bachelor) level. I used to come up with questions and try and answer them late at night when I couldn’t go to sleep (Why is the Atlantic ocean more salty than the Pacific ocean? This kind of stuff). Those were questions that I didn’t know the correct answer of at the time (and some of my questions there might not be an answer) and it definitely helped me when I was then asked what geometry of sound receivers I would use if I were to build an array for SOFAR floats, and it made me feel safer going into the exam, knowing that I had answered all questions that I could come up with previously as well as I could.

And of course you can just tell students that they will have to teach about a topic, since anticipating having to teach already leads to improved learning. Then you can reflect later on how thinking they would have to teach led them to use different learning strategies, and whether they might want to use those in the future even when they were not expecting having to teach.

I even see a similar effect with having a blog. Now, when I take pictures of water somewhere, I observe pretty carefully, anticipating that I will write about what I saw and that someone might ask questions about it. That definitely makes me put a little extra effort into observing and thinking about what might be going on there!

Check out the original blog post on explaining to yourself as a mechanism for learning by Dr. Lombrozo — there is a really nice example in there that I definitely want to use in future workshops to make that exact point. You will enjoy it, too!

Mirjam Glessmer and Timo Lüth leading a workshop for university instructors

You learn better when you think that you will have to teach

Leave a reply

Have you ever worked as student tutor? Then you’ve probably felt like you understood the content of the course you tutored a million times better after tutoring it. Or at least that’s what I hear over and over again: People feel like they understood a topic. Then they prepare to teach it, and realise how much more there was to understand and that they actually understood it.

And there is research that shows that you don’t actually need to teach in order to get the deeper understanding, it is enough to anticipate that you will teach: “Expecting to teach enhances learning and organization of knowledge in free recall of text passages” by Nestojko, Bui, Kornell & Bjork (2014).

In that article, two groups of participants are given texts that they are to study. One group is told that they will be tested on the text, the other one that they will have to teach someone else who then will be tested. After all participants study the text, they are then all tested (and nobody gets to teach). But it turns out that even expecting to teach had similar benefits to what we see in student tutors who actually taught: Participants expecting to teach have a better recall of the text they had to study, can answer more questions about it and especially questions regarding main points.

So what does that mean for teaching? As the authors say: “Instilling an expectation to teach […] seems to be a simple, inexpensive intervention with the potential to increase learning efficiency at home and in the classroom.” And we should definitely use that to our advantage! :-)

How to know for sure whether a teaching intervention actually improved things

Leave a reply

How do we measure whether teaching interventions really do what they are supposed to be doing? (Spoiler alert: In this post, I won’t actually give a definite answer to that question, I am only talking about a paper I read that I found very helpful, and reflecting on a couple of ideas I am currently pondering. So continue reading, but don’t expect me to answer this question for you! :-))

As I’ve talked about before, we are currently working on a project where undergraduate mathematics and mechanics teaching are linked via online practice problems. Now that we are implementing this project, it would be very nice to have some sort of “proof” of its effectiveness.

My (personal) problem with control group studies
Control group studies are likely the most common way to “scientifically” determine whether a teaching intervention had the desired effect. This has rubbed me the wrong way for some time — if I am so convinced that I am improving things, how can I keep my new and improved course from half of the students that I am working to serve? Could I really live with myself if we, for example, measured that half of the students in the control group dropped out within the first three or four weeks of our undergraduate mathematics course, while of the experimental group, only much fewer students dropped out, and much later in the semester? On the other hand, if our intervention had such a large effect, shouldn’t we be measuring it (at least once) in a classical control group study, so we know for sure what its effect is, in order to convince stakeholders at our and other universities that our intervention should be adopted everywhere? If the intervention really improves this much, everybody should see the most compelling evidence so that everybody starts adopting the intervention, right?

A helpful article
Looking for answers to the questions above, I asked Nicki for help, and she pointed me to a presentation by Nick Tilley (2000), that I found really eye-opening and helpful for framing those questions differently, and starting to find answers. The presentation is about evaluation in a social sciences context, but easily transferable to education research.

In this presentation, Tilley first places the proposed method of “realistic evaluation” in the larger context of philosophy of science. For example Popper (1945) suggests using small-scale interventions to deal with specific problems instead of large interventions that address everything at once, and points to the opportunities to investigate the extent to which the theories (on which those small-scale interventions were built) can be tested and improved. Similarly, Campbell (1999) talks about “reforms as experiments”. So the “realistic evaluation” paradigm has been around for a while, partly in conflict with how we do science “conventionally”.

Reality is too complex for control group studies
Then, Tilley talks about classical methods, specifically control group experiments, and argues that — in contrast to what is portrayed in washing detergent ads, for example — studys are typically too complex to directly transfer results between different contexts. In contrast to what science typically does, we are also not investigating a law of nature, where the goal is to understand a mechanism causing a regularity in a given context. Rather, we are investigating how we can cause a change in a regularity. This means we are asking the question “what works for whom in what circumstances?”. With our intervention, we might be introducing different mechanisms, triggering a change in balance of several mechanisms, and hence change the regularities under investigation (which, btw, is our goal!) — all by changing the context.

The approach for evaluations of interventions should therefore, according to Tilley, be “Context Mechanism Outcome Configurations” (CMOC), which describe the interactions between context, mechanism and outcome. In order to create such a description, one needs to clearly describe the mechanisms (“what is it about a measure which may lead it to have a particular outcome pattern in a given context?”), context (“what conditions are needed for a measure to trigger mechanisms to produce particular outcome patterns?”), outcome pattern (“what are the practical effects produced by causal mechanisms being triggered in a given context?” and this finally leads to CMOCs (“How are changes in regularity (outcomes) produced by measures introduced to modify the context and balance of mechanisms triggered?”).

Impact of CCTV on car crimes — a perfect example for control group studies?
Tilley gives a great example for how this works. Investigating how CCTV affects rates of car crimes seems to be easily measured by a classical control group setup. Just install the cameras and compare their crime rates with those of parking spaces without cameras! However, once you start thinking about mechanisms through which the CCTV cameras could influence crime rates, there are lots of different possible mechanisms. There are eight named explicitly in the presentation, for example offenders could be caught thanks to CCTV and go to jail, hence crime rates would sink. Or, criminals might not choose to commit crimes, because the risk of being caught increased due to CCTV, which would again result in lower crime rates. Or people using the car park might feel more secure in using it and therefore start using it more, making it busier at previously less busy times, making car theft more difficult and risky, leading to sinking crime rates.

But then, we also need to think about context, and how car parks and car park crimes potentially differ. For example, crime rate can be the same whether there are a few very active criminals, or many not as busy ones. So catching the similar number of offenders might have a different effect, depending on context. Or the pattern of usage of car parks might depend on working hours of people working close by. So if the dominant CCTV mechanism would be to increase confidence in usage, this would not really help because the busy hours are dedicated by people’s schedules, not how safe they feel. If this would lead to higher usage, however, more cars being around might mean more car crimes because there are more opportunities, yet still a decreased crime rate per use. Another context would be that thieves might just look for new targets outside of the one car park that is now equipped with CCTV, thereby just displacing the problem elsewhere. And there are a couple more contexts mentioned in the presentation.

Long story short: Even for a relatively simple problem (“how does CCTV affect car crime rate?”), there is a wide range of mechanisms and contexts which will all have some sort of influence. Just investigating one car park with CCTV and a second one without will likely not lead to results that help solve the car crime issue once and for all everywhere. First, theories of what exactly the mechanisms and contexts are for a given situation need to be developed, and then other methods of investigation are needed to figure out what exactly is important in any given situation. Do people leave their purses sitting out visibly in the same way everywhere? How are CCTV cameras positioned relative to the cars being stolen? Are usage pattern the same in two car parks? All of this and more needs to be addressed to sort out which of the context-mechanism theories above might be dominant at any given car park.

Back to mathematics learning and our teaching intervention
Let’s get back to my initial question that, btw, is a lot more complex than the example given in the Tilley-presentation. How can we know whether our teaching intervention is actually improving anything?

Mechanisms at play
First, let’s think about possible mechanisms at play here. “What is it about a measure which may lead it to have a particular outcome pattern in a given context?” Without claiming that this is a comprehensive list, here are a couple of ideas:
a) students might realize that they need mathematics to work on mechanics problems, increasing their motivation to learn mathematics
b) students might have more opportunity to receive feedback than before (because now the feedback is automated), and more feedback might lead to better learning
c) students might appreciate the effort made by the instructors, feel more valued and taken seriously, and therefore be more motivated to put in effort
d) students might prefer the online setting over classical settings and therefore practice more
e) students might have more opportunity to practice because of the flexibility in space and time given by the online setting, leading to more learning
f) students might want to earn the bonus points they receive for working on the practice problems
g) students might find it easier to learn mathematics and mechanics because they are presented in a clearer structure than before

Contexts
Now contexts. “What conditions are needed for a measure to trigger mechanisms to produce particular outcome patterns?” Are all students and all student difficulties with mathematics the same? (Again, this is just a spontaneous brain storm, this list is nowhere near comprehensive!)
– if students’ motivation to learn mathematics increased because they see that they will need it for other subjects (a), this might lead to them only learning those topics where we manage to convey that they really really need them, and neglecting all the topics that might be equally important but where we, for whatever reasons, just didn’t give as convincing an example
– if students really value feedback this highly (b), this might work really well, or there might be better ways to give personalised feedback
– if students react to feeling more valued by the instructor (c), this might only work for the students who directly experienced a before/after when the intervention was first introduced. As soon as the intervention has become old news, future cohorts won’t show the same reaction any more. It might also only work in a context where students typically don’t feel as valued so that this intervention sticks out
– if students prefer the online setting over classical settings generally (d), or appreciate the flexibility (e), this might work for us while we are one of the few courses offering such an online setting. But once other courses start using similar settings, we might be competing with others, and students might spend less time with us and our practice problems again
– if students mainly work for the bonus points (f), their learning might not be as sustainable as if they were intrinsically motivated. And as soon as there are no more bonus points to be gained, they might stop using any opportunity for practice just for practice’s sake
– providing students a structure (g) might make them depend on it, harming their future learning (see my post on this Teufelskreis).

Outcome pattern
Next, we look at outcome patterns: “what are the practical effects produced by causal mechanisms being triggered in a given context?”. So which of the mechanisms identified above (and possibly others) seem to be at play in our case, and how do they balance each other? For this, we clearly need a different method than “just” measuring the learning gain in an experimental group and compare it to a control group. We need a way to identify the mechanisms at play in our case, and those that are not. We then need to figure out the balance of those mechanisms. Is the increased interest in mathematics more important than students potentially being put off by the online setting? Or is the online setting so appealing that it compensates for the lack of interest in mathematics? Can we show students that we care about them without rolling out new interventions every semester, and will that motivate them to work with us? Do we really need to show the practical application of every tiny piece of mathematics in order for students to want to learn it, or can we make them trust us that we are only teaching what they will need, even if they aren’t yet able to see what they will need it for?

This is where I am currently at. Any ideas of how to proceed?

CMOCs
And finally, we have reached the CMOCs (“How are changes in regularity (outcomes) produced by measures introduced to modify the context and balance of mechanisms triggered?”). Assuming we have identified the outcome patterns, we would need to figure out how to change those outcome patterns, either by changing the context, or by changing the balance of mechanisms being triggered.

After reading this article and applying the concept to my project (and I only read the article today, so my thoughts will hopefully evolve some over the next couple of weeks!), I feel that the control group study that everybody seems to expect from us is not as valid as most people might think. As I said above, I don’t have a good answer yet for what we should do instead. But I found it very eye-opening to think about evaluations in this way and am confident that we will figure it out eventually! Luckily we have only run a small-scale pilot at this point, and there is still some time before we start rolling out the full intervention.

What do you think? How should we proceed?

How to learn most efficiently when participating in a MOOC

Leave a reply

How to learn most efficiently when participating in a MOOC? Yes, I’ll admit, that title promises quite a lot. But there is a new article by Yong and Lim (2016) called “Observing the Testing Effect using Coursera Video-Recorded Lectures” that tells us a lot about how (not) to learn. We have talked about the testing effect before: repeated testing leads to better results on examinations that repeated studying does. And it is confirmed again in this study.

Why am I so excited about this? Because both video-based studying and testing are becoming more and more common these days, and both are sometimes made out to be really bad ideas.

We find video-based learning in most aspects of our lives now (at least if we are talking about lives similar to mine ;-)) — I always follow one or two Coursera courses at the time, and I love watching TED talks. Most softwares I use have video tutorials, and in fact I talked about how I liked the video tutorials of the Monash simple climate model interface only on Tuesday. And whenever I get stuck with a task, I watch video tutorials on youtube to get me going again. And of course many of the lectures at my university are being recorded and many students rely on re-watching them when studying for exams. And, of course, there is the One Planet — One Ocean MOOC that I am involved in preparing. So obviously I see value in video lectures. Even though many people believe that re-watching a lecture does not provide the same experience as seeing it “live”, I don’t think that matters much for lectures in which there is not a lot interaction between lecturer and audience. If you can make yourself use them wisely, I think video lectures are a great substitute for lectures you — for whatever reasons — can’t watch live.

But this is also the biggest issue I have with video lectures: they can easily seduce us into believing that we are learning, when we in fact are not. For example, when I say that I am “following” those Coursera MOOCs, what that means is that I have videos playing while I do something else (like writing emails or cleaning my apartment), i.e. I am not listening carefully, and I never ever do the tests and quizzes they provide. Yet, I still feel like I am learning something. I might or might not* be, but in any case I am not using those resources as effectively as I could be, and in fact most people aren’t.

And testing, I get it: Educators typically don’t like designing tests, because it is really hard. And most students don’t like taking tests, again because it is really hard, so tests have a really bad reputation all around. Especially repeated testing and e-assessment (like we are developing for mathematics and mechanics) people really love to hate!

But this is where the Yong & Lim (2016) study comes in. They showed a short (<3min) Coursera lecture to their participants. Depending on the group, during study time, they showed the clip either once and then tested three times, showed it three times and tested once, or showed it four times. Initial recall right after the study period is best for the group that watched the same clip four times, but it turns out that both groups that test during studying perform significantly better on a test a week after the study period: testing as part of studying (and in contrast to just repeatedly watching a clip) helped anchor the new knowledge significantly better.

From this is it clear that we should definitely be taking advantage of the tests provided with video lectures! Or if there are no tests available, like with TED talks**, instead of watching a lecture over and over again, test ourselves on it: Can I remember the main points? What were the reasons for x or the steps in y? Why did she say z?

And, more importantly, as educators we should take these results to heart, too.  If testing is this important, we need to provide good tests to students, and we need to encourage them to use them to practice.

One scary fact to end this post with: Of the 30 idea units presented in the videos of the study, the best group retained on average only about half until a week after watching those videos. And the worst group only one-third. I didn’t see those videos so I can’t speak about how well they were made and whether the tests addressed all of those 30 idea units, but I wouldn’t bet on students remembering more of the videos I want them to learn from. Which really gives me something to think about.

*watching those videos and feeling good about doing something productive might actually just give me the illusion of competence

**or if we feel that the tests are really bad, which does happen

—

Yong, P., & Lim, S. (2016). Observing the Testing Effect using Coursera Video-Recorded Lectures: A Preliminary Study Frontiers in Psychology, 6 DOI: 10.3389/fpsyg.2015.02064

How your behavior as an instructor influences how your students behave during peer instruction phases

Leave a reply

It probably doesn’t come as a surprise to you that how you behave as an instructor influences how your students work during peer instruction phases. But do you know what you can do to make sure that student discussions are reaching the level of critical thinking that you want? I.e., how do you construct classroom norms? There is a paper by Turpen and Finkelstein (2010) that investigates just that.

In their study, they focus on three factors of classroom culture: faculty-student collaboration, student-student collaboration and sense-making vs answer-making. For this, they use Mazur-like sequence of Peer Instruction (PI) (except that they usually omit the first silent phase) and compare their observations of instructor behavior with student observations.
On the continuum between low and high faculty-student collaboration, there are a couple of behaviors in which mainly those instructors engage who have a high collaboration with students: leaving the stage during PI phases to walk around and listen to or engage in student discussions, answering student questions, and hear student explanations publicly (often several explanations from different students). Here students have many opportunities to discuss with the instructor, and the correct response is often withheld until the students have reached a consensus. Unsurprisingly, in classes where instructors are on the high end of faculty-student collaborations, students talk to the instructor more often, have lower thresholds of asking questions, and feel more comfortable discussing with the instructor.
Looking at student-student collaboration, there are again instructor practices that appear helpful. For example, low-stakes grading does provoke competitive behavior the same way high-stakes grading would.
When using clickers, collaboration is more prevalent when discussion phases are sufficiently long, when collaboration is explicitly encouraged (“talk to your neighbor!”), and when the instructor often models scientific discourse. Modeling scientific discourse (“can you explain your assumption?”) is more effective when the instructor talks to student groups during peer instruction and they have the chance to practice the behavior rather than being one out of several hundred students listening passively, but even modeling the behavior you want in front of the class is better than not doing it.
Sense-making (in contrast to answer-making) can be encouraged by the instructor through practices like explicitly putting emphasis on sense-making, reasoning, discussion, rather than just picking an answer, which means that ample time for discussions needs to be given.
Another practice is providing explanations for correct answers (also in the lecture notes) rather than just which answer was correct.
I find it really interesting to see that the observations made by researchers on concrete teaching practices can be related to what students perceive the classroom norms in a particular course are. This means that you can explicitly employ those behaviors to influence the norms in your own classroom and create a climate where there is more interaction both between the students and yourself, and among the students. So next time you are frustrated about how students aren’t asking questions even though they obviously haven’t understood a concept, or about how they just pick a random answer without sufficiently thinking about the reasons, maybe try to encourage the behavior you want by explicitly stating what you want (and why) and by modeling it yourself?

—
Turpen, C., & Finkelstein, N. (2010). The construction of different classroom norms during Peer Instruction: Students perceive differences Physical Review Special Topics – Physics Education Research, 6 (2) DOI: 10.1103/PhysRevSTPER.6.020123

How your students might be hurting themselves by skipping classes

Leave a reply

Mandatory attendance is seldomly done in german higher education. The system relies on a series of examinations, and whoever passes those get the degree, no matter how much or how little time they have spent inside university buildings*. At the same time, there is a push for mandatory attendance because people feel that only if they force students to be physically present in class, they can make sure students learn what they are supposed to be learning, because they feel students can pass examinations with good grades without ever having set foot in a class, thereby missing out on a lot of learning they should have done**.

And then Ib (Hi Ib! :-)) recently asked me about an article on the importance of student attendance by Schulmeister (2015, “Abwesenheit von Lehrveranstaltungen. Ein nur scheinbar triviales Problem“). In that meta study, about 300 articles from many different countries are brought together to ponder the question of mandatory attendance.

The motivation is that one of the German Federal States recently changed its laws and now prohibits making attendance at university compulsory. The two main reasons are that attendance (and more generally, learning) is seen as the personal responsibility of students, and that students may depend on working to fund their studies. However, Schulmeister argues, many studies have shown that even though personal responsibility for outcomes is a huge motivator, there is no way to “force” someone to take on personal responsibility. And for the need to work to finance being at university, recent studies show that most students don’t work out of the necessity to earn money, but because they would like to have a higher income to be able to splurge on more things. Hence those two arguments don’t seem to carry a lot of weight. But what are the reasons for students not attending class?

There are a couple of “external factors” that affect student attendance. Students who live further away have higher attendance rates than those who live close by, maybe because they aren’t as tempted to have a quick nap at home in the middle of the day and then never come back to university. The weather also plays a role: the worse the weather, the lower student attendance. On the other hand students miss class more often due to vacations during summer. And attendance even depends on the day of the week!

But there are also other reasons for students to decide to stay away from class. Being tired or expecting the class to be boring are often mentioned, and most reasons appear trivial. Some students — interestingly, typically those with low grades — mention that they stay away because the teaching is bad. And studies find that students are convinced that it doesn’t matter for their learning outcome whether they attended class or not.

In fact, students often claim that they can compensate for not being in class by studying at home. And that might be the case if someone missed a single meeting for important reasons. However once people miss a couple of classes, on average they don’t compensate for it by studying more at home. On the contrary – students who miss a lot of classes often don’t even use the resources provided in learning management systems or by their peers. And even when they do, it cannot compensate for the missed attendance. Attendance is an important predictor of student success.

A big part of the discussion is whether personal freedom of students is limited if they were to be forced to attend classes. Some say that students are grown-ups, so it should be up to them to decide. On the other hand, studies show that those students who miss more classes hurt themselves by earning lower grades. Studies also show that the more classes someone attends, the higher their learning outcomes and the lower the risk to fail classes or drop out of university. So might it even be the responsibility of teachers to ensure students don’t hurt themselves, even if it meant limiting their personal freedom?

So what does all of this mean for us?

First, students need to be aware that they are, in fact, hurting themselves by staying away from classes. There are enough studies that have shown this, no matter what they might believe. And there are further studies that show that being aware of this alone already leads to increased attendance.

Second, we need to be aware that making attendance mandatory will make weaker students perform better (and the weaker students are also those who miss more classes in the first place).

Third, if we want mandatory attendance, policies that punish for missing class are more successful than those rewarding attendance (in most studies – not all). This seems to contradict the classical “dangle a carrot rather than threaten with a whip“.

But in the end, the best way to ensure high learning outcomes is probably the middle ground between mandatory attendance and complete laissez-faire. A compromise might be to monitor student attendance and use extended absence as a reason to warn students about the dangers of missing classes, and to provide mentoring and education on how learning works. And to keep negotiating with our students how much freedom they want and need and how much we are willing to provide to keep them from harming themselves.

What is your take on student attendance? Should they decide for themselves whether  or not they want to attend, or should attendance be mandatory?

And Ib, what else would you like to know about this study? :-)

—

*Of course there are courses where attendance is or can be made compulsory, for example certain lab courses or student cruises. And even without mandatory attendance there are courses where you have to submit work continuously throughout the semester, making attendance compulsory for logistical reasons. But those are not the norm.

**To which I would reply — well, if your examination actually tested everything you want students to know and be able to do after your class, you would make sure that only those students pass that actually mastered everything. And then it would not matter how and where they learned it! Not relying on your examinations to “filter out” students who have not learned “enough” means that your examinations failed, not necessarily your teaching…

—

Rolf Schulmeister (2015). Abwesenheit von Lehrveranstaltungen. Ein nur scheinbar triviales Problem Studien zur Anwesenheit in Lehrveranstaltungen

Why you should shuffle practice problems rather than blocking them

3 Replies

We like to get into the flow when practicing something, and we like to have our students concentrate on one particular type of problem at a time until they have mastered it, before moving on to the next. But is that really the best way of learning? Spoiler alert: It is not!

In a 2014 study, Rohrer, Dedrick and Burgess show the benefits of interleaved mathematics practice for problems that are not superficially similar. If problems are superficially similar, it makes intuitive sense that one needs to – at least at some point – practice several types together, because clearly distinguishing different kinds of problems and choosing the appropriate approach to solving it is not easy since the problems themselves look so similar. But for problems that look already very different one might think that blocking similar problems and practicing on them until they are mastered, and then moving on to the next type of problem might be a good choice, since one can really concentrate on each type individually and make sure one masters it.

However, this is not what the data shows. Mean test scores in their study (on an unannounced test two weeks after a nine-week practice period) were twice as high for students who had practiced interleaved problems than for those who had been objected to blocked study. Why is that the case?

There are many possible reasons.

One not even connected to interleaving or blocking is that the spacing effect comes into play: just by learning about a topic spaced in chunks over a longer period of time, the learning gain will be higher.

But interleaving itself will help students learn to distinguish between different kinds of problems. If all problems students encounter in any given lesson or homework assignment are of the same kind, they cannot learn to distinguish this kind of problem from other kinds. Being able to distinguish different kinds of problems, however, is obviously necessary to pick the appropriate strategy to solving a problem, which in itself is obviously necessary to actually solving the problem.

So why can’t student learn this in blocked practice? For one, they don’t even need to look for distinguishing features of a given problem if they know that they will find its solution by applying the exact same strategy they used on the problems before, which will also work for the problems after. So they might get a lot of practice executing a strategy, but likely will not learn under which circumstances using this strategy is appropriate. And the strategy might even just be held in short-term memory for the duration of practice and never make it into long term memory since it isn’t used again and again. So shuffling of types of problems is really important to let students both distinguish different types of problems, and associate the correct strategy to solving each type.

If you are still not convinced, there is another study by Rohrer and Taylor (2007) that shows part of what you might be expecting: That practice performance of “blockers” (i.e. students who practice in blocks rather than mixed) is substantially higher than that of “mixers”. Yet, in a later test on all topics, mixers very clearly outperformed blockers here, too.

So what does that mean for our teaching? Shuffle practice problems and help students learn how to discriminate between different kinds of problems and associate the right approach to solving each kind!

—

Rohrer, D., & Taylor, K. (2007). The shuffling of mathematics problems improves learning Instructional Science, 35 (6), 481-498 DOI: 10.1007/s11251-007-9015-8

Rohrer D., Dedrick R.F., & Burgess K. (2014). The benefit of interleaved mathematics practice is not limited to superficially similar kinds of problems. Psychonomic bulletin & review, 21 (5), 1323-30 PMID: 24578089

Does multitasking hurt learning? Show ’em!

Leave a reply

I am reading the “Faculty Focus” mailing list, and a side-note in one of their recent posts, “Why policies fail to promote better learning decisions” by Lolita Paff, really struck a chord with me.

The article is about how to modify policies (like no screens! compulsory attendance! etc) to help students understand why behaving in a way the policies tries to enforce is actually beneficial to them and their learning. She refers to the article “The effect of multitasking on the grade performance of business students” by Ellis, Daniels, Jauregui (2010), where they show the effect of multitasking by splitting a class in two, and allowing one half to text while the other half has to switch off their phones. It turns out that the half that wasn’t multitasking performed significantly better on a test later.

So far, so not surprising. But what Paff suggests is really simple: Rather than telling your class about how multitasking is harming their learning, or even talking explicitly about the Ellis et al. paper, re-do this experiment with your class! In times of clickers in most (many? some?) classrooms and online-testing as abundant as it is, doing this for a class period, then testing, then showing the results is really not a big deal any more. And how much more impressive for your students to see how one half of the class performs significantly better than the other than just hearing that multitasking might not be such a good idea? I would certainly like to give this a try next time I’m teaching a class where I feel that students are multitasking too much.

P.S.: Maybe you shouldn’t split your class front vs back to get those results or other factors might come into play ;-)

—

Yvonne Ellis, Bobbie Daniels, & Andres Jauregui (2010). The effect of multitasking on the grade performance of business students Research in Higher Education Journal

And even more on motivation

Leave a reply

Last week we talked about motivation quite a bit: First about why do students engage in academic tasks?, then about how motivation is proportional to the expectation of achieving a goal. Today I want to bring it all together a bit more, by presenting two other theories (both also described in the Torres-Ayala and Herman (2012) paper, which — should you not have read it yet — I strongly recommend you look into!).

The self-determination theory describes three components of motivation: Autonomy (i.e. being able to determine what you learn, when you learn it and how you learn it), competence (feeling like what you are learning is giving you (more) options to achieve what you want to achieve) and relatedness (feeling connected to a group).

Folie4

Self-determination theory

Those are all components that you, the instructor, do have some influence on. For example a feeling of autonomy can be fostered as easily as giving students the choice of three problem sets and asking them to choose the one they want to work on. Or to let them choose the group they want to work with rather than prescribing groups yourself. Or even only letting them determine the order in which you talk about homework questions.

A feeling of competence is a little more difficult for you to influence, but can be achieved by giving problem sets that gradually become more difficult, instead of giving them really challenging problems right away.

And a feeling of relatedness can be achieved for example by letting students choose who they want to work with, and by making sure you observe the group processes and intervene when necessary.

So far, so good.

There is a fourth theory in the paper, that I also drew little pictures for, but which when preparing for my upcoming workshop for TU Dresden, I chose to drop for now. After all, there is only so much theory one can take in at a time, and I know that what the participants of the workshop come for are methods, methods, methods. Which I might actually give them!

Anyway, I still want to look at the expectancy-value theory here.

Expectancy-value theory basically connects motivational beliefs with achievement behavior.

If you believe you can achieve your goal (expectancy) and reaching that goal is important to you (value), this will modify your behavior. For example, you will likely choose to practice more, and on harder problems than people who don’t have the same beliefs. You will likely be more persistent in pursuing your goal. The quality of your effort will be higher, your cognitive engagement will be higher, and your actual performance will also be better.

Folie5

Expectancy-value theory

There are a lot of studies that link student beliefs with their behavior, and I find this super interesting. I’ll definitely get back to reading and writing about this very soon!

—
Ana T. Torres-Ayala, & Geoffrey L. Herman (2012). Motivating Learners: A Primer for Engineering Teaching Assistants American Society for Engineering Education