Tag Archives: questions

So what exactly are we testing?

Leave a reply

Asking the right questions is really difficult.

Last week, a paper by Gläser and Riegler was presented in the journal club at my work (can’t find it online yet, so can’t link to it, sorry!). Even though the paper itself dealt with the effectiveness of so-called “Brückenkurse” (i.e. 2-week courses that are offered to incoming students to bring them up to speed in mathematics before they take up their regular university courses), what ended up fascinating me much more is how one the choice of the test question was really unfortunate.

The authors were trying to compare additive and proportional reasoning capabilities of the students. Additive reasoning is of the kind “if M and G share a cake and M eats 2/3rd of it, how much does G get?” or “If you want to be at school five minutes earlier than usual, how much earlier than usual should you be leaving from home?”. Proportional reasoning, on the other hand, is something like “M is driving at constant speed. After one hour, she has driven 15km. How far had she driven after 30 minutes?”. Browsing now, I see that there is tons of literature on how children develop additive and proportional reasoning skills which I haven’t read yet, so please go look for yourself if you want to know more about that.

Anyway, the question the authors asked to gauge the additive reasoning skills of the students, went something like this:

A rectangle with a diagonal length of 2 cm is uniformly scaled up, such that its circumference grows by 2.5 cm. The new diagonal is now 3 cm long. Now a similar rectangle, with a 7cm long diagonal, is scaled up such that its circumference grows by 2.5 cm. How long is the diagonal of the new rectangle?

And then they offer multiple choice answers for both the result and the explanations.

Wanna figure it out for yourself? Go ahead! I won’t talk about the answer until much further down in the figure caption…

We did not actually solve this question during the discussion, but the ideas bounced around all focussed on “a2 + b2 = c2” or “sine and cosine!” or other stuff prompted by a right triangle – likely the kind of associations that students taking that test would also have.

Since we weren’t aware that we were looking at a question to be solved with additive reasoning, the simplest solution didn’t occur to us. Maybe not surprisingly, since similarity in geometry means that one shape can be produced from another only by stretching and/or rotating and/or flipping it, with all angles staying the same and the proportion of all lengths staying the same, too, which seems to be all about proportionality rather than additive reasoning.

glaeser_and_riegler_proportional_reasoning

The main steps in discovering that additive reasoning actually works in this case. From the question we know that adding a similar rectangle with 2.5 cm circumference increased the diagonal by 1 cm, from the drawing above we see that this holds no matter the size of the original rectangle as long as similarity is given (which it is in the question), hence the length of the diagonal in the question will increase from 7 to 8 cm.

Looking for an additive reasoning solution, in the end that one is very easy to find (see figure above). However, looking at those exercises was a great reminder for how much we are conditioned to react to certain stimuli in specific ways. See a right triangle? a2 + b2 = c2! Mathematics test? Must be some complicated solution and not just straight-forward adding up of two numbers! There is a lot of research on how correct problem-solving strategies are triggered in situations where they are not applicable, but it was a good (and scary!!) reminder to experience first hand how none* of us 15 or so colleagues came up with the correct strategy right away to solve this really very simple problem. This really needs to have implications on how we think about teaching, especially on how we condition students to react to clues as to what kind of strategy they should pick to solve a given problem. Clearly it is important to have several strategies ready at hand and to think a little bit about which one is applicable in a given situation, and why.

—

* edit: apparently one colleague did come up with the correct answer for the correct reasons, but didn’t let the rest of us know! Still – one out of 14 is a pretty sobering result.

Should we ask or should we tell?

Leave a reply

Article by Freeman et al., 2014, “Active learning increases student performance in science, engineering, and mathematics”.

Following up on the difficulties in asking good questions described yesterday, I’m today presenting an article on the topic “should we ask or should we tell?”.

Spoiler alert – the title says it all: “Active learning increases student performance in science, engineering, and mathematics”. Nevertheless, the recent PNAS-article by Freeman et al. (2014) is really worth a read.

In their study, Freeman and colleagues meta-analyzed 225 studies that compared student learning outcomes across science, technology, engineering and mathmatics (STEM) disciplines depending on whether students were taught through lectures or through active learning formats. On average, examination scores increased by 6% under active learning scenarios, and students in classes with traditional lecture formats were 1.5 times more likely to fail than those in active learning classes.

These results hold for all STEM disciplines and through all class sizes, although it seems most effective for classes with less than 50 students. Active learning also seems to have a bigger effect on concept inventories than on traditional examinations.

One interesting point the authors raise in their discussion is whether for future research, traditional lecturing is still a good control, or whether active learning formats should be directly compared to each other.

Also, the impact of instructor behavior and of the amount of time spent on “active learning” are really interesting future research topics. In this study, even lectures with only as little as 10-15% of their time devoted to clicker questions counted as “active”, and even a small – and doable – change like that has a measurable effect.

I’m really happy I came across this study – really big data set (important at my work place!), rigorous analysis (always important of course) and especially Figure 1 is a great basis for discussion about the importance of active learning formats and it will go straight into the collection of slides I have on my whenever I go into a consultation.

Check out the study, it is really worth a read!

On asking questions

Leave a reply

How do you ask questions that really make students think, and ultimately understand?

I’ve only been working at a center for teaching and learning for half a year, but still my thinking about teaching has completely transformed, and still is transforming. Which is actually really exciting! :-) This morning, prompted by Maryellen Weimer’s post on “the art of asking questions”, I’m musing about what kind of questions I have been asking, and why. And how I could be asking better questions. And for some reason, the word “thermocline” keeps popping up in my thoughts.

What a thermocline is, is one of the important definitions students typically have to learn in their intro to oceanography. And the different ways in which the term is used: as the depth range where temperatures quickly change from warm surface waters to cold deep waters, as, more generally, the layer with the highest vertical temperature gradient, or as seasonal or permanent thermoclines, to name but a few.

I have asked lots of questions about thermoclines, both during lectures, in homework assignments, and in exams. But most of my questions were more of the “define the word thermocline”, “point out the thermocline in the given temperature profile”, “is this a thermocline or an isotherm” kind, which are fine on an exam maybe, than of a kind that would be really conductive to student learning. I’ve always found that students struggled a lot with learning the term thermocline and all the connected ones like isotherm, halocline, isohaline, pycnocline, isopycnal, etc.. But maybe that was because I haven’t been asking the right questions? For example, instead of showing a typical pole-to-pole temperature section and pointing out the warm surface layer, the thermocline, and the deep layer*, maybe showing a less simplified section and having the students come up with their own classification of layers would be more helpful? Or asking why defining something like a thermocline might be useful for oceanographers, hence motivating why it might be useful to learn what we mean by thermocline.

In her post mentioned above, Maryellen Weimer gives several good pieces of advice on asking questions. One that I like a lot is “play with the questions”. The main point is that “questions promote thinking before they are answered”. So rather than trying to make students come up with the correct answer as quickly as possible after the question has been posed, why not let them produce multiple answers and discuss the pros and cons before settling on one of the answers? Or why not ask a question, not answer it right away, and come back to asking it over the course of the lesson or even over several lessons? I think the fear is often that if students don’t hear the right answer right away, they’ll remember a wrong answer, or lose interest in the question. However, even though this does sound plausible, this might not be how learning actually works.

A second piece of advice that I really liked in that post is “don’t ask open-ended questions if you know the answer you’re looking for”. Because what happens when you do that is, as we’ve probably all experienced, that we cannot really accept any answer that doesn’t match the one we were looking for. Students of course notice, and will start guessing what answer we were looking for, rather than deeply think about the question. This is actually a problem with the approach I suggested above: When asking students to come up with classifications of oceanic layers from a temperature section – what if they come up with something brilliant that does unfortunately not converge on the classical “warm upper layer, thermocline, cold deep layer” classification? Do we say “that’s brilliant, let’s rewrite all the textbooks” or “mmmh, nice, but this is how it’s been done traditionally”? Or what would you say?

And then there is the point that I get confronted with all the time at work; that “thermocline” is a very simple and very basic term, one that one needs to learn in order to be able to discuss more advanced concepts. So if we spent so much of our class time on this one term, would we ever get to teach the more complicated, and more interesting, stuff? One could argue that unless students have a good handle on basic terminology there is no point in teaching more advanced content anyway. Or that students really only bother learning the basic stuff when they see its relevance for the more advanced stuff. And I actually think there is some truth to both arguments.

So where do we go from here? Any ideas?

—

* how typical a plot to show in an introduction to oceanography that one is, is coincidentally also visible from the header of this blog. When I made the images for the header, I just drew whatever drawings I had made repeatedly on the blackboard recently and called it a day. That specific drawing I have made more times than I care to remember…

How to pose questions for voting card concept tests (post 2/3)

1 Reply

Different ways of posing questions for concept tests are being presented here

Concept tests using voting cards have been presented in this post. Here, I want to talk about different types of questions that one could imagine using for this method.

1) Classical multiple choice

In the classical multiple choice version, for each question four different answers are given, only one of which is correct. This is the tried and tested method that is often pretty boring.

An example slide for a question with one correct answer

However, even this kind of question can lead to good discussions, for example when it is introducing a new concept rather than just testing an old one. In this case, we had talked about different kinds of plate boundaries during the lecture, but not about the frame of reference in which the movement of plates is described. So what seemed to be a really confusing question at first was used to initiate a discussion that went into a lot more depth than either the textbook or the lecture, simply because students kept asking questions.

2) Several correct answers

A twist on the classical multiple choice is a question for which more than one correct answer are given without explicitly mentioning that fact in the question. In a way, this is tricking the students a bit, because they are used to there being only one correct answer. For that reason they are used to not even reading all the answers if they have come across one that they know is correct. Giving several correct answers is a good way of initiating a discussion in class if different people chose different answers and are sure that their answers are correct. Students who have already gained some experience with the method often have the confidence to speak up during the “voting” and say they think that more than one answer is correct.

3) No correct answer

This is a bit mean, I know. But again, the point of doing these concept tests is not that the students name one correct answer, but that they have thought about a concept enough to be able to answer questions about the topic correctly, and sometimes that includes having the confidence to say that all answers are wrong. And it seems to be very satisfying to students when they can argue that none of the answers that the instructor suggested were correct! Even better when they can propose a correct answer themselves.

4) Problems that aren’t well posed

This is my favorite type of question that usually leads to the best discussions. Not only do students have to figure out that the question isn’t well posed, but additionally we can now discuss which information is missing in order to answer the question. Then we can answer the questions for different sets of variables.

ABCD_lake

One example slide for a problem that isn’t well posed – each of the answers could be correct under certain conditions, but we do not have enough information to answer the question.

For example for the question in the figure above, each of the answers could be correct during certain times of the year. During summer, the temperature near the surface is likely to be higher than that near the bottom of the lake (A). During winter, the opposite is likely the case (B). During short times of the year it is even possible that the temperature of the lake is homogeneous (C). And, since the density maximum of fresh water occurs at 4degC, the bottom temperature of a lake is often, but not inevitably, 4degC (D). If students can discuss this, chances are pretty high that they have understood the density maximum in freshwater and its influence on the temperature stratification in lakes.

5) Answers that are correct but don’t match the question.

This is a tricky one. If the answers are correct in themselves but don’t match the question, it sometimes takes a lot of discussing until everybody agrees that it doesn’t matter how correct a statement is in itself; if it isn’t addressing the point in question, it is not a valid answer. This can now be used to find valid answers to the question, or valid questions to the provided answers, or both.

This is post no 2 in a series of 3. Post no 1 introduced the method to the readers of this blog, post no 3 is about how to introduce the methods to the students you are working with.

A, B, C or D?

4 Replies

Voting cards. A low-tech concept test tool, enhancing student engagement and participation. (Post 1/3)

Voting cards are a tool that I learned about from Al Trujillo at the workshop “teaching oceanography” in San Francisco in 2013. Basically, voting cards are a low-tech clicker version: A sheet of paper is divided into four quarters, each quarter in a different color and marked with big letters A, B, C and D (pdf here). The sheet is folded such that only one quarter is visible at a time.

A question is posed and four answers are suggested. The students are now asked to vote by holding up the folded sheet close to their chest so that the instructor sees which of the answers they chose, whereas their peers don’t.

Voting cards are sheets of paper with four different colors for the four quarters, each marked with a big A, B, C or D.

This method is great because it forces each individual student to decide on an answer instead of just trying to be as invisible as possible and hope that the instructor will not address them individually. Considering different possible answers and deciding on which one seems most plausible is important step in the learning process. Even if a student chose a wrong answer, remembering the correct answer will be easier if they learn it in the context of having made a commitment to one answer which then turns out wrong, rather than having not considered the different options in enough detail to decide on one. “I thought A made sense because of X. But then we discussed it and it turns out that because of Y and Z, C is the correct answer” is so much more memorable than “I didn’t care and it turned out it was D”. Since the answers are only visible to the instructor and not to the other students, the barrier of voting is a lot lower because potentially embarrassing situations are being avoided. It is, however, also much harder to just observe the peers’ votes and then follow the majority vote.

In addition to helping students learn, this method is also beneficial to the instructor. The instructor sees the distribution of answers with one glance and rather than guessing how many students actually understand what I was talking about, I can now make an informed choice of the next step. Should I have students discuss with their neighbor to find an agreement and then ask the class to vote again? Elaborate more on the concept before asking students to discuss among themselves? Ask individual students to explain why they chose the answer they chose? Knowing how much students understood is very helpful in choosing the right method moving forward with your teaching. And even without staring directly at specific students, it is easy to observe from the corner of the eye whether students have trouble deciding for an answer or whether they make a quick decision and stick to it.

I have been using this method in this year’s GEOF130 lecture, and in a recent Continue. Stop. Start. feedback that I asked my students to fill in, every single student (who handed back the form, but that’s a topic for a different post) mentioned how the “A, B, C, D questions” or “quizzes” (which I both interpret as meaning the voting cards) help them learn and that I should definitely continue using them.

This post is number 1 of 3 on the topic of voting cards. Post no 2 will give examples of different types questions/answers that work well with this methods (for example always having only one correct answer might not be the most efficient strategy to foster discussions), and how to use them to maximize benefit for your teaching. Post no 3 will focus on introducing voting cards as a new method with least resistance by focussing on benefits to student learning and reassuring them on how the instructor will handle the information gained from seeing everybody vote.