# Guest post by Kjersti Daae: Using voting cards to increase student activity and promote discussions and critical thinking

I got permission to publish Kjersti Daae‘s iEarth conversation on teaching (with Torgny Roxå and myself in April 2021) on my blog! Thanks, Kjersti :-) Here we go:

I teach in an introductory course in meteorology and oceanography (GEOF105) at the geophysical institute, UiB. The students come from two different study programs:

Most students do the course in their third semester. They have not yet learned all the mathematics necessary to dive into the derivation of equations governing the ocean processes. Therefore, we focus on conceptual knowledge and understand the governing ideas regarding central ocean processes, such as global circulation and the influence of Earth’s rotation and wind on the ocean currents. The students need to learn how to describe the various processes and mechanisms included in the curriculum. I, therefore, use voting cards to promote student discussions during lectures.

I first heard about voting cards from Mirjam’s blog “Adventures in Oceanography and Teaching”. The method is relatively simple. You pose a question with four alternatives A,B,C,D, accompanied by different colours for easy recognition. The students have a printout each with the four letters on it.They spend a few minutes thinking about the question and prepare their answer. Then they fold their paper so that only one letter/colour shows, and hold it up and provide direct feedback to the teacher. The questions can, among others, be used to checking if the students understand a concept or let the students guess the outcome of something they haven’t learned yet.

However, I prefer to use voting cards to promote discussions among peers. This procedure is following the Think-pair-share method developed by Lyman (1981). By carefully selecting alternative answers, I can make it hard for the students to choose the correct answer, or the answers can be formulated so that the students can argue for more than one correct answer. When the students hold up their answers, they can look around at the other students’ responses and find someone with a different response than themselves. Then they can pair up and discuss why they answer differently and see if they can agree on one common answer before sharing their opinion with the rest of the class. During this exercise, the students practice talking about science and arguing for various answers/outcomes based on the voting cards’ questions.The exercises serve at least two purposes:

1. The student practice answering/discussing relevant questions for the final exam.
2. The students get active instead of listening passively to the lecturer.

Usually, I can see the students becoming very tired after 10-15 minutes of passive listening. These voting questions “wake up” the students, and after one such question, they tend to stay focused for another 10-15 minutes.

I think the voting cards work really well. When I display a question, the students usually move from a relaxed position to sitting more straight and preparing for being active. I can hear them discussing what they are supposed to. I also get very good feedback and responses in whole-class discussions/summaries following the discussions in pairs. Such summaries are especially interesting if multiple answers can be correct, depending on how the students argue. I can select responses from students based on their visible letters and make sure we can hear different solutions to the same question. During a semester, I see a clear development in the way students reflect on the various questions and express critical thinking governing oceanographic processes. The exercises show the students how important argumentation is. An answer with a well-founded argumentation and critical thinking is worth much more than just the answer/letter. My observation is consistent with Kaddoura (2013), who found that the think-pair-share method increased nursing students’ critical thinking.

Lyman, F. (1981). “The responsive classroom discussion.” In Anderson, A. S. (Ed.), Mainstreaming Digest, College Park, MD: University of Maryland College of Education.

Kaddoura, M. (2013). «Think Pair Share: A Teaching Learning Strategy to Enhance Students’ Critical Thinking», EducationalResearchQuarterly, v36 n4 p3-24

# #Methods2Go: Ideas for starting off your university classes (inspired by EM Schumacher’s work)

“Methoden to go” by E.-M. Schumacher, which you see in the picture above, is a handy collection of well- and less-well-known methods for university teaching, organised by the six different phases of a typical session (getting into a topic, learning about a topic, discussing it, applying knowledge, securing results, and ending a lesson). It’s a collection of colourful flashcards that are loosely bound together, with a short description of a method on either side. I love the format — it’s playful and great to browse for inspiration; flipping through the cards is fun!

I recently re-discovered my copy and want to share a couple of the methods with you: the ones that sparked images in my head right away. But check out the method pool on the constructif website (in German, but many of the methods have english titles so you can either guess or google them) for a more comprehensive overview!

# Today: methods to start off your lessons

## Awaking interest

One idea to start off your lessons is to find something that sparks student interest in the topic you want to discuss. You could for example use quotes, snippets from movies, or provocative statements. These work especially well when students have an easy way to relate to them, for example because they are related to things that are relevant in their own lives or to their future in the profession.

Examples that come to mind:

1. A really fun question like Kjersti‘s below: How can you cool down a beer most efficiently when you are outdoors? Isn’t this intriguing even when you have no idea what the lecture is about?

Multiple choice question by Kjersti Daae, used with permission

This can be used as a multiple choice question at the beginning of the class, or just shown then and only picked up again at the end of the class, hopefully inspiring the students to pay attention in order to figure out the answer along the way.

2. Fun memes. I remember starting classes on ocean salinity with one showing a shark and saying something like “Did you know? The ocean is salty because of the tears of misunderstood sharks that just want to play” (tried to find the original source, but there are so many variations of this out there that I couldn’t be sure). Even though it’s probably obvious that this is not the answer we are going for, it still raises the question “why is the ocean salty” in a fun and playful way.

3. Interesting applications. I used to have a picture of a car with its heavy load poking through the front windscreen — it had clearly not been secured properly when the driver suddenly had to brake. Inertia can really be tricky… What’s great about such a picture is that it makes the relevance of an otherwise quite abstract and hard to imagine concept absolutely obvious.

I like having questions and pictures like these up on the screen while students enter the (virtual) room to get them thinking about the upcoming session. If you are really ambitious (in a good way!) you could also rotate through a slide deck with several types of prompts for thought/discussion…

## Fast networking

Students interview each other on the topic of the upcoming lesson (maybe there could be an overall question or prompt that they are trying to find information for?) and visualise the results. This activates prior knowledge and, through the visualization bit, also puts different snippets of knowledge into relation with each other.

What’s different from “talk to your neighbour about this for a minute”? The clear roles of interviewer/interviewee facilitate a conversation more easily, especially with students that don’t know each other well and/or are shy.

## Living statistics (sometimes also called sociometry)

This is a method that I have used a lot but had forgotten about now that things have been online for me in what feels like forever: asking questions and assigning spaces in the room for different answer options, and have students move around the room to answer them. This method is great when facilitating students getting to know each other, e.g. asking them to place themselves on an imaginary map of where they were born (without too clearly prescribing what is where, so that students need to talk to each other to figure out where to place themselves relative to each other), how much prior knowledge they have, what fields they come from in interdisciplinary courses, … It’s usually easier to remember who stood close by in response to a certain question than to remember everybody that had put their hand up, and especially in large classes where students don’t know each other yet, that is really helpful!

## Place mat

For the “place mat” method, three or four students sit around a table together and simultaneously write their thoughts on a given question on a large sheet of paper, each in their own corner. After a while they then compile their thoughts into common notes in the middle of the piece of paper. Those common thoughts are then later shared with the whole group.

I really like this method because I am a big fan of note-taking, both to facilitate individual thinking as well as in group discussions. When I teach virtually, I often use a shared google slides document, in which each group is taking notes on their own slide. This is great for several reasons: a) students take notes so no ideas get lost between when they talk about it and later present it to the large group, b) I can “spy” on the groups’ progress and adjust the length of breakout sessions without interrupting groups by popping in on them, c) I get an idea of what they are discussing and can prepare a strategy for how I want to bring the points from different groups up in the following discussion with the whole group.

That’s it for today! We’ll continue next #TeachingTuesday with “methods to acquire knowledge”!

What other methods do you like for active starts of your lessons?

# Taxonomy of multiple choice questions

Examples of different kinds of multiple choice questions that you could use.

Multiple choice questions are a tool that is used a lot with clickers or even on exams, but they are especially on my mind these days because I’ve been exposed to them on the student side for the first time in a very long time. I’m taking the “Introduction to evidence-based STEM teaching” course on coursera, and taking the tests there, I noticed how I fall into the typical student behavior: working backwards from the given answers, rather than actually thinking about how I would answer the question first, and then looking at the possible answers. And it is amazing how high you can score just by looking at which answer contains certain key words, or whether the grammatical structure of the answers matches the question… Scary!

So now I’m thinking again about how to ask good multiple choice questions. This post is heavily inspired by a book chapter that I read a while ago in preparation for a teaching innovation: “Teaching with Classroom Response Systems – creating active learning environments” by Derek Bruff (2009). While you should really go and read the book, I will talk you through his “taxonomy of clicker questions” (chapter 3 of said book), using my own random oceanography examples.

I’m focusing here on content questions in contrast to process questions (which would deal with the learning process itself, i.e. who the students are, how they feel about things, how well they think they understand, …).

Content questions can be asked at different levels of difficulty, and also for different purposes.

Recall of facts

In the most basic case, content questions are about recall of facts on a basic level.

Which ocean has the largest surface area?

A: the Indian Ocean

B: the Pacific Ocean

C: the Atlantic Ocean

D: the Southern Ocean

E: I don’t know*

Recall questions are more useful for assessing learning than for engaging students in discussions. But they can also be very helpful at the beginning of class periods or new topics to help students activate prior knowledge, which will then help them connect new concepts to already existing concepts, thereby supporting deep learning. They can also help an instructor understand students’ previous knowledge in order to assess what kind of foundation can be built on with future instruction.

Conceptual Understanding Questions

Answering conceptual understanding questions requires higher-level cognitive functions than purely recalling facts. Now, in addition to recalling, students need to understand concepts. Useful “wrong” answers are typically based on student misconceptions. Offering typical student misconceptions as possible answers is a way to elicit a misconception, so it can be confronted and resolved in a next step.

At a water depth of 2 meters, which of the following statements is correct?

A: A wave with a wavelength of 10 m is faster than one with 20 m.

B: A wave with a wavelength of 10 m is slower than one with 20 m.

C: A wave with a wavelength of 10 m is as fast as one with 20 m.

D: I don’t know*

It is important to ask yourself whether a question actually is a conceptual understanding question or whether it could, in fact, be answered correctly purely based on good listening or reading. Is a correct answer really an indication of a good grasp of the underlying concept?

Classification questions

Classification questions assess understanding of concepts by having students decide which answer choices fall into a given category.

Which of the following are examples of freak waves?

A: The 2004 Indian Ocean Boxing Day tsunami.

B: A wave with a wave height of more than twice the significant wave height.

C: A wave with a wave height of more than five times the significant wave height.

D: The highest third of waves.

E: I don’t know*

Or asked in a different way, focussing on which characteristics define a category:

Which of the following is a characteristic of a freak wave?

A: The wavelength is 100 times greater than the water depth

B: The wave height is more than twice the significant wave height

C: Height is in the top third of wave heights

D: I don’t know*

This type of questions is useful when students will have to use given definitions, because they practice to see whether or not a classification (and hence a method or approach) is applicable to a given situation.

Explanation of concepts

In the “explanation of concepts” type of question, students have to weigh different definitions of a given phenomenon and find the one that describes it best.

Which of the following best describes the significant wave height?

A: The significant wave height is the mean wave height of the highest third of waves

B: The significant wave height is the mean over the height of all waves

C: The significant wave height is the mean wave height of the highest tenth of waves

E: I don’t know*

Instead of offering your own answer choices here, you could also ask students to explain a concept in their own words and then, in a next step, have them vote on which of those is the best explanation.

Concept question

These questions test the understanding of a concept without, at the same time, testing computational skills. If the same question was asked giving numbers for the weights and distances, students might calculate the correct answer without actually having understood the concepts behind it.

To feel the same pressure at the bottom, two water-filled vessels must have…

A: the same height

B: the same volume

C: the same surface area

D: Both the same volume and height

E: I don’t know*

Or another example:

If you wanted to create salt fingers that formed as quickly as possible and lasted for as long as possible, how would you set up the experiment?

A: Using temperature and salt.

B: Using temperature and sugar.

C: Using salt and sugar.

D: I don’t know.*

Ratio reasoning question

Ratio reasoning questions let you test the understanding of a concept without testing maths skills, too.

You are sitting on a seesaw with your niece, who weighs half of your weight. In order to be able to seesaw nicely, you have to sit…

A: approximately twice as far from the mounting as she does.

B: approximately at the same distance from the mounting as she does.

C: approximately half as far from the mounting as she does.

D: I don’t know.*

If the concept is understood, students can answer this without having been given numbers to calculate and then decide.

Another type of question that I like:

Which of the following sketches best describes the density maximum in freshwater?

If students have a firm grasp of the concept, they will be able to pick which of the graphs represents a given concept. If they are not sure what is shown on which axis, you can be pretty sure they do not understand the concept yet.

Application questions

Application questions further integrative learning, where students bring together ideas from multiple sessions or courses.

Which has the biggest effect on sea surface temperature?

A: Heating through radiation from the sun.

B: Evaporative cooling.

C: Mixing with other water masses.

D: Radiation to space during night time.

E: I don’t know.*

Students here have the chance to discuss the effect sizes depending on multiple factors, like for example the geographical setting, the season, or others.

Procedural questions

Here students apply a procedure to come to the correct answer.

The phase velocity of a shallow water wave is 7 m/s. How deep is the water?

A: 0.5 m

B: 1 m

C: 5 m

D: 10 m

E: 50 m

F: I don’t know*

Prediction question

Have students predict something to force them to commit to once choice so they are more invested in the outcome of an experiment (or even explanation) later on.

Which will melt faster, an ice cube in fresh water or in salt water?

A: The one in fresh water.

B: The one in salt water.

C: No difference.

D: I don’t know.*

Or:

Will the radius of a ball launched on a rotating table increase or decrease as the speed of the rotation is increased?

A: Increase.

B: Decrease.

C: Stay the same.

D: Depends on the speed the ball is launched with.

E. I don’t know.*

Critical thinking questions

Critical thinking questions do not necessarily have one right answer. Instead, they provide opportunities for discussion by suggesting several valid answers.

Iron fertilization of the ocean should be…

A: legal, because the possible benefits outweigh the possible risks

B: illegal, because we cannot possibly estimate the risks involved in manipulating a system as complex as the ecosystem

C: legal, because we are running a huge experiment by introducing anthropogenic CO2 into the atmosphere, so continuing with the experiment is only consequent

D: illegal, because nobody should have the right to manipulate the climate for the whole planet

For critical thinking questions, the discussion step (which is always recommended!) is even more important, because now it isn’t about finding a correct answer, but about developing valid reasoning and about practicing discussion skills.

Another way to focus on the reasoning is shown in this example:

As waves travel into shallower water, the wave length has to decrease

I. because the wave is slowed down by friction with the bottom.

II. because transformation between kinetic and potential energy is taking place.

III. because the period stays constant.

A: only I

B: only II

C: only III

D: I and II

E: II and III

F: I and III

G: I, II, and III

H: I don’t know*

Of course, in the example above you wouldn’t have to offer all possible combinations as options, but you can pick as many as you like!

Choose one best answer out of several possible answers that all have their merits.

Your rosette only lets you sample 8 bottles before you have to bring it up on deck. You are interested in a high resolution profile, but also want to survey a large area. You decide to

A: take samples repeatedly at each station to have a high vertical resolution

B: only do one cast per station in order to cover a larger geographical range

C: look at the data at each station to determine what to do on the next station

In this case, there is no one correct answer, since the sampling strategy depends on the question you are investigating. But discussing different situations and which of the strategies above might be useful for what situation is a great exercise.

And for those of you who are interested in even more multiple choice question examples, check out the post on multiple choice questions at different Bloom levels.

* while you would probably not want to offer this option in a graded assessment, in a classroom setting that is about formative assessment or feedback, remember to include this option! Giving that option avoids wild guessing and gives you a clearer feedback on whether or not students know (or think they know) the answer.

# How to ask multiple-choice questions when specifically wanting to test knowledge, comprehension or application

Multiple choice questions at different levels of Bloom’s taxonomy.

Let’s assume you are convinced that using ABCD-cards or clickers in your teaching is a good idea. But now you want to tailor your questions such as to specifically test for example knowledge, comprehension, application, analysis, synthesis or evaluation; the six educational goals described in Bloom’s taxonomy. How do you do that?

I was recently reading a paper on “the memorial consequences of multiple-choice testing” by Marsh et al. (2007), and while the focus of that paper is clearly elsewhere, they give a very nice example of one question tailored once to test knowledge (Bloom level 1) and once to test application (Bloom level 3).

For testing knowledge, they describe asking “What biological term describes an organism’s slow adjustment to new conditions?”. They give four possible answers: acclimation, gravitation, maturation, and migration. Then for testing application, they would ask “What biological term describes fish slowly adjusting to water temperature in a new tank?” and the possible answers for this question are the same as for the first question.

Even if you are not as struck by the beauty of this example as I was, you surely appreciate that this sent me on a literature search of examples how Bloom’s taxonomy can help design multiple choice questions. And indeed I found a great resource. I haven’t been able to track down the whole paper unfortunately, but the “Appendix C: MCQs and Bloom’s Taxonomy” of “Designing and Managing MCQs” by Carneson, Delpierre and Masters contains a wealth of examples. Rather than just repeating their examples, I am giving you my own examples inspired by theirs*. But theirs are certainly worth reading, too!

Bloom level 1: Knowledge

At this level, all that is asked is that students recall knowledge.

Example 1.1

Which of the following persons first explained the phenomenon of “westward intensification”?

1. Sverdrup
2. Munk
3. Nansen
4. Stommel
5. Coriolis

Example 1.2

In oceanography, which one of the following definitions describes the term “thermocline”?

1. An oceanographic region where a strong temperature change occurs
2. The depth range were temperature changes rapidly
3. The depth range where density changes rapidly
5. An isoline of constant temperature

Example 1.3

Molecular diffusivities depend on the property or substance being diffused. From low to high molecular diffusivities, which of the sequences below is correct?

1. Temperature > salt > sugar
2. Sugar > salt > temperature
3. temperature > salt == sugar
4. temperature > sugar > salt

Bloom level 2. Comprehension

At this level, understanding of knowledge is tested.

Example 2.1

Which of the following describes what an ADCP measures?

1. How quickly a sound signal is reflected by plankton in sea water
2. How the frequency of a reflected sound signal changes
3. How fast water is moving relative to the instrument
4. How the sound speed changes with depth in sea water

Bloom level 3: Application

Knowledge and comprehension of the knowledge are assumed, now it is about testing whether it can also be applied.

Example 3.1

What velocity will a shallow water wave have in 2.5 m deep water?

1. 1 m/s
2. 2 m/s
3. 5 m/s
4. 10 m/s

Example 3.2

Which instrument would you use to make measurements with if you wanted to calculate the volume transport of a current across a ridge?

1. CTD
3. ARGO float
4. Winkler titrator

This were only the first three Bloom-levels, but this post is long enough already, so I’ll stop here for now and get back to you with the others later.

Can you see using the Bloom taxonomy as a tool you would use when preparing multiple-choice questions?

If you are reading this post and think that it is helpful for your own teaching, I’d appreciate if you dropped me a quick line; this post specifically was actually more work than play to write. But if you find it helpful I’d be more than happy to continue with this kind of content. Just lemme know! :-)

* If these questions were used in class rather than as a way of testing, they should additionally contain the option “I don’t know”. Giving that option avoids wild guessing and gives you a clearer feedback on whether or not students know (or think they know) the answer. Makes the data a whole lot easier to interpret for you!

# Clickers

Remember my ABCD voting cards? Here is how the professionals do audience response.

Remember my post on ABCD voting cards (post 1, 2, 3 on the topic)?

I then introduced them as “low tech clickers”. Having never worked with actual clickers then, I really really liked the method. And I still think it’s a neat way of including and activating a larger group if you don’t have clickers available. But now that I have worked with actual clickers, I really can’t imagine going back to the paper version.

So what makes clicker that much better than voting cards?

Firstly – students are truly anonymous. With voting cards nobody but the instructor sees what students picked. But having the instructor see what you pick is still a big threshold. And to be honest – as the instructor, you do tend to remember where the correct answers are typically to be found, so it is totally fair that students hesitate to vote with voting cards.

Secondly – even though you as the instructor tend to get a visual impression of what the distribution of answers looked like, this is only a visual impression. The clicker software, however, keeps track of all the answers, so you can go back after your lecture and check the distributions. You can even go back a year later and compare cohorts. No such thing is possible with the voting cards unless you put in a huge effort and a lot of time.

Third – the distribution can be visualized in real time for the students to see. While with the voting cards I always tried to tell the students what I saw, this is not the same thing as seeing a bar diagram pop up and seeing that you are one out of two students who picked this one option.

If you read German – go here for inspiration. My colleague is great with all things clicker and I have learned so much from him! Most importantly (and I wish I had known this back when I used the voting cards): ALWAYS INCLUDE THE “I DON’T KNOW” OPTION. Especially when you make students to pick an answer (as I used to do) – if you don’t give them the “I don’t know” option, all you do is force them to guess, and that can really screw up your distribution as I recently found out. But more about that later…

P.S.: If I convinced you and you are now looking for alternatives to paper voting cards but can’t afford to buy a clicker system – don’t despair. I might write a post about it alternative solutions at some point, but if you want to get a couple of pointers before that post is up, just shoot me an email…

# Introducing voting cards (post 3/3)

How do you introduce voting cards as a new method in a way that minimizes student resistance?

As all new methods, voting cards (see post on the method here, and on what kind of questions to ask here) first seem scary. After all, students don’t know what will happen if they happen to chose the wrong answer. Will they be called out on it by the instructor? Will everybody point at them and laugh? And even if they chose the correct answer, will the instructor make them explain why they chose that answer?

Some of my students in a staged photo. They are showing their favorite color to demonstrate the method for you. Thanks for posing for me!

When I introduce voting cards to a new group of students, I make sure to talk through all issues before actually using the cards. It is important to reassure the students that wrong answers will not be pointed out publicly, for example. It helps to use a very simple question that does not have right or wrong answers (“Which of these four colors is your favorite? Show me the one you like best!”) for the very first vote, so students get to experience the process without there being anything at stake. While showing their favorite color, they see that they cannot actually see their neighbors’ choices without making it very obvious (at least not in the classical lecture theatre setting that we are in, but even in other settings it is difficult). Hence their peers cannot actually see their own choice, either, without again making it very obvious.

In the picture above, students are very happy to show their votes to everybody – after all, there is no wrong answer and I asked them to pose. But this is what it typically looks like after students have gotten used to the method. During the first classes, voting usually looks more like this: Very close to the chest, held with both hands, shielding it from the neighbors.

During the first classes, voting usually looks like pictured above: Very close to the chest, held with both hands, shielding it from the neighbors.

Still there is probably going to be some resistance about committing to one answer because, after all, the instructor will still see it. But in my experience this can be overcome when the reasons for choosing the method are made sufficiently clear – that it benefits them to commit to one answer, because making thought processes explicit helps their learning. That it helps me, because I get a better feel of whether everybody understood a concept or only just the two vocal students, and whether I need to go into more detail with a concept or not. That it is a great basis for discussions.

Photo of an actual vote. In fact of the first vote after I asked them to pose for a staged photo (the one shown above). This question was clearly too easy!

After a couple of classes, voting cards are not even needed any more (although it can’t hurt to hand them out – it feels like less pressure if you could fall back on holding something up rather than speaking in public); discussion starts without having to be initiated through a voting process and subsequent questions for clarification. Also if they chose to still vote, students get much more daring in the way they hold up the cards – they stop caring about whether their peers can see what they voted for. So all in all a great technique to engage students.

# How to pose questions for voting card concept tests (post 2/3)

Different ways of posing questions for concept tests are being presented here

Concept tests using voting cards have been presented in this post. Here, I want to talk about different types of questions that one could imagine using for this method.

1) Classical multiple choice

In the classical multiple choice version, for each question four different answers are given, only one of which is correct. This is the tried and tested method that is often pretty boring.

An example slide for a question with one correct answer

However, even this kind of question can lead to good discussions, for example when it is introducing a new concept rather than just testing an old one. In this case, we had talked about different kinds of plate boundaries during the lecture, but not about the frame of reference in which the movement of plates is described. So what seemed to be a really confusing question at first was used to initiate a discussion that went into a lot more depth than either the textbook or the lecture, simply because students kept asking questions.

A twist on the classical multiple choice is a question for which more than one correct answer are given without explicitly mentioning that fact in the question. In a way, this is tricking the students a bit, because they are used to there being only one correct answer. For that reason they are used to not even reading all the answers if they have come across one that they know is correct. Giving several correct answers is a good way of initiating a discussion in class if different people chose different answers and are sure that their answers are correct. Students who have already gained some experience with the method often have the confidence to speak up during the “voting” and say they think that more than one answer is correct.

This is a bit mean, I know. But again, the point of doing these concept tests is not that the students name one correct answer, but that they have thought about a concept enough to be able to answer questions about the topic correctly, and sometimes that includes having the confidence to say that all answers are wrong. And it seems to be very satisfying to students when they can argue that none of the answers that the instructor suggested were correct! Even better when they can propose a correct answer themselves.

4) Problems that aren’t well posed

This is my favorite type of question that usually leads to the best discussions. Not only do students have to figure out that the question isn’t well posed, but additionally we can now discuss which information is missing in order to answer the question. Then we can answer the questions for different sets of variables.

One example slide for a problem that isn’t well posed – each of the answers could be correct under certain conditions, but we do not have enough information to answer the question.

For example for the question in the figure above, each of the answers could be correct during certain times of the year. During summer, the temperature near the surface is likely to be higher than that near the bottom of the lake (A). During winter, the opposite is likely the case (B). During short times of the year it is even possible that the temperature of the lake is homogeneous (C). And, since the density maximum of fresh water occurs at 4degC, the bottom temperature of a lake is often, but not inevitably, 4degC (D). If students can discuss this, chances are pretty high that they have understood the density maximum in freshwater and its influence on the temperature stratification in lakes.

5) Answers that are correct but don’t match the question.

This is a tricky one. If the answers are correct in themselves but don’t match the question, it sometimes takes a lot of discussing until everybody agrees that it doesn’t matter how correct a statement is in itself; if it isn’t addressing the point in question, it is not a valid answer. This can now be used to find valid answers to the question, or valid questions to the provided answers, or both.

This is post no 2 in a series of 3. Post no 1 introduced the method to the readers of this blog, post no 3 is about how to introduce the methods to the students you are working with.

# A, B, C or D?

Voting cards. A low-tech concept test tool, enhancing student engagement and participation. (Post 1/3)

Voting cards are a tool that I learned about from Al Trujillo at the workshop “teaching oceanography” in San Francisco in 2013. Basically, voting cards are a low-tech clicker version: A sheet of paper is divided into four quarters, each quarter in a different color and marked with big letters A, B, C and D (pdf here). The sheet is folded such that only one quarter is visible at a time.

A question is posed and four answers are suggested. The students are now asked to vote by holding up the folded sheet close to their chest so that the instructor sees which of the answers they chose, whereas their peers don’t.

Voting cards are sheets of paper with four different colors for the four quarters, each marked with a big A, B, C or D.