Eight criteria for authentic assessment; my takeaways from Ashford-Rowe, Herrington & Brown (2014)

1 Reply

“Authentic assessment” is a bit of a buzzword these days. Posing assessment tasks that resemble problems that students would encounter in the real world later on sounds like a great idea. It would make learning, even learning “for the test”, so much more relevant and motivating, and it would prepare students so much better for their lives after university. So far, so good. But what does authentic assessment actually mean when we try to design it, and is it really always desirable to its fullest meaning?

Ashford-Rowe, Herrington & Brown (2014) have reviewed the literature and, through a process of discussion and field testing, came up with eight critical elements of authentic assessment, which I am listing as the headers below, together with my thoughts on what it would mean to implement them.

1. To what extent does the assessment activity challenge the student?

For assessment to be authentic, it needs to mirror real-world problems that are not solved by just reproducing things that students learned by heart, but by creating new(-to-the-student) solutions, including analysing the task in order to choose the relevant skills and knowledge to even approach the task with.

This clearly sounds great, but as an academic developer, my mind goes directly to how this is aligned with both learning outcomes and learning activities. My fear would be that it is easy to make the assessment way too challenging if the focus is on authentic assessment alone and students are not practising on very similar tasks before already.

2. Is a performance, or product, required as a final assessment outcome?

Real-world problems require actual solutions, often in form of a product that addresses a certain need. In an assessment context, we need to balance the ideal of a functional product that provides a solution to a problem with wanting to check whether specific skills have been acquired. I.e. what would happen if students found a solution that perfectly solved the problem they were tasked with, but did it in some other way without demonstrating the skills we had planned on assessing? Would that be ok, or do we need to provide boundary conditions that make it necessary or explicit that students are to use a specific skill in their solutions?

This facet of authentic assessment strongly implicates that assessment cannot happen only in a matter of hours in a written closed-book exam, but requires more time and likely different formats (even though many of my authentic products in my job actually are written pieces).

3. Does the assessment activity require that transfer of learning has occurred, by means of demonstration of skill?

Transfer is a highest level learning outcome in both Bloom’s and the SOLO taxonomy, and definitely something that students should learn — and we should assess — at some point. How far the transfer should be, i.e. if skills and knowledge really need to be applied in a completely different domain, or just on a different example, and where the boundary between those two is, is open for debate though. And we can transfer skills that we learned in class to different contexts, or we can bring skills that we learned elsewhere into the context we focussed on in class. But again we need to keep in mind that we should only be assessing things that students actually had the chance to learn in our courses (or, arguably, in required courses before).

4. Does the assessment activity require that metacognition is demonstrated?

It is obviously an important skill to acquire to self-assess and self-direct learning, and to put it into the bigger context of the real world and ones own goals. If assessment is to mirror the demands of real-world tasks after university, reflecting on ones own performance might be a useful thing to include. But this, again, is something that clearly needs to be practice and formative feedback before it can be used in assessment.

5. Does the assessment require a product or performance that could be recognised as authentic by a client or stakeholder? (accuracy)

This point I find really interesting. How close is the assessment task to a real-world problem that people would actually encounter outside of the classroom setting? Before reading this article, that would have been my main criterion for what “authentic assessment” means.

6. Is fidelity required in the assessment environment? And the assessment tools (actual or simulated)?

Continuing the thought above: If we have a task that students might encounter in the real world, do we also provide them with the conditions they would encounter it in? For an authentic assessment situation, we should be putting students in an authentic(-ish) environment, where they have access to the same tools, the same impressions of their surroundings, the same sources of information as one would have if one was confronted with the same problem in the real world. So we need to consider how/if we could even justify for example not letting students use internet searches or conversations with colleagues when working on the task!

7. Does the assessment activity require discussion and feedback?

This follows nicely on my thoughts on the previous point. In the real world, we would discuss and receive feedback while we are working on solving a problem. If our assessment is to be authentic, this also needs to happen! But do we also assess this aspect (for example in the reflection that students do in order to demonstrate metacognition, or by assessing the quality of the discussion and feedback they give to each other), or do we require it without actually assessing it, or do we just create conditions in which it is possible and beneficial to discuss and give&receive feedback, without checking whether it actually occurred? Also, who should the students discuss with and get feedback from: us, their peers, actual authentic stakeholders? Literature shows that peer feedback is of comparable quality to teacher feedback, and that students learn both from giving and receiving, so maybe including a peer-feedback loop is a good idea. Real stakeholders would surely be motivating, but depending on the context that might be a little difficult to arrange.

8. Does the assessment activity require that students collaborate?

Collaboration is of critical importance in the real world. (How) do we include it in assessment?

I really enjoyed thinking through these eight critical elements of authentic assessment, and it definitely broadened my understanding of authentic assessment considerably, both in terms of its potential and the difficulties to implement it. What are your thoughts?

Ashford-Rowe, K., Herrington, J., & Brown, C. (2014). Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39(2), 205-222.

Dealing with collective tragedy as a teacher, according to my reading of Huston & DiPetro (2007)

Leave a reply

When “collective tragedy”, for example terrorism, or natural catastrophes like hurricanes and floods, or pandemics, happens, it is difficult to decide whether, and how, to act as teacher. Should we acknowledge the event, and if so to what extent and in what form, or is it better to stick to business as usual to give students a sense of normalcy and stability? Collective tragedies often involve not just dealing with traumatic losses and experiences, which would be difficult to talk about in itself, but often also relate to controversial topics of politics, or culture, or religion, which might make us hesitant to bring the topic up in class.

Looking at how students experienced their teachers’ reactions to 9/11, Huston & DiPietro (2007) recommend to “do something, just about anything”.

What type of events “deserve” a reaction?

Criteria that Huston & DiPietro (2007) identify for what should be addressed in the classroom include magnitude and scale of events (so for example national events that are covered broadly in the media), how close something happened to the campus (directly on campus, same city), how likely it is to impact students (or their friends and families) directly (what they don’t mention but I am thinking about here are also international students that are affected by events in other countries!), and how likely students are to identify with the victims (maybe due to victims being the same age or being involved in the same practice of a sport or subject).

When in doubt whether or not to react to something, Huston & DiPietro (2007) recommend looking out for “situational cues”: Do students seem affected by an event, which might become visible for example by student-led events on the campus, like demonstrations or vigils? Is the event on mine, the teacher’s, mind so much that I have difficulties shaking it off when I start class? Do students even bring it up to you, or do you overhear them talking about the topic? Then it is probably worth addressing.

What kinds of reactions are helpful?

After 9/11, 11% of the faculty that responded to a specific study said they did not speak of the attacks at all. But of those other faculty that did react, reactions ranged from really low-effort, small scale actions, like a minute of silence, large efforts like making the attacks topic of (parts of) the course. Generally, there was a lot of confusion about their role as a teacher, and what that meant for what they should do, and still afterwards what they should have done, both for instructors who had reacted and those that didn’t.

While it is unclear what effects reactions or the lack thereof actually did have on students, this is what students report:

  • Completely ignoring was perceived as the instructor not taking the situation seriously enough, not caring about how the students did, and generally as “terrible”.
  • 78% of students report that they appreciated when instructors suggested ways to become active to, in some small way, make the situation better (like where to donate blood or where to give to charity), which is called “problem-focussed coping” and has been found to often be effective at reducing stress by giving people agency.
  • 69% of the students found being offered extensions of deadlines, or other alternative ways to deal with the workload of the course, helpful to counteract the mental load of dealing with a traumatic event
  • !! Acknowledging the event but not adjusting anything in class was perceived as really unhelpful !!

In addition to the students’ self-reports, clinical research found psychological interventions, for example journal writing, or an active approach to get involved in community efforts to help others, effective ways to deal with trauma.

So in summary, my impression is that it is (as always!) important to show humanity as a teacher: Acknowledging that something terrible is happening, showing that we care about students by asking them how they are doing and trying to lessen their burdens as far as that is within our powers, and giving them the opportunity to reflect on what is going on and possibly digest it by scientifically approaching the topic in the frame of the course, as well as supporting them in dealing with the real-world effects of the tragedy by suggesting ways to become active for the benefit of the community. That is quite a tall order, but I think it is good to keep in mind that it is better to do anything positive, no matter how small, than doing nothing.

Huston, T. A., & DiPietro, M. (2007). 13: In the Eye of the Storm: Students’ Perceptions of Helpful Faculty Actions Following a Collective Tragedy. To improve the academy, 25(1), 207-224.

Microaggressions: How intent and impact don’t always go together.

Leave a reply

I’ve recently started including the topic of microaggressions in my academic development workshops, and here is one reflection on the topic (including the super helpful sandals&boots-analogy by Presley Pizzo). I initially wrote this for a newsletter to all teachers at my faculty, but then I also wrote a second – much more hands-on-“three-things-to-do” – version, which was ultimately the one that was deemed more fitting for the target audience. But I still like this one, so here I’m giving you both. So without further ado:

“That’s not what I said, and it’s definitely not what I meant, and do you really think someone like me would do such a thing?”

Continue reading →

My notes from last week’s ICED22 conference

Leave a reply

Starting out with a wave watching picture from my walk before the first day of the conference. When in Aarhus, I had to get in the 90 minute walk before the first presentation and visit the infinite bridge!

Last week, I attended my first ICED, The International Consortium for Educational Development’s, conference. And since I took an insane amount of notes which are almost unreadable now and will become completely unreadable as I forget the context, I thought I’d go through them & write them up nicely (mostly for my own benefit, but if you are curious to read about my experience, you are welcome to do so! :-)).

Continue reading →

Reducing bias and discrimination in teaching: an annotated, incomplete — WORK IN PROGRESS! — list of references

Leave a reply

Already at the time of posting, I have added to my to-read list for an updated version of this post. Please let me know of any additional literature I should include, and of any other comments you might have! As it says in the title, this work is incomplete and in progress!

“The rights perspective, which is also called the justice or democracy perspective, means that everyone should have the right to the same education and career. Gender – or other “dividing categories” – should therefore not affect career opportunities. The research system perspective emphasizes the importance of finding the best people for research, while the socio-economic perspective points out that it is a waste of public resources not to make use of the most suitable candidates. Finally, the epistemological perspective focuses on the fact that increased diversity among researchers leads to greater diversity and thus to better quality in research.” (translated after Schnaas, 2011)

.There are many reasons for why we should strive to reduce bias in academia. This is a collection of references I curated for their relevance regarding creating conditions in which students can focus on learning rather than their gender, race, sexuality, disability, … and where biases regarding student identities are reduced. This document is incomplete (the focus right now is very much on gender; but I am assuming that a lot of the findings are probably transferrable to other minorities, and I will look into that literature in the future).

.To make this collection relevant in the context of my work at LTH, I have included articles from the “LTHs pedagogiska inspirationskonferens” data base. I used the search terms (on 1.4.2022) “bias” (0 results), diskriminering (1), discrimination (1), “gender” (6), “genus” (1), “kön” (2), mångfald (0), UDL (2). Some of these articles were not relevant to this topic, others included several of the search terms. In the text, sources from LTH are highlighted as such, and are linked to directly from the text.

.The intent of this document is not that everything mentioned here be included in all academic development courses I teach, rather it gives us “aces up our sleeves” that we can bring in if/when appropriate.

.The structure of this document follows the main topics of the “Introduction to Teaching and Learning in Higher Education” course at LTH (NOTE: THE LINKS BELOW ONLY WORK IF YOU LOOK AT THE EXPANDED VERSION OF THE BLOGPOST!):

The student
– Who are we trying to communicate with?
– LTH’s students’ stereotypes of “typical engineers” are different from how they see themselves (Soneson & Torstensson, 2013)
– Research in learning and teaching is done on a sample that might not be relevant in all contexts
— Almost exclusively WEIRD (Henrich et al., 2010)
— Often male-dominated, with a binary understanding of gender, where the achievements of men are used as the gold standard (Traxler et al., 2016)
– Gender-based discrimination does exist
— #metoo in Sweden: #akademiuppropet (Salmonsson, 2020)
— #metoo at Lund University (Agardh et al., 2020)
— #metoo at LTH (Wrammerfors, 2018)
— Almost 4 in 5 female students experience sexual harassment at least once a year and that affects their motivation to study STEM subjects (Leaper & Starr, 2019)
— Increasing awareness about gender discrimination and sexual harassments helps women realize that it is not their fault that they are being targeted (Weisgram & Bigler, 2007)
– Female and male students attribute their own performance in different ways (Beyer, 1998)
— Physical science career interest is supported by discussion of underrepresentation of women (Hazari et al., 2013)

Student learning
– Students achieve more when they believe that they can develop their abilities, and we can influence that (Yeager & Dweck, 2012)
— Even non-feedback comments can influence student mindset and performance (Smith et al., 2018)
— Activating stereotypes can trigger student underperformance (Steele, 2011)
– Intersectionality: Some students belong to several disadvantaged categories simultaneously (Phoenix & Pattynama, 2006)
– There is a tendency for teachers to explain performance based on the student’s gender (Espinoza et al., 2014)

Course design
– Everybody should be able to participate
— Making learning accessible for everyone: Universal design for learning (Brand et al., 2012)
— Decisions on accommodations due to disabilities are often biased against specific disabilities (Druckman, 2021)
– Choosing learning outcomes, materials & physical space wisely
— Learning outcomes and examples might not appeal to everyone in the same way (Stadler et al., 2000)
— Textbooks and other materials can perpetuate (and activate) stereotypes (Taylor, 1979)
— Including (narratives of) role models for everyone can balance stereotype threat (McIntyre et al., 2003)
— Make sure to include all relevant voices (“Decolonizing the curriculum”; Dessent et al., 2022)
— The physical environment influences who feels welcome and participates (Cheryan et al., 2009)

Learning activities
– Participation matters, and there is a gender gap
— The person who speaks most, not who makes the best points, emerges as leader (MacLaren et al., 2020)
— There is a gender gap in participation in Scandinavia (Ballen et al., 2017)
– Active learning can help reduce gaps
— Interactive engagement reduces gender gap in physics (Lorenzo et al., 2006)
— “Reductions in achievement gaps only occur when course designs combine deliberate practice with inclusive teaching” (Aguillon et al., 2020; Theobald et al., 2020)
— Women like the connections provided by PBL (Reynolds, 2003)
– Students working in small groups
— Optimal group size for (physics) problem solving is three (Heller & Hollabaugh, 1992)
— High-ability groups don’t always perform best (Heller & Hollabaugh, 1992)
— Male students ignore female students’ input to their own detriment (Heller & Hollabaugh, 1992)
— When assigning groups, cluster minorities rather than stretching them across as many groups as possible (Stoddard et al., 2020)
— “Women-only exercise groups” sometimes recommended by teachers at LTH
– Focussed sessions (interventions) can help decrease achievement gaps
— Help students see that they belong in the classroom and that adversity is normal, temporary, and surmountable (Cohen et al., 2006; Hammarlund et al., 2022)
— Help students remember their values (Martens et al., 2006; Cohen et al., 2009; Mijake et al., 2010)

Communication
– Global English (Aarup Jensen et al., 2017)
– If you are new to the Swedish educational system, be aware of cultural differences (Natalle, 2012)
– It matters to students that teachers make an effort to know their names (Tip: name tents!) (Cooper et al., 2017)
– Sensitive language
— Gender-sensitive communication
— Preferred gender pronouns
— Disability-sensitive communication
— Questionable terminology and better alternatives
– When and how to address issues of diversity and inclusion in teaching (when it is not the topic of the course)
— Examples of gender and equality sensitive situations in teaching
— Interrupting microaggressions (Thurber & DiAngelo, 2018)
— Purposefully observing gender-relevant situations: “genusobservatörer” (Carstensen, 2006)
— What kind of resistance to expect when gender becomes a topic of conversation (Carstensen, 2006)
— “Harassment” is a really tricky (and potentially not helpful) concept (Carstensen, 2016)

Assessment
– It matters who gives an exam because it can activate stereotype threats (Marx & Roman, 2002; Marx & Goff, 2005)
– If you have to ask about demographics, do it after the test so you don’t activate stereotype threat (Danaher & Crandall, 2008)
– Multiple-choice questions and gender bias
— Multiple choice questions vs constructed response questions (Weaver & Raptis, 2001)
— Negative marking for multiple-choice questions (Funk & Perrone, 2016)
— Women are more conservative and timid test takers than men (Pekkarinen, 2015)

The teacher
– Your own growth-mindset matters (Hammarlund, 2022)
– Gender bias is everywhere and it is also acting against teachers
– Women have to be better than their male peers to be perceived as equal to them in academic hiring decisions (Eaton at al., 2020)
– Student evaluations of teaching are biased (Heffernan, 2021; but there are ways to use them for good nevertheless! Roxå et al., 2021)
– There is a backlash for men violating gender stereotypes (Moss-Racusin et al., 2018)
– There is documented gender bias at LTH
— Teachers at LTH have, on average, a “Slight automatic association for Male with Science and Female and with Liberal Arts” (Allanson et al., 2021)
— Some teachers at LTH write in ways that suggest that men are superior to women (Berndtsson & Thern, 2012)
– Talking about gender, race, and other prejudices and biases is difficult!
— Why it’s so hard for white people to talk about racism (DiAngelo, 2018)
— Sometimes “nothing happens” and still something happens (Husu, 2020)
— “Visibility paradox”: women can be simultaneously highly visible and invisible (Husu, 2020)
– Legal aspects: Defamation lawsuits (Damström, 2020)
– What can you do to support equality?
— “Gender mainstreaming” (European Commission, 2000)
— Women: don’t downplay the effects of gender! (Korvajärvi, 2021)
— Men: accept evidence of gender biases in STEM! (Handley et al., 2015)
— Install institutional strategies for gender equity! (Laursen & Austin, 2020)
— Be careful to not write biased letters of references! (Madera et al., 2009)

References

.

Continue reading →

The Curious Construct of Active Learning: A guest post by K. Dunnett (UiO) on Lombardi et al. (2021)

Leave a reply

‘Active Learning’ is frequently used in relation to university teaching, especially in Science, Technology, Engineering and Mathematics (STEM) subjects where expository lecturing is still a common means of instruction, especially in theoretical courses. However, many different activities and types of activities can be assigned this label. This review article examines the educational research and development literature in 7 subject areas (Astronomy, Biology, Chemistry, Engineering, Geography, Geosciences and Physics) to explore exactly what is meant by ‘active learning’, its core principles and defining characteristics.

Active Learning is often presented or described as a means of increasing student engagement in a teaching situation. ‘Student engagement’ is another poorly defined term, but is usually taken to involve four aspects: social-behavioural (participation in sessions and interactions with other students); cognitive (reflective thought); emotional and agentic (taking responsibility). In this way, ‘Active Learning’ relates to the opportunities that students have to construct their knowledge. On the other hand, and in relation to practice, Active Learning is often presented as the antithesis of student passivity and traditional expository lecturing in which student activity is limited to taking notes. This characterisation is related the behaviour of students in a session.

Most articles and reviews reporting the positive impact of Active Learning on students’ learning don’t define what Active Learning is. Instead, most either list example activities or specify what Active Learning is not. This negative definition introduces an apparent dichotomy which is not as clear as it may initially appear. In fact, short presentations are an important element of many ‘Active Learning’ scenarios: it is the continuous linear presentation of information that is problematic. Most teaching staff promote interactivity and provide opportunities for both individual and social construction of knowledge while making relatively small changes to previously presentation-based lectures.

That said, the amount of class time in which students are interacting directly with the material does matter. One example of measurement of the use and impact of Active Learning strategies (or activities that require students to interact with the material they are learning) in relation to conceptual understanding of Light and Spectroscopy found that high learning gains occur when at least 25% of scheduled class time is spent by students on Active Learning strategies. Moreover, the quality of the activities and their delivery, and the commitment of both students and staff to their use, are also seen as potentially important elements in achieving improved learning.

In order to develop an understanding of what Active Learning actually means, groups in seven disciplinary areas reviewed the discipline-specific literature, and the perspectives were then integrated into a common definition. The research found that presentations of Active Learning in terms of either students’ construction of knowledge via engagement, or in contrast to expository lecturing were used within the disciplines, although the discipline-specific definitions varied. For example, the geosciences definition of Active Learning was:

”Active learning involves situations in which students are engaged in the knowledge-building process. Engagement is manifest in many forms, including cognitive, emotional, behavioural, and agentic, with cognitive engagement being the primary focus in effective active learning,”

while the physics definition was that:

”Active learning encompasses any mode of instruction that does not involve passive student lectures, recipe labs, and algorithmic problem solving (i.e., traditional forms of instruction in physics). It often involves students working in small groups during class to interact with peers and/or the instructor.”

The composite definition to which these contributed is that:

”Active learning is a classroom situation in which the instructor and instructional activities explicitly afford students agency for their learning. In undergraduate STEM instruction, it involves increased levels of engagement with (a) direct experiences of phenomena, (b) scientific data providing evidence about phenomena, (c) scientific models that serve as representations of phenomena, and (d) domain-specific practices that guide the scientific interpretation of observations, analysis of data, and construction and application of models.”

The authors next considered how teaching and learning situations could be understood in terms of the participants and their actions (Figure 1 of the paper). ‘Traditional, lecture-based’ delivery is modelled as a situation where the teacher has direct experience of disciplinary practices, access to data and models, and then filters these into a simplified form presented to the students. Meanwhile, in an Active Learning model students construct their knowledge of the discipline through their own interaction with the elements of the discipline: its practices, data and models. This knowledge is refined through discussion with peers and teaching staff (relative experts within the discipline), and self-reflection.

The concluding sections remark on the typical focus of Discipline Based Educational Research, and reiterate that student isolation (lack of opportunities to discuss concepts and develop understanding) and uninterrupted expository lecturing are both unhelpful to learning, but that ”there is no single instructional strategy that will work across all situations.”

The Curious Constrauct of Active Learning
D. Lombardi, T. F. Shipley and discipline teams.
Psychological Science in the Public Interest. 2021, 22 (1) 8-43
https://doi.org/10.1177%2F1529100620973974

Catching up on my reading of Trigwell and Prosser et al. (1996, 1999, 2000)

Leave a reply

There is a whole body of “historical” research by Trigwell and Prosser and co-authors that I’ve successfully ignored until now, but that is being referenced in conversations around me so much that I cannot longer ignore it. So I read a couple of articles that — I hope — give me a good overview over their body of work, which I’ll summarise here.

Continue reading →