Summaries of three inspiring articles on assessment (Wiliams, 2011), workload (D’Eon & Yasinian, 2021), and quality (Harvey & Stensaker, 2008)

In my group of academic development colleagues at LTH, we just opened up an internal call for the one (or two, or three, or more) articles that are most influential for our current thinking. And I want to make sure I read them all, so here are summaries of the first (and, as of just now, only) three. I’ll add to the list if and when I receive more… And I can say that they are all inspiring in very different ways, but now I am done for today! :-D

First up, sent by Torgny:

“What is assessment for learning?” by Wiliam (2011)

Wiliam (2011) Is an overview article over how the term “assessment for learning” developed and changed meaning over time, and relates this to classroom practice. Here is my short summary:

In contrast to assessment that is done after the learning process, “assessment for learning” and “formative assessment” are terms used for processes that guide learning towards intended goals during the learning process. These concept developed, because it was recognised that the same instruction does not lead to the same results independent of who the learners are, and that failure to learn is not necessarily the learners’ fault. Bloom stated at some point that learners would be able to learn better if they received “feedback” and “correctives”.

“Feedback” as a term is not very useful, however: Feedback is only the information about a gap between a desired and an actual state, it does not, in itself, do anything to close a gap. Specifically, it does not necessarily contain any kind of information for how the learner can improve their learning. So in that sense, giving feedback is pointless.

There has been a whole lot of research on assessment and classroom learning that is summarized next. Basically: There are a bunch of conflicting results. One thought that I found interesting, though, is that there are basically four eight ways to respond to feedback: change behaviour to reach the goal, modify the goal, abandon the goal, or reject the feedback. Only two responses, increasing effort and finding a bigger goal, will actually achieve more learning than if no feedback was given. So how do we get students to pick one of these two choices? Wiliam (2021) quotes Shute (2008)’s guidelines to enhance learning (make feedback specific, give clear guidance for how to improve, make it about the task, not the learner, make it as simple as possible while keeping it as complex as neccessary, …) and on the timing (for procedural learning give feedback immediately).

After more elaboration on what different authors mean by formative assessment, Wiliams (2021) refers back to their definition from 2009:

“Practice in a classroom is formative to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited. (Black & Wiliam, 2009, p. 9)”

Two features come out as especially important:

  1. The assessment should not just show a gap between what should be and what is, but also contain information that can help improve instruction
  2. The learner must act on it in a way that supports their learning: Students can take one of two pathways: The “growth pathway” (with the goal to learn more), or the “well-being pathway” (with the goal to minimize harm, which can include more learning, but can also mean disengaging).

If we can design assessment in such a way to meet both criteria, it is likely to lead to more student engagement and better learning outcomes.

The next two articles were sent by Per:

“Student work: a re-conceptualization based on prior research on student workload and Newtonian concepts around physical work” by D’Eon & Yasinian (2021)

The article by D’Eon & Yasinian (2021) begins by making a very strong case for the importance of considering student workload, by giving an overview over all the negative effects a too high workload has on students: It does not only hinder their learning and kill their motivation, it also makes plagiarism and other forms of cheating more likely and has effects on students’ social life and health. But there is no good definition of what workload actually means: sometimes it’s defined in “objective” terms like for example as time-on-task (but who records and reports, and how “on task” does something have to be to count?), other times in “subjective” terms, for example perceived effort. In any case, gut feeling tells us that there is a link between how much effort students put into something, and how well they succeed. D’Eon & Yasinian (2021) suggest a link, based on Newton:

SW = Eff × Ach

where SW is student work (not workload, to point out that it is the work a student actually does, not what is imposed on them by a course), Eff is effort, and Ach is achievement. In this equation, effort is then seen as analogue the force that acts on the course requirements, the mass, to move them to a given achievement, over a given distance.

But what exactly is student academic effort? D’Eon & Yasinian (2021) identify four different, but interrelated, domains: cognitive (how much cognitive capacity is required? This also depends on prior knowledge, for example), physical (being awake, getting to- and from campus, buying required textbooks, …), psychological (organising themselves, keeping up motivation, …), and social (working in groups etc). Each of the four dimensions obviously relies on resources that are unequally distributed between students, for example cognitive ability, health, high intrinsic motivation, a good network. Institutions can help making resources more equally available to all students by, for example, providing materials or coaching for free to everybody, or making deadlines flexible so they can be adapted to work schedules. This model is really helpful to show that considering all four domains, and what resources we could provide students with so their effort ends up in the dimesion we want it to end up (probably the cognitive one), is directly relevant to student work, and that the course load (as specified in the syllabus) and course demands (as experienced by students) are two very different beasts. And the way this is presented speaks to my physical oceanographer’s heart :)

So now on to article no 3:

“Quality culture: Understandings, boundaries and linkages” by Harvey & Stensaker (2008)

I summarised this article before (good thing I have a blog as an external memory), but here is my summary of my summary: It is really helpful to clarify what we actually mean when we talk about “quality”, because it can mean very different things to very different people! For example, is striving for quality about being exceptional in what we do? Or is it that we are consistently meeting pre-defined measures? Or that outcomes meet a purpose? Or do we mean value for money? Or are we talking about improvement over time?

When then wanting to improve quality of whatever flavour, it is important to consider the quality culture in which this is supposed to happen. A 2×2 matrix of strong-to-weak group control and strong-to-weak external rules leads to four types of quality culture: responsive (strong group control, strong external rules), where impulses from the outside are mostly taken up as opportunities and only sometimes are perceived as imposed and constricting ownership and degrees of freedom; reactive (weak group-control, strong external rules), where measures of quality are imposed from the outside and are dealt with when necessary, but not in an integrated way; regenerative (strong group control, weak external rules), where external expectations are incorporated as far as they are perceived useful to further the internal agenda; and reproductive (weak group control, weak external rules), where the status quo is maintained with as little influence from the outside as possible. Quality insurance strategies are of course most likely to be successful if they work with the existing quality culture.

So those are the three “key articles” so far. Any comments? What other article should we include in the list? Please let me know! :-)

D’Eon, M. & Yasinian, M. (2021): Student work: a re-conceptualization based on prior research on student workload and Newtonian concepts around physical work, Higher Education Research & Development, DOI: 10.1080/07294360.2021.1945543

Harvey, L., & Stensaker, B. (2008). Quality culture: Understandings, boundaries and linkages. European journal of Education, 43(4), 427-442.

Wiliam, Dylan. “What is assessment for learning?.” Studies in educational evaluation 37.1 (2011): 3-14.

One thought on “Summaries of three inspiring articles on assessment (Wiliams, 2011), workload (D’Eon & Yasinian, 2021), and quality (Harvey & Stensaker, 2008)

  1. Pingback: Summaries of two more inspiring articles recommended by my colleagues: On educational assessment (Hager & Butler, 1996) and on variables associated with achievement in higher ed (Schneider & Preckel, 2017) - Adventures in Oceanography and Teaching

Leave a Reply