Tag Archives: co-creation

Co-creating rubrics? Currently reading Fraile et al. (2017)

I’ve been a fan of using rubrics — tables that contain assessment criteria and a scale of quality definitions for each — not just in a summative way to determine grades, but in a formative way to engage students in thinking about learning outcomes and how they would know when they’ve reached them. Kjersti has even negotiated rubrics with her class, which she describes and discusses here. And now I read an article on “Co-creating rubrics: The effects on self-regulated learning, self-efficacy and performance of establishing assessment criteria with students” by Fraile et al. (2017), which I will summarise below.

Fraile et al. (2017) make the argument that — while rubrics are great for (inter-)rater reliability and many other reasons, students easily perceive them as external constraints that dampen their motivation and might lead to shallow approaches to learning, not as help for self-regulated deep learning. But if students were involved in creating the rubric, they might feel empowered and more autonomous because they are now setting their own goals and monitoring their performance against those, thus using it in ways that actually supports their learning.

This argument is then tested in a study on sports students, where a treatment group co-creates rubrics, whereas a control group uses those same rubrics afterwards. Co-creation of the rubric meant that after an introduction to the content by the teacher, students listed criteria for the activity and then discussed them in small groups. Criteria were then collected and clustered and reduced down to about eight, for which students, in changing groups, then produced two extreme quality definitions for each. Finally, the teacher compiled everything into a rubric and got final approval from the class.

So what happened? All the arguments above sounded convincing, however, results of the study are not as clear-cut as one might have hoped. Maybe the intervention wasn’t long enough or the group of students was too small to make results significant? But what does come out is that in thinking aloud protocols, the students who co-created the rubrics were reporting more self-regulated learning. They also performed better on some of the assessed tasks. And they reported more positive perceptions of rubrics, especially of transparency and understanding of criteria.

What do we learn from this study? At least that all indications are that co-creating rubrics might be beneficial to student learning, and that no drawbacks came to light. So it seems to be a good practice to adopt, especially when we are hoping for benefits beyond what was measured here, for example in terms of students feeling ownership for their own learning etc..


Fraile, J., Panadero, E., & Pardo, R. (2017). Co-creating rubrics: The effects on self-regulated learning, self-efficacy and performance of establishing assessment criteria with students. Studies in Educational Evaluation, 53, 69-76.

Three ways to think about “students as partners”

As we get started with our project #CoCreatingGFI, we are talking to more and more people about our ideas for what we want to achieve within the project (for a short summary, check out this page), which means that we are playing with different ways to frame our understanding of co-creation and students as partners (SaP).

For the latter, I just read an article by Matthews et al. (2019) that identifies three ways that SaP is commonly being written about. Reading this article was really useful, because it made me realise that I have been using aspects of all three, and now I can more purposefully choose in which way I want to frame SaP for each specific conversation I am having.

In the following, I am presenting the three different perspectives and commenting on how they relate to how I’ve been talking — and thinking — about SaP.

Imagining through Metaphors

Metaphors are figures of speech where a description is applied to something it isn’t literally applicable to, but where it might help to imagine a different (in this case, desired) state.

“Students as partners” as a metaphor evokes quite strong reactions occasionally, because it can be perceived as a complete loss of power, authority and significance by teachers; and likewise as too much work, responsibility, stress by students. We moved away from “students as partners” as a metaphor and towards “co-creation”, because when speaking about “students as partners”, we were constantly trying to explain who the students were partnering with, and what “partnership” would mean in practice. So while we were initially attracted to the metaphor and the philosophy behind it, it ended up not working well in our context.

Speaking about the “student voice”, on the other hand, is something that I’m still doing. To me, it implies what Matthews et al. (2019) describe: students powerfully and actively participating in conversations, and actually being heard. But they also warn that this metaphor can lead to structures in which power sharing becomes less likely, which I can also see: if we explicitly create opportunities to listen to students, it becomes easy to also create other situations in which there explicitly is no space for students.

Building on concepts

When grounding conversations on accepted concepts from the literature, it makes it a lot easier to argue for them and to make sure they make sense in the wider understanding in the field.

In our proposal for Co-Create GFI, we very explicitly build all our arguments on the concept of “communities of practice”. Maybe partly because I was in a very bad Wenger phase at around that time, but mostly because it gave us language and concepts to describe our goal (teachers working together in a community on a shared practice), because it gave us concrete steps for how to achieve that and what pitfalls to avoid.

Also in that proposal as well as in our educational column in oceanography, we use “student engagement” as the basis for the co-creation we are striving for. In our context, there is agreement that students should be engaged and that teachers should work to support student engagement, so starting from this common denominator is a good start into most conversations.

Another concept mentioned by Matthews et al. (2019) are “threshold concepts”, which isn’t a concept we have used in our own conversations about SaP, but which I found definitely helpful to consider when thinking about reactions towards the idea of SaP.

Matthews et al. (2019) point out that while building on concepts can be grounding and situating the way I describe above, it can also be disruptive.

Drawing on Constructs

Of the three ways of talking about SaP, this is the one we’ve used the least. Constructs are tools to help understand behaviour by basically putting a label on a drawer, such as identity, power, or gender. Looking at SaP through the lens of different constructs can help see reality in a different way and change our approach to it, or as Matthews et al. (2019) say: “revealing can lead to revisiting”.

I know it’s not the intention of the article, but I am wondering if taking on that lens just for fun might not reveal new and interesting things about our own thinking…


Kelly E. Matthews, Alison Cook-Sather, Anita Acai, Sam Lucie Dvorakova, Peter Felten, Elizabeth Marquis & Lucy Mercer-Mapstone (2019) “Toward theories of partnership praxis: an analysis of interpretive framing in literature on students as partners”. In: teaching and learning, Higher Education Research & Development, 38:2, 280-293, DOI: 10.1080/07294360.2018.1530199

How much of the work should the teacher vs the student do? Teaching as a dance, inspired by Joe Hoyle

This week I spent in a really interesting position: Sitting in the back of a workshop on “introduction to teaching and learning in higher education”, occasionally giving inputs, for example on microaggressions or Universal Design for Learning. And, this morning, about dance as a metaphor for learning and teaching.

I first came across this metaphor in Joe Hoyle’s blogpost on “how much of the work should you do?”. In there, he argues that “in a dance, both parties need to do half of the work but one party does have to lead. Likewise, in a class, both parties need to do half of the work but one party does have to lead. As the teacher, you are the one who has to lead. And, it is that leading that will encourage your students to get up and do their half of the work so that the class will go beautifully well every single class session”. For me, this metaphor works beautifully, not only about how much “work” people should put in, but also that there are different skills involved in teaching/leading and learning/following: the most excellent lead can only do so much if there isn’t a willingness to (learn how to) follow, and likewise the best follower cannot do much without a strong lead (or they might eventually even start leading themselves out of the follower role. I actually took up Lindy Hop, where you change partners all the time, with the explicit goal of training to not take over the lead but let myself be led by people who aren’t very good at leading, and adapting to different styles of (not) leading, hoping that I could transfer that into my professional life. Worked only so-so ;-)).

What I also really like is that in dance, it is becoming much more common that lead and follower switch roles — and this is where I see big potential to expand the metaphor towards co-creating learning. The lead can give the follower the chance to do “turns and stuff” (here my language to talk about dance in English is breaking down) by themselves, which might be compared to giving students a little choice, for example letting students do think-pair-share, where they get the opportunity to do something by themselves for a little while, but in a safe and controlled environment (see lower levels on our “co-creating learning in oceanography” framework). As we move higher up in the framework, we give students more freedom, but also more responsibility, until at the very top, we might actually consider giving up the lead and “just” follow the students’ lead. Which, again, doesn’t say about the amount of work or skill that goes into either learning or teaching, just about who takes on the lead.

So yeah, I think this is a really nice metaphor for teaching and learning :)

P.S.: For a beautiful example of how both leading and following takes enormous skill, check out the youtube video below.

Funded: Our project on “co-creation to promote active learning and communities of practice”

This week, we got super exciting news: Kjersti‘s and my proposal to the active learning call by the Norwegian Directorate for Higher Education and Competence (HK-dir) got funded (perfect timing, since our article on co-creating learning on oceanography was also published this week!)!

Below I’m sharing the translation of an interview that Kjersti gave to the MatNat faculty, but here is a quick overview over what we are planning to do:

Co-creation to promote active learning and communities of practice

The project’s goal is to use “Co-creation to promote active learning and communities of practice” at the Geophysical Institute at the University of Bergen. We work towards this goal in four work packages (called AP (“arbeidspakke”) in my cheesy illustration below):

In many courses at GFI, the seeds of co-creation are in place and being cultivated already. Our AP1 is about supporting and strengthening those efforts by evaluating and iteratively improving them in some specific courses, in order to gain more experience at our institution and create pilot projects that can serve as proof of concept and that we and others might learn from. In AP2, we help ground those efforts by creating supportive boundary conditions at GFI in terms of looking at how the organisation is structured, whether there are places where student voices could be elevated, and whether the administrative framework could better support co-creation at an institutional level. AP3 is then about engaging more and more teachers and students in other courses in co-creation, and supporting this development by creating meeting places and conversations about the topic, and supporting evaluation and discussion of results. Lastly, we are not doing this alone: AP4 brings together expert advice we are receiving as well as our efforts to share what we are learning. We have the support of the iEarth community, and specifically an advisory board with internationally renowned experts on co-creation and leading academic change processes to help us. As our efforts flower and bear fruit, we will produce a range of publications, infographics, “how-to guides” and many other formats to share our learnings with both the scientific community and interested practitioners.

We are super excited to start working on this with our great colleagues at GFI and within iEarth, and most importantly with our students!

If you are curious about our thoughts on how to get started with co-creating in oceanography (or any other subject, really), Kjersti and I just published an article with some really easy and then some a little more advanced examples (Glessmer & Daae, 2021).


Kjesti’s interview

Kjersti gave an interview to the MatNat faculty who wrote an article about our project. Here is the translation of the questions she was asked and the answers she gave:

How did you come up with the idea for this project?

The Geophysical Institute (GFI) is a partner in the center of excellence iEarth. Together with Mirjam Glessmer (co-author, and Adjunct Associate Professor in iEarth), I have had the opportunity to participate in many discussions with inspiring researchers in both geosciences and education-related research fields. We quickly got in touch with Catherine Bovill and Torgny Roxå (both Adjunct Associate Professors in iEarth) and the Geoscience Education working group at the University of Oslo. All their expertise in the field of co-creation and changing academic cultures fit perfectly with what we want to achieve at GFI. The application therefore was inspired by, and builds on, positive experiences with testing new ways of teaching in introductory courses at GFI with our colleagues there, and dialogue with colleagues and professionals from iEarth.

What is the major weakness of today’s teaching in your subject, and what do you want to improve?

Teaching at all levels, including at universities, is changing. More and more people are moving away from lectures and instead trying out new research-based teaching methods where the focus is on active involvement of students. Through instructional methods that activate the students, the students practice skills such as discussion, analysis, problem solving, sketching, etc. Research shows that students learn more and better from active forms of teaching, even if they do not necessarily experience it that way, or prefer this form of teaching. Teachers therefore appreciate support and guidance in making this transition to more and more active forms of teaching and learning in dialogue with students and leadership.

What does your focus on co-creation and community of practice mean?

Focusing on co-creation and community of practice is largely about changing the relationship between teachers and students, in order to provide students with the best environment for learning during their studies. The students are our most important “customers”. It is important that they are included in everything that happens at the department and university, that they are seen and heard, and that they are given the opportunity to influence their own studies and thus lives.

“Co-creation” encompasses a wide range of student activity and engagement, from individual activities during a single teaching session to larger activities that take place over long time, where students take on responsibility for shaping their learning together with their teachers. In co-creation activities, all participants have the right to contribute equally, but not necessarily in the same way. An increased degree of co-creation can help make teaching more inclusive and increase student engagement; at the same time, students learn more, they experience learning as more relevant, and they develop as democratic citizens. If you are curious about specific examples of co-creation activities, you can take a look at the article Mirjam Glessmer and I recently published in the magazine Oceanography (https://tos.org/oceanography/article/co-creating-learning-in-oceanography).

“Communities of practice” are groups of people who share common interests, where the participants know each other, collaborate on common goals, and develop through the exchange of knowledge. This means that teachers and students encourage and support each other in various forms of development.

So ultimately both co-creation and communities of practice are tools towards more dialogue: between students and teachers as well as within both groups individually.

What kind of responses have you received to the idea in the professional environment and from the authorities?

The very process of writing the application has affected how we think about teaching. We have had many good discussions about teaching and learning with teachers, students, and administration at both GFI and in the new network of colleagues we have found through iEarth. This has been a great help in the development of the idea and the project application, and we have received a lot of support and encouragement to move forward with our plans.

What is the common denominator for the work packages?

The common denominator for the work packages is a change in relations between students, teachers, and administration. Both students and teachers must want change and learn about how change can happen in a good way for all parties. In addition, we must put boundary conditions in place that make the changes possible at the departmental level.

What is culture created to wanting to change teaching?

Everyone involved with a university has their own opinion on how teaching at the university is or should be. This perception often reflects a traditional understanding of the role of teachers and students, where teachers must lecture on subject matter and students must acquire the subject matter and be measured by how well they can reproduce it. As long as these expectations persist, it is difficult to change the relationships between teachers and students. We want to influence teachers and students to change their focus so that teachers learn more with the students, and the students inspire the teachers. This is already happening to some extent, and through this project we want to support and strengthen this change process.

When the project is finished, what is the most important experience you will have gained?

Through a common and consistent focus on co-creation and community of practice, GFI will provide students with the best prerequisites for learning during their studies. We want to be an educational institution that helps students develop on both an academic and a personal level. This is achieved through a better dialogue between students, teachers, and administration and through a continuous development of the teaching culture at the department. When the project is completed, we hope to see a cultural change towards “more students” that is founded in the department and that continues to grow beyond the project. We also want to discuss our experiences with the higher education community and hope to inspire more people to get involved in co-creation and community of practice with the goal of improving education.

Just published: Co-Creating Learning in Oceanography

Kjersti and I just had an article published: “Co-Creating Learning in Oceanography” (Glessmer & Daae, 2021)!

In this article, we discuss ways in which to share responsibility for learning between teachers and students on a continuum from “just” actively engaging students towards fully shared responsibility, i.e. “co-creation” or “Students as Partners”. We give 13 different examples from our own praxis, starting from very easy things like how to use multiple-choice questions to promote discussion and critical thinking or giving students the choice of several examples from which they can learn the same content, and gradually work our way up to more and more interesting methods, like for example negotiating rubrics of learning outcomes with students.

The article itself is accompanied by a website where we elaborate on our 13 different examples. Check it out, and let us know what you think! And if you have any experiences with co-creating learning that you would like to share, we would love to hear from you and add a guest post on your experiences to our collection! :)


Reference:

Glessmer, M.S., and K. Daae. 2021. Co-creating learning in oceanography. Oceanography 34(4), https://doi.org/10.5670/oceanog.2021.405.

Negotiating a rubric of learning outcomes and letting students pick the format in which they show they’ve mastered the learning outcomes

I’m still inspired by Cathy’s work on “co-creation”, and an episode of “Lecture Breakers” (I think the first one on student engagement techniques where they talked about letting students choose the format of the artefact they do for assessment purposes; but I binge-listened, and honestly, they are all inspiring!). And something that Sam recently said stuck with me — sometimes the teacher and the students just have “to play the game”. Assessment is something that needs to happen, and there are certain rules around it that need to be followed, but there are also a lot of things that can be negotiated to come to a consensus that works for everybody. So, as a teacher, just be open about your role in the game and the rules you yourself are bound by and the ones you are open to negotiate, and then start discussing! Anyway, the combination of those three inputs gave me an idea that I would like your feedback on.

Consider you want to teach a certain topic. Traditionally you would ask students to do a certain activity. You have specific learning outcomes you want your students to reach. Whether or not they reach those outcomes, you would evaluate by asking a certain set of questions to see whether they answer them correctly, or maybe by asking them to produce an artefact like an essay or a lab report. And that would be it.

But now consider you tell students that there is this specific topic you want to teach (and why you want to teach it, how it relates to the bigger picture of the discipline and what makes it relevant. Or you could even ask them to figure that out themselves!) and that they will be free to produce any kind of artefact or performance they want for the assessment. Now you could share your learning outcomes and tell them about what learning outcomes matter most to you, and why. And then you could start discussing. Do students agree on the relative importance of learning outcomes that you show in the way you are weighing them? Are there other learning outcomes that they see as relevant that you did not include (yet)?

Once that is settled (possibly by voting, or maybe also coming to a consensus in a discussion, depending on your group and your relationship to them. And of course you can set the boundary conditions that maybe some learning outcomes need to count for at least, or not more, a certain threshold), you are ready for the next important discussion. How could students show that they have mastered a learning outcome? What kind of evidence would they have to produce? What might count as having met the outcome, what would still count as “good enough”?

Now that it’s clear what the learning outcomes are and what they mean in terms of specific skills that will need to be demonstrated, you could let students add one learning outcome that they define themselves and that is related to the format of the artefact that they want to produce (possibly public speaking with confidence when presenting the product, learning to use some software to visualise, or analysing a different dataset than you gave them themselves, …). You could have already included 10% (or however much you think that skill should “be worth”) in the rubric, or negotiate it with students.

While negotiating learning outcomes, students will already have needed to think about how each learning outcome will become visible with their chosen way of presentation, and this should be talked through with you beforehand and/or documented in a meta document, so that a very artistic presentation does not obscure that actual learning has taken place.

How much fun would it be when people can choose to give a talk, do a short video, present a poster, design an infographic, rhyme a science poem, or whatever else they might like? I imagine it would be super motivating. Plus it would help students build a portfolio that shows their subject-specific skills acquired in our class alongside other skills that they think are fun or important to develop. And maybe some artefacts could be used in science communication, engaging other people by hooking them via a format they are interested in, and then maybe they also get interested in the content? I’ve seen hugely creative ideas when we asked students to write blog posts about phenomena we had investigated in the rotating DIYnamics tanks, like a Romeo-and-Juliet-type short novel on two water drops, or an amazing comic — and there they were confined to writing. What if they could also choose to make objects like my pocket wave watching guide, or to perform a play?

I guess it could be overwhelming when the content is very difficult, the task is very big, and students then also have to consider how to show that they learned it, in a way that isn’t pre-determined. Also timing might be important here so this task does not happen at the same time as other deadlines or exams. And obviously when you suggest this to your students, they might still all want to pick the same, or at least a traditional, format, and you would have to be ok with this if you take them seriously in these negotiations. What do you think? What should we consider and look out for when trying to implement something like this?

Co-creating learning and teaching (Bovill, 2020)

Maybe it was because of the contexts in which I encountered it, but I always perceived “co-creation” as an empty buzzword without any actionable substance to it. I have only really started seeing the huge potential and getting excited about it since I met Catherine Bovill. Cathy and I are colleagues in the Center for Excellence in Education iEarth, and I have attended two of her workshops on “students as partners” and now recently read her book (Bovill, 2020). And here are my main takeaways:

Speaking about students as partners can mean very many, very different things. The partnership between students and teachers is “a collaborative, reciprocal process through which all participants have the opportunity to contribute equally, although not necessarily in the same ways, to curricular or pedagogical conceptualizations, decision-making, implementation, investigation, or analysis” (Cook-Sather et al., 2014). For me, understanding the part about contributing equally, although not necessarily in the same ways really helped to get over objections like “but I am responsible for what goes on in my course and that students have the best possible environment for their learning. How can I put part of that responsibility on students? And can they even contribute in a meaningful way when they are not experts yet?” and the key is that they are contributing as equals, but that does not mean that we are sharing responsibility or tasks (or anything necessarily!) 50/50.

Including students as partners to co-create their learning and teaching leads to many advantages: the forms of teaching and learning that are created in such a process are more engaging to students and more human in general. Since it feels more relevant to students, learning is enhanced, and becomes more inclusive. The student also experience new roles which helps them in becoming more independent, secure, and responsible. And it seems to be a lot of fun for the teacher, too, because a lot of new opportunities for positive interactions are created.

“Students as partners” does not mean that one necessarily has to jump into the pool at the deep end and re-design the whole curriculum from scratch. There is a whole continuum of increasing student participation where a teacher only gradually shares more and more control, and every small move towards more participation is a step in the right direction. This includes many smaller steps I’ve implemented in my teaching already, without even realising that that could be counted as working towards “students as partners”!

Some of those small steps suggested in the book and that can already have a positive impact include

  • Reserving one or two lessons at the end of the semester for perspectives or topics that students would like included (which I personally have really good experiences with!).
  • Giving student questions back into the group with the question “what do you think? and why?”, sharing the power to answer questions rather than claiming it solely for the teacher.
  • Doing a “note-taking relay”: at regular intervals, the teacher stops and gives time for students to take notes. Students do take notes and then pass them on to their neighbour. At the next note-taking break, they take notes on that piece of paper in front of them, and then pass it on to the next neighbour. They are thus creating a documentation of the class with and for each other.
  • Invite students to create study guides or resources for next year’s students.
  • Invite them to design infographics, slides, diagrams on important topics, or present their own role plays of different theories in fictitious situations, which then are used in teaching of their own class.

Especially this last point I think I might have underestimated until now. When I saw my name mentioned in the newsletters of my two favourite podcasts this week, it made me feel super proud! If students only feel a fraction of that pride when their work is featured in a course as something that other people can learn from, it is something we should be doing MUCH MORE!

Other things that come to my mind that share responsibility in small ways or strengthen relationships:

If you (and they!) so choose, students could also become partners on bigger parts of the course, and especially on designing their own assessment, and in evaluating the class. Here are some examples described in the book:

  • In one of her own courses on the topic of educational research (which probably included how to gather data in order to evaluate teaching and learning), Cathy invited students to pick aspects of her course which they wanted to evaluate, and then work with her to design an evaluation, analyse the data and present their findings.
  • She also describes how she invited Master students to co-design dissertation learning outcomes, and that it was possible to include it in the official university regulations: In addition to the ones that are prescribed for all students, each student gets to design one individually in collaboration with their supervisor.
  • Another idea she presents is to give students key words and let them create their own essay titles including those keywords. They have the freedom to choose what question they find most interesting related to a certain topic, while the teacher can make sure the important keywords from their point of view are included. But it is then important that students and teacher work together to make sure the scope is right and there is enough literature to answer that question!
  • And it is possible to let students vote on the weighting of different assessment components towards their final grade. This could even be done with boundary conditions that, e.g., each assignment will have to count for at least a certain percentage. Apparently the outcomes of such votes do not vary much from year to year, but still it is increasing student buy-in a lot!

Or, going further along that continuum of students as partners, students can get involved in the whole process of designing, conducting, evaluating and reporting on a course.

  • Cathy presents an example of a business course where student groups come up with business ideas in the beginning and then everybody discusses what students would need to learn in order to make those ideas become reality. Those topics are then presented to each other by different student groups.
  • The point above reminds me of something I heard on a podcast, where the students also got involved in presenting materials and the teacher gave them the choice of which topics they wanted to present themselves and which topics they would prefer taught by the teacher. This sounds like a great idea to give the students the opportunity to pick the topics they are really interested in and at the same time leave the seemingly less attractive topics (or those where they would really value the teacher’s experience in teaching them) to the teacher.
  • A project I am currently working on with Kjersti and Elin, where we bring together students that took a class the previous year with students who are taking it this year in order for them to do some tank experiments together, but working towards different learning outcomes depending on their level. Here the older students help the younger ones by engaging in dialogue with them and acting as role models, while also “learning through teaching”. We are working on engaging the students in designing the learning environment, and it is super exciting!
  • In a recent iEarth Digital Learning Forum, Mattias and Guro described the process of completely re-designing a course in dialogue between the teacher and a team of students. And not only did they co-design the course, they also presented it together (which is a step that is really easy to forget when the partnership isn’t fully internalized yet!).

I really like the framework of “students as partners” as a reminder to think about including students in a different way, and especially to think about it as a continuum where it’s ok — and even encouraged! — to start small, and then gradually build on it. And I am excited about trying more radical forms of “students as partners” in the future!


Bovill, C. (2020). Co-creating learning and teaching: Towards relational pedagogy in higher education. Critical Publishing.

Cook-Sather, A., Bovill, C., & Felten, P. (2014). Engaging students as partners in learning and teaching: A guide for faculty. John Wiley & Sons.