If we want to do a valid assessment of what a specific student can do, we need to know what information they had available when producing the artifact we are evaluating, who they could communicate with, and what tools they had access to. And we might want to restrict access to some or all of those, to some degree or completely. Dawson et al. (2023) develop a taxonomy of restrictions that I find really useful as an overview!

This is my rendition of their taxonomy:

Now the question is of course what restrictions are desirable in what context, and that, of course, depends on the learning outcomes. But there is also the question of feasibility: Can I even enforce the restrictions I want? And if I can’t — what does that do to the validity of my assessment if I am assuming students don’t have access to something that helps them, but in practice they do? And in addition to that validity, there is also the “consequential validity” to consider: students will prepare differently for exams depending on whether they will or will not have access to certain resources, even though the learning outcomes might be the same. So what are the implications of the imposed restrictions on students’ approach to learning? Also, putting a students in a situation where they constantly feel controlled and distrusted will do something to how they act and learn in the long term, and in a way that might not be what we want. Then, authenticity is another consideration — what resources will students have available when they actually do their jobs in the future, and is it reasonable / desirable to not allow them those same resources in an exam situation? It can of course be!

Lastly, values. Depending on what we value highest, for example “freedom” might lead to “no restrictions”, while “equity” might mean a lot of restrictions in place to level the playing field. Etc.

I thought it was super interesting to think about restrictions in this way, and I recommend you read the full paper!

Dawson, P., Nicola-Richmond, K., & Partridge, H. (2023). Beyond open book versus closed book: a taxonomy of restrictions in online examinations. Assessment & Evaluation in Higher Education, 1-13.