Sharing stories and practices of assessment in emergency remote teaching

Guest post by Sukaina Walji, University of Cape Town, South Africa

During 2020, at the Centre for Innovation in Learning and Teaching (CILT) at the University of Cape Town we scaled up various initiatives to support academics and teaching staff to move their courses to online learning. Much of what we did resonates with what many learning design and learning technology teams did globally and included the following activities:

• We produced guidelines and resources customised to our context of advocating for a broadly asynchronous teaching “low-tech” teaching mode;
• We ran webinars and workshops starting with broad and general ‘what it means to teach online’ to specific guidance on thinking about how to manage and produce appropriate instructional content, design activities and assessment;
• Learning designers worked with individual academics and teaching teams to review courses and provide consultations;
• Our digital media unit supported people to make lecture videos and other multimedia materials from home and supported video captioning.

Much of this work has been incredibly generative with productive partnerships, cross-institutional learning and ongoing conversations about the place of online learning in a predominantly residential university in the future.

Methods for encouraging reflection

While there often hasn’t been very much time for formal reflection, one area we have been able to delve a bit more deeply into has been around assessment practices. Late in 2020 we started a project to collect case studies of assessment practices during the first period of our emergency remote teaching phase where most assessment had to be online. I serve on the university’s Assessment Framework Working Group, and we were increasingly aware that assessment in particular was challenging in many ways. During our deliberations, we decided to start a project to collect case studies of what was happening in the assessment space to provide a space for different voices to emerge.

This involved developing a short survey inviting people to share an example or two of assessment practices. We then followed up with interviews with people who wanted to share something in more depth. The plan was to collect examples which allowed people to share something they were proud of or where they wanted to reflect and thought their experiences might be helpful. We were also keen to collect examples where things hadn’t gone so well. The focus was as much on sharing as a process as it was about the examples, with the aim to build a community around assessment practices. Sharing is a form of generosity and vulnerability, and I knew that we were asking a lot of people in very challenging and stressful circumstances to take time out, share something that is very personal especially where things might not have gone as well as they would have expected.

From October 2020 to February 2021 we collected just over 20 individual case studies. The interviews were semi structured and asked people to talk through:

1. The purpose of the assessment (whether it was diagnostic, formative, summative);
2. Whether this was an adapted version of an assessment or something new they were trying;
3. How they went about designing the assessment including their processes and tools used;
4. How they administered the assessment and what happened including how students responded and any other factors that influenced the process and the experience;
5. The outcomes, including reflection on lessons learned, whether this was something they would carry forward, whether they felt the assessment reflected student learning as previously and what further areas of support they felt were needed.

The individual cases stand alone and make interesting and sometimes inspiring reading. The interviewers, who were experienced learning design staff, gained much from hearing from on the ground teaching staff, while the academics had an hour or so of being able to reflect and share, prompted by questions from genuinely interested and knowledgeable colleagues. The interviews are summarised as shareable examples of practice for fellow colleagues and that could be used as teaching materials or prompts for reflection.

Analysis and reflection

As we were reviewing the cases, a number of patterns emerged that prompted us to cluster the cases into themes that related to purpose and which has allowed a broader view of the types of assessment practices emerging as a result of the online pivot. We found a clusters of cases that were concerned with:

• mitigating the effects of going online and in particular strategies to design for exam integrity;
• enhancing current practices, with innovative or purposeful course (re)design decisions that adapted or expanded existing assessment practices;
• improvements where assessment practices in the online space improved on what had been offered previously;
• difficulties, where due to the specific disciplinary need, it was not possible to adequately assess in the online space.

While this is a small sample, it is interesting to think about these clusters, and consider what it is that drives assessment design and how people articulate their purposes. The online space has prompted many teaching staff to re-think their assessment practices based on what is now not possible and what is perhaps newly possible.

Implications for staff and faculty development

In terms of staff or faculty development initiatives, the focus has been on the ability to reflect on, capture and share experiences on assessment that go beyond dry ‘learning design’ how-tos and checklists or a tools focus which promises to deal with integrity or collusion. The reality of designing and managing assessments in an online space has been incredibly difficult and continues to prove challenging. Assessment is a socio-cultural practice tied into the context and the culture of an institution, a discipline and a faculty. While guidelines and practitioner notes are helpful, we know many staff find it difficult to apply what they are guided towards. Learning from fellow teachers with similar contexts including what sometimes doesn’t work provides another approach to staff/faculty development with the potential for meaningful interactions, a community of peers and purposeful learning together. We might think this happens organically, but many staff work in isolated and lonely spaces within a closed course environment, so a project such as purposeful sharing of case studies brokered by a centrally managed project offers another avenue for sharing and community building.

Sukaina Walji is currently the acting director of the Centre for Innovation in Learning and Teaching at the University of Cape Town and Head: Online Education and Course and Curriculum Design. She has a background in digital communications, learning design and online education project management. She tweets at @sukainaw.

This post is part of the “Around the world” series on faculty development. Watch this space in the coming months for more inspiration on professional development approaches in Higher Education from around the globe.