Are you looking to start offering assessments online but aren’t sure how to go about it? Understandably, it can feel a bit overwhelming with all the options available. So, why not start your explorations into digital assessments by undertaking a proof of concept or pilot? This is a good way to explore the topic without significant commitment of time and money and ensures that you can understand what’s possible for your organisation.
So, what steps do you need to follow if you want a successful outcome for your digital assessment pilot?
Context
There are many advantages to switching to digital assessments, but not all of these may be relevant to your organisation. It’s important to understand your drivers for the pilot as well as the strategic context. Having an appreciation of this will ensure that your efforts are aligned with your strategy and clarify the benefits and outcomes you’re hoping to achieve.
Other aspects you may need to consider include your vision for digital assessments, the benefits not just for you but also the learners, and the risks involved. In the latter case, are there any technical or political threats to the success of the pilot? And if so, how can these be mitigated?
Objectives and Success Criteria
A digital assessment pilot should have clear objectives and a set of success criteria against which the efficacy of the work can be assessed. Further, for digital assessments it is less about technology and more about cultural change – about people and processes. Objectives need to address this aspect as much as the technology involved. All members of the project team should understand their role and the outcomes expected.
The potential success criteria could be many, as the benefits of switching to online assessments are numerous. The key is to look for the factors that will be of most benefit to your organization and measure those. You may also wish to group them according to your target audience – the learner, the learning and development manager, senior management, and so on.
Your Requirements
Start your pilot by establishing clear requirements. There’s value in documenting and understanding the current assessment process and soliciting any feedback from those involved. Explore with them what a digital assessment process would do to improve the experience for all stakeholders. What works well? What doesn’t? What innovations in assessment should be explored? And don’t forget to speak to the learners themselves!
Prioritise the requirements as a team using a methodology such as MoSCoW – Must have, Should have, Could have, Won’t have (this time). The ideal plan is to ensure that the pilot delivers all the requirements defined as Must, Should or Could. Keep a good record for later iterations or developments.
From a technology perspective, it’s important to be clear on the features your organisation would expect from a platform. Define your high-level requirements to cover the types of assessment, the monitoring and delivery of those assessments, marking and grading, reporting, accessibility, usability, integration, and support. Consider the role of cloud services and mobile apps. Will you require a level of proctoring? Also think about the value that machine learning or artificial intelligence might be able to bring, particularly in relation to marking and grading.
Select the Courses / Pilot Champions
The next stage is to select the right courses for your digital assessment pilot. There are several factors that should be considered when selecting the courses:
- Ideally a small number of courses
- Limit the number of learners for the pilot – ideally no more than a hundred or so, but no less than 25
- The courses should ideally contain several types of assessment, e.g., an essay, question/answer quiz, etc.
- Be clear on how the learners are going to be assessed
- The courses should be established ones, with the course tutors having ideally taught/marked the course previously
- Consider whether to offer the opportunity of mock assessments to aid familiarisation
- The approach to securing feedback from all involved in the assessment!
Be sure to identify your champions early on and engage them in decision-making for the selection and delivery of the digital assessment pilot.
Other Issues to Consider
Make sure that the assessment is accessible – not just in the sense that it can be viewed by someone using a screen reader, but that it utilises commonly available technology. Also ideally make the assessment authentic, in that it asks the learner to apply their knowledge to real-life situations or contexts.
Consider, too, the timing, particularly with an international learning community. Consider undertaking the first pilot on-site and/or in one time zone to ensure that the foundations are correct before exploring further pilots and implementations across time zones or online.
Start Your Digtial Assessment Pilot!
As commented earlier, the move to digital assessments is as much about people as it is about technology. A successful implementation of digital assessments needs to be underpinned by a change management programme that is engaged throughout the entire journey.
The move from traditional to digital assessments is an exciting adventure with lots of benefits to realise for the administrators, the tutors, and the learners. A successful pilot is an integral part of that journey, a part that contributes to success.
For more digital education insights be sure to subscribe to our blog.
