When I was preparing for my private pilot flight test, my instructor pulled me aside and gave me a briefing on the examiner! He told me about this examiner’s preferences, likes and dislikes, and we discussed how I would modify my actions to ensure that I was able to make the examiner “happy”. This was a case of performing to the examiner’s ideas rather than to a set of specified competencies and I needed to perform differently on this flight test than how I was taught if I wanted to pass my test with this Examiner. My instructor had “Handling Notes” on all the local Examiners!
Although there are still some examiners who insist on performance their way, we have come a long way in education and assessment since the days of my Private Pilot Flight Test. In today’s world, the mantra for assessment of performance is RVFF:
- Reliable
- Valid
- Flexible
- Fair
A great performance assessment tool ensures that we achieve a reliable assessment such that each student is assessed to the same (objective) standard and we can be confident that a student who is deemed competent performed to the same level as the other students who were deemed competent. This means that the assessment of a student’s performance would be the same no matter who conducted the assessment – the assessment should not be based on the examiner’s standards, but rather, on the syllabus.
The assessment must also be valid. This includes the notion that the assessment actually assesses what we have said it did! I once failed a vehicle driving test because the examiner said that although I was below the speed limit, I went around a corner on a bitumen road too fast – his reasoning was that had it been a gravel road, my driving would have been unsafe! It wasn’t gravel road! (You can see I am still traumatised by this 40 years later!). The Examiner based his fail assessment on my ability to drive on an unprepared surface based on my performance driving on a bitumen road without telling me that the scenario was a gravel road situation. Assessments must be valid.
The assessment should also be flexible, taking into account environmental circumstances, the student’s circumstances, and providing the best opportunity for the student to demonstrate competence. This is not a matter of allowing for poor performance, but it does mean that if the wind is gusty, a landing may not be a “greaser” and thus a “fail”.
Finally, the assessment must be fair. That is, the assessment (amongst other things) must not demand more than what is required in the syllabus of training with respect to standard, accuracy, timeliness or even content. The examiner must not hold his or her desired technique as “the way”, and fail a student if they did not do things in the way the examiner likes them done.
So how do we train students to present to examiners who can then conduct Reliable, Valid, Flexible and Fair assessment? Using the notion that we should “start with the end in mind” our detailed lesson plans must be written in the shadow of the assessment process.
“detailed lesson plans must be written in the shadow of the assessment process”
That is, we write detailed lesson plans to ensure that we prepare the student for the final assessment. If a student must demonstrate competence using a specific technique, then we must ensure that we teach them that technique and ensure that our lesson plan contains those detail of technique for every instructor to be aware of. If a student must demonstrate competence in a particular process that requires five steps, we must ensure that the lesson plan specifies those 5 steps and then our instructors can then teach those steps. It would not be fair to require at the test, that a student perform in a particular way, and not to have taught them that particular way. It would also not be a reliable assessment of performance if some students were deemed competent without having demonstrated the required technique or process that is considered acceptable. Likewise, it would not be a reliable assessment if other students we deemed not competent because they didn’t perform the way the Examiner liked. Thus, our detailed lesson plans must include all of this information to ensure that students are well prepared and so that our assessment are reliable, valid, flexible, and fair.
Creating detailed lesson plans with all this information will ensure that our examiners have a good opportunity to make reliable and fair assessments because they will know what all the students have been taught, and they can assess to this same acceptable standard. That’s the power of great lesson plans.
Do your lesson plans have sufficient detail of technique, knowledge, process and accuracy to ensure standardisation and prepare your students for reliable and fair assessments?