Aviation Training Experts™

handbook

Aviation Instructor’s Handbook

FAA-H-8083-9B Version 2020

Chapter 6

Assessment

The collaborative assessment process in learner-centered grading uses two broad rubrics: one that assesses the learner’s level of proficiency on skill-focused maneuvers or procedures, and one that assesses the learner’s level of proficiency on single-pilot resource management (SRM), which is the cognitive or decision-making aspect of flight training.

The performance assessment dimensions for each type of rubric are as follows:

Maneuver or Procedure “Grades”

  • Describe—at the completion of the scenario, the learner is able to describe the physical characteristics and cognitive elements of the scenario activities but needs assistance to execute the maneuver or procedure successfully.
  • Explain—at the completion of the scenario, the learner is able to describe the scenario activity and understand the underlying concepts, principles, and procedures that comprise the activity, but needs assistance to execute the maneuver or procedure successfully.
  • Practice—at the completion of the scenario, the learner is able to plan and execute the scenario. Coaching, instruction, and/or assistance will correct deviations and errors identified by the instructor.
  • Perform—at the completion of the scenario, the learner is able to perform the activity without instructor assistance. The learner will identify and correct errors and deviations in an expeditious manner. At no time will the successful completion of the activity be in doubt. (“Perform” is used to signify that the learner is satisfactorily demonstrating proficiency in traditional piloting and systems operation skills).
  • Not observed—any event not accomplished or required.

For example, a learner can describe a landing and can tell the flight instructor about the physical characteristics and appearance of the landing. On a good day, with the wind straight down the runway, the learner may be able to practice landings with some success while still functioning at the rote level of learning. However, on a gusty crosswind day the learner needs a deeper level of understanding to adapt to the different conditions. If a learner can explain all the basic physics associated with lift/drag and crosswind correction, he or she is more likely to practice successfully and eventually perform a landing under a wide variety of conditions.

Assessing Risk Management Skills

  • Explain—the learner can verbally identify, describe, and understand the risks inherent in the flight scenario, but needs to be prompted to identify risks and make decisions.
  • Practice—the learner is able to identify, understand, and apply SRM principles to the actual flight situation. Coaching, instruction, and/or assistance quickly corrects minor deviations and errors identified by the instructor. The learner is an active decision maker.
  • Manage-Decide—the learner can correctly gather the most important data available both inside and outside the flight deck, identify possible courses of action, evaluate the risk inherent in each course of action, and make the appropriate decision. Instructor intervention is not required for the safe completion of the flight.

In SRM, the learner may be able to describe basic SRM principles during the first flight. Later, he or she is able to explain how SRM applies to different scenarios that are presented on the ground and in the air. When the learner actually begins to make quality decisions based on good SRM techniques, he or she earns a grade of manage-decide. The advantage of this type of grading is that both flight instructor and learner know exactly where the learning has progressed.

Let’s look at how the rubric in Figure 6-4 might be used in a flight training scenario. During the postflight debriefing, flight instructor Linda asks her learner, Brian, to assess his performance for the day using the Replay, Reconstruct, Reflect, and Redirect guided discussion questions described in the Learner-Centered Assessment section presented earlier in this chapter. Based on this assessment, she and Brian discuss where Brian’s performance falls in the rubrics for maneuvers/procedures and SRM. This part of the assessment may be verbally discussed or, alternatively, Brian and Linda separately create an assessment sheet for each element of the flight.

Figure 6-4. Rubric for assessing flight training maneuvers.
Figure 6-4. Rubric for assessing flight training maneuvers.

When Brian studies the performance levels, he decides he was at the “Perform” level since he had not made any mistakes. Where he had rated the item as “Perform,” Linda had rated it as “Practice.” During the ensuing discussion, Brian understands where he needs more practice before his performance is at the “Perform” level.

This approach to assessment has several key advantages. One is that it actively involves the learner in the assessment process and establishes the habit of healthy reflection and self-assessment that is critical to being a safe pilot. Another is that these grades are not self-esteem related, since they do not describe a recognized level of prestige (such as A+ or “Outstanding”), but rather a level of performance. The learner cannot flunk a lesson. Instead, he or she demonstrates a particular level of flight and SRM skills.

Both instructors and learners may initially be reluctant to use this method of assessment. Instructors may think it requires more time, when in fact it is merely a more structured, effective, and collaborative version of a traditional postflight critique. Also, instructors who learned in the more traditional assessment structure must be careful not to equate or force the dimensions of the rubric into the traditional grading mold of A through F. One way to avoid this temptation is to remember that evaluation should be progressive: the learner may achieve a new level of learning during each lesson. For example, in flight one, a task might be a “describe” item. By flight three, it is a “practice” item, and by flight five, it is a “manage-decide” item.

The learner may be reluctant to self-assess if he or she has not had the chance to participate in such a process before. Therefore, the instructor may need to teach the learner how to become an active participant in the collaborative assessment.

Choosing an Effective Assessment Method

When deciding how to assess learner progress, aviation instructors can follow a four-step process.

  1. Determine level-of-learning objectives.
  2. List indicators of desired behaviors.
  3. Establish criterion objectives.
  4. Develop criterion-referenced test items.

This process is useful for tests that apply to the cognitive and affective domains of learning, and also can be used for skill testing in the psychomotor domain. The development process for criterion-referenced tests follows a general-to-specific pattern. [Figure 6-5]

Figure 6-5. The development process for criterion-referenced tests follows a general-to-specific pattern.
Figure 6-5. The development process for criterion-referenced tests follows a general-to-specific pattern.

Instructors should be aware that authentic assessment may not be as useful as traditional assessment in the early phases of training, because the learner does not have enough information about the concepts or knowledge to participate fully. As discussed in Chapter 3, The Learning Process, when exposed to a new topic, learners first tend to acquire and memorize facts. As learning progresses, they begin to organize their knowledge to formulate an understanding of the things they have memorized. When learners possess the knowledge needed to analyze, synthesize, and evaluate (i.e., application and correlation levels of learning), they can participate more fully in the assessment process.