ORA.ATO.230 Training manual and operations manual

TRAINING MANUAL Training manuals for use at an ATO to conduct integrated or modular flight training courses should include the following:

(a)The training plan:

(b)Briefing and air exercises:

(c)Flight training in an FSTD, if applicable: Structure generally as for (b)

(d)Theoretical knowledge instruction:

THEORETICAL KNOWLEDGE COURSE DESIGN REQUIREMENTS An ATO that delivers theoretical knowledge instruction for professional pilot licences should ensure that:

(a)the courses are designed and developed using the instructional systems design (ISD) methodology, which is supported by a robust and effective management system;

(b)the courses include a standardised and dynamic assessment and testing system;

(c)instructors that deliver KSA instruction have received appropriate training covering at least learning styles, teaching methods, facilitation techniques, threat and error management (TEM), the applicable competencies, and the content of the subject(s) and exercises that they are to deliver;

(d)the recurrent training of instructors is conducted at least annually;

(e)the instructors that are responsible for assessing Area 100 KSA have received appropriate training regarding the assessment(s) that they are to conduct, and are to be standardised to ensure that the assessment grades awarded are consistent across the ATO; this standardisation should include at least familiarisation with the performance indicators, the ATO’s word pictures for grading, and the ATO’s debriefing system; and

(f)recurrent standardisation training is conducted at least annually to ensure continued inter-rater reliability.

AREA KSA 100 02 AND 100 03 LEARNING OBJECTIVES, ASSESSMENTS AND RECORDS

(a)An ATO that delivers theoretical knowledge instruction for professional pilot licences should ensure that for the learning objectives (LOs) in topics 100 02 and 100 03 of Area 100 KSA there are at least two summative assessments and at least one formative assessment. The summative assessments are to be documented in the student’s training records. Both the summative assessments and the formative assessment(s) should be debriefed.

(b)The formative assessment(s) should:

(1)be designed such that the student has the opportunity to ask questions and develop competencies in most of the LOs in 100 02 and 100 03 of Area 100 KSA;

(2)be conducted during the training; the ATO may in addition conduct a formative evaluation (continuous assessment) over a specified phase of the course; and

(3)be conducted by an instructor that is trained to deliver the formative assessment.

(c)The summative assessments should:

(1)be designed so that they collectively give the student the opportunity to demonstrate competency in all LOs in 100 02 and 100 03 of Area 100 KSA; each individual summative assessment may address some of the LOs in 100 02 and 100 03 of Area 100 KSA;

(2)be satisfactorily completed before the student is recommended by the ATO for their first attempt to take the final theoretical knowledge examination paper, and the outcome of the assessments should be included in the student’s training record;

(3)require that for a student to be considered that they have achieved a ‘Satisfactory’ standard, they:

(i)meet at least 35 % (which defines the term ‘some’ used in the word pictures) of the indicators relevant to the assessment exercise, in each competency;

(ii)have an overall positive effect on the outcome or completion of the exercise without any external input from the instructor, or where the assessment requires the instructor to facilitate the exercise, without the instructor providing any knowledge or corrective input to assist in the completion of the exercise; and

(4)be conducted by an instructor that is trained to deliver the summative assessments.

(d)The training manual should include the following elements regarding the theoretical knowledge training and assessment of the LOs in topics 100 02 and 100 03 of Area 100 KSA:

(1)the positions, or range of positions, of the formative assessment exercise(s) and summative assessment exercises in the training programme;

(2)a description of the summative assessments, including a matrix that shows which Area 100 KSA LOs are covered in each exercise;

(3)the grading system of the Area 100 KSA summative assessment and a description of the ATO’s minimum required standard;

(4)the template for the information about Area 100 KSA to be included in the student’s training record, which should include at least the dates and result (‘Pass’ or ‘Fail’) of the summative assessments and the date and score of the mental maths test;

(5)the method of assessment debrief for each summative and formative assessment;

(6)for a student who performs below the satisfactory standard in a summative assessment(s), the method to further develop the student’s competencies and how to conduct the reassessment.

(e)Access to the information on Area 100 KSA kept in the student’s training records should be restricted to the student and authorised ATO personnel, and should not be disclosed outside the ATO. The information on the record should first be de-identified before it is used to support course design improvements.

AREA 100 04 LEARNING OBJECTIVES: MENTAL MATHS TEST AND RECORDS

(a)An ATO that delivers theoretical knowledge instruction for professional pilot licences should ensure that at least one KSA mental maths test is conducted and the outcome(s) documented in the student’s training records.

(b)The mental maths test(s) may be written or oral in format and should, where possible, be scenario-based, with at least two questions per LO in topic 100 04 of Area 100 KSA.

(c)The minimum score to pass the Area 100 KSA mental maths test(s) should be 75 % of the marks allocated to a test. However, a higher pass mark may be defined by the ATO.

(d)The mental maths test(s) should be satisfactorily completed before the student is recommended by the ATO for their first attempt to take their final theoretical knowledge examination paper.

ASSESSMENT OF STUDENTS IN AREA 100 KSA

(a)The Area 100 KSA formative assessment(s) and summative assessments may include but not be limited to: written planning exercises combining multiple subjects; practical exercises using training devices (if available); scenario-based oral board (viva voce); scenario-based communications exercises; written assignments or project work; and preparation and delivery of group or individual presentations.

(b)The format of formative and summative assessment debriefs should be effective, highlighting the student’s strengths and weaknesses and enabling future improvement.

AREA 100 KSA WORD PICTURES

(a)‘Word pictures’ are a proven assessment tool that standardises pilot core competencies, and can be used to assess student’s competency in the Area 100 KSA LOs in topics 100 02 and 100 03. Word pictures describe the student’s performance. Each word picture is associated with a numerical grade; within the range of grades, the minimum acceptable standard is defined. Additionally, a word picture describing performance that falls below the minimum satisfactory standard should be included in the range, as well as additional word pictures that relate to grades which exceed this minimum satisfactory standard. Word pictures enable the standardisation of the assessment performance and facilitate inter-rater reliability within an ATO.

(b)This GM provides two examples of word pictures.

(c)The most commonly used word pictures are shown in Section A below. They are based on performance indicators, which explain what the student should demonstrate in order to attain the specific Area 100 KSA LOs that are addressed by the assessment exercise. Word pictures are formed of elements that contain the following:

(1)HOW MANY of the performance indicators were observed and, where relevant, HOW OFTEN;

(2)HOW WELL the competency was demonstrated in the assessment exercise to have an overall positive effect on the outcome or completion of the assessment exercise;

(3)the level of success in the OUTCOME of the assessment exercise.

(d)An ATO could establish its own set of word picture descriptions as long as they are comparable in the grading of each competency, similar to the ‘Communication’ and ‘Application of knowledge, UPRT and resilience’ word pictures example in Section B below.

(e)The advantage of word pictures is that they provide meaningful and standard data to enable identification of individual, crew, class, instructor and ATO trends, which can be analysed in order to provide feedback for further improvement or development.

(f)An ATO should ensure that the detailed information obtained through its grading in Area 100 KSA is de-identified before using it to support course improvement. SECTION A — EXAMPLE 1 AREA 100 KSA WORD PICTURE GRADE LEVELS (USING INDICATORS)

(g)The example shown below in this Section contains the most commonly used word pictures, which are formed of elements that contain the following:

(1)HOW MANY of the performance indicators in the table further below relevant to that summative assessment were observed in that competency (as a percentage);

(2)HOW WELL the competency was demonstrated in the assessment; and

(3)the level of success in the OUTCOME of the summative assessment.

(h)In order to satisfactorily complete an Area KSA 100 summative assessment, the student should reach at least the minimum satisfactory level in each competency covered by that assessment. In case the student fails to reach the minimum satisfactory level in each competency, the student should repeat the summative assessment or another summative assessment that covers the competency(ies) where performance was previously assessed as unsatisfactory.

Table 1: Example generic competency framework that can be applied for assessing the student’s level of performance

AREA 100 KSA ASSESSMENT PERFORMANCE INDICATORS

(i)The performance indicators that relate to the LOs in topics 100 02 and 100 03 of Area 100 KSA can be used to form a word picture.

Table 2: Performance indicators relevant to the LOs in topics 100 02 to 100 03 of Area 100 KSA

SECTION B — EXAMPLE 2 AREA 100 KSA WORD PICTURES (USING DESCRIPTIONS)

(j)An ATO may devise its own word pictures for each of the competencies, to be used alongside or combined with those given in EXAMPLE 1. A descriptive word picture typically includes descriptive examples that the ATO’s instructors could readily identify and then equate to given competency levels. Below are two examples (for ‘Communication’ and ‘Application of knowledge, UPRT and resilience’).

Table 3: Example word pictures using descriptions

AREA 100 KSA EXERCISES AND ASSESSMENTS Exercises and assessments are to be interwoven into the theoretical knowledge training, utilising a range of learning styles; they should address subject or cross-subject topics, with the application of threat and error management (TEM) and, where possible, be scenario-based. The exercises and assessments do not need to be confined to a classroom.

(a)Area 100 KSA exercises may be of short duration within a lesson, and the student’s performance in the exercises does not need to be recorded, although the main subject and KSA learning points are likely to be discussed (or for distance learning, reviewed) within a post exercise debrief or lesson summary. To allow for flexibility and development, the exercises do not need to be specified in the training plan.

(b)When a single formative assessment is specified in the training plan, it is likely to be extensive as it will cover many of the LOs in Area KSA 100 02 and 100 03. Alternatively, an ATO may specify a number of shorter-duration formative assessments that each covers a narrower range of LOs, and these may build in terms of content difficulty.

(c)The exercises and formative and summative assessments may include but not be limited to: scenario planning exercises combining multiple subjects; practical exercises using training devices (where available); oral communication exercises; written assignments and/or project work; discussions; the preparation and delivery of group or individual presentations and discussions; and enable scenario-based content and individual, pair or group situation(s).

(d)The type of assessment and the environment should be recorded in the ATO’s training plan.

AREA 100 KSA INSTRUCTION AND ASSESSMENT TRAINING

(a)The following material has been developed to provide additional guidance to organisations to help them develop an effective KSA 100 instruction and assessment training programme that satisfies the provisions in AMC2 ORA.ATO.230 (c) to (f).

(b)An ATO should ensure that an instructor who conducts the Area 100 KSA formative assessment(s) has received adequate training to be familiar with the:

(1)relevant competencies and performance indicators;

(2)Area 100 KSA Learning Objectives (LOs);

(3)formative assessment(s) that they will conduct including: the applicable LOs, purpose and content of the assessment(s) and position(s) in the training plan, assessment resources, assessment environment;

(4)Area 100 KSA grading system, including familiarisation with the performance indicators and the ATO’s word pictures; and

(5)student debrief methods and procedure.

(c)An ATO should ensure that an instructor who conducts the Area 100 KSA summative assessments has received adequate training to be familiar with:

(1)the summative assessments that they will conduct including: the applicable LOs, purpose and content of the exercise(s) and position(s) in the training plan, assessment resources, assessment environment, and the minimum acceptable level;

(2)the assessment feedback, evaluation and development process; and

(3)KSA candidate appeal procedure.

(d)An Area 100 KSA instruction and assessment course should include practical training on the conduct of an assessment, including grading to achieve inter-rater reliability, and the debrief under supervision.

INSTRUCTIONAL SYSTEMS DESIGN

(a)The instructional systems design (ISD) provides a systematic and iterative process for course design based on educational best practices. There are several effective ISD models in use today, with the analyse, design, develop, implement and evaluate (ADDIE) framework being generic to all. The purpose of using ISD to design training courses is to facilitate the students’ efficient and effective acquisition of knowledge and skills based on current training needs.

(b)To provide evidence of the effective use of the ISD methodology in the design and continued development of their course(s), an ATO may use documentation and records that relate to the ISD phases.

(d)ADDIE model example. The ‘analysis’, ‘design’, ‘development’, ‘implementation’ and ‘evaluation’ phases of the ADDIE model are shown below with brief phase descriptions. [Figure or form omitted from this preview — available in the Avioverse workspace library.]

ALL ATOs, EXCEPT THOSE PROVIDING FLIGHT TEST TRAINING

OPERATIONS MANUAL The operations manual for use at an ATO conducting integrated or modular flight training courses should include the following:

(a)General:

(1)a list and description of all volumes in the operations manual;

(2)administration (function and management);

(3)responsibilities (all management and administrative staff);

(4)student discipline and disciplinary action;

(5)approval or authorisation of flights;

(6)preparation of flying programme (restriction of numbers of aircraft in poor weather);

(7)command of aircraft;

(8)responsibilities of the PIC;

(9)carriage of passengers;

(10)aircraft documentation;

(11)retention of documents;

(12)flight crew qualification records (licences and ratings);

(13)revalidation (medical certificates and ratings);

(14)flight duty period and flight time limitations (flying instructors);

(15)flight duty period and flight time limitations (students);

(16)rest periods (flight instructors);

(17)rest periods (students);

(18)pilots’ log books;

(19)flight planning (general);

(20)safety (general): equipment, radio listening watch, hazards, accidents and incidents (including reports), safety pilots etc..

(b)Technical:

(1)aircraft descriptive notes;

(2)aircraft handling (including checklists, limitations, maintenance and technical logs, in accordance with relevant requirements, etc.);

(3)emergency procedures;

(4)radio and radio navigation aids;

(5)allowable deficiencies (based on the master minimum equipment list (MMEL), if available).

(c)Route:

(1)performance (legislation, take-off, route, landing etc.);

(2)flight planning (fuel, oil, minimum safe altitude, navigation equipment etc.);

(3)loading (load sheets, mass, balance and limitations);

(4)weather minima (flying instructors);

(5)weather minima (students – at various stages of training);

(6)training routes or areas.

(d)Personnel training

(1)appointments of persons responsible for standards/competence of flight personnel;

(2)initial training;

(3)refresher training;

(4)standardisation training;

(5)checks to verify that training personnel is proficient with regard to all internal ATO standards, processes and procedures;

(6)upgrading training;

(7)ATO personnel standards evaluation.