ORO.FC.231 Evidence-based training

EBT PROGRAMME SUITABILITY An operator’s EBT programme is one in which:

(a)training is focused on development of competencies, rather than repetition of tasks;

(b)the development of the programme is based on data-driven EBT training topics with a link to the operator’s competency framework;

(c)training needs are addressed through training based on underlying competencies;

(d)the programme includes:

(1)an evaluation phase to identify training needs based on competencies and collect population-based data; to identify the training needs means, the root cause of the deficiency observed should be identified rather than the symptoms of the deficiency;

(2)a manoeuvres training phase (skill retention): to train skill-based manoeuvres (body memory actions). These manoeuvres should place a significant demand on a proficient pilot; and

(3)a scenario-based training phase to focus on identified training needs based on competencies rather than repetition of tasks;

(e)the programme includes the conduct of objective observations based on a competency framework, and documents evidence of the behaviour observed;

(f)there is a customisation of syllabi:

(1)The operator should describe in the operations manual the procedure to customise syllabi. It should include how to:

(i)select the example scenario elements within a training topic that should be included in the EBT programme; and

(ii)contextualise the example scenario elements based on the operator’s operational data (e.g. input from SMS, FDM programme, etc.) and training data.

(2)This customisation should be based on evidence both internal and external to the operator;

(g)performance is evaluated using a competency-based grading system;

(h)instructors grade competencies based on observable behaviours (OBs);

(i)instructors grade the pilot using a defined methodology — observe, record, classify and assess/evaluate (ORCA) is recommended;

(j)instructors have completed the EBT instructor standardisation;

(k)instructors have sufficient concordance based on defined criteria (instructor concordance assurance programme);

(l)the analysis of the pilot's performance is used to determine competency-based training needs;

(m)there is a range of teaching styles during simulator training to accommodate trainee learning needs; and

(n)facilitation techniques in debriefing are incorporated.

UPSET PREVENTION AND RECOVERY TRAINING (UPRT) FOR COMPLEX MOTOR-POWERED AEROPLANES WITH A MAXIMUM OPERATIONAL PASSENGER SEATING CONFIGURATION (MOPSC) OF MORE THAN 19 Operators approved for EBT should follow the provisions for upset prevention and recovery training (UPRT) contained in AMC1 ORO.FC.220&230 ‘Operator conversion training and checking & recurrent training and checking’. These provisions should be included in the tables of assessment and training topics detailed in ORO.FC.232.

PERSONNEL CONDUCTING ASSESSMENT AND PROVIDING TRAINING

(a)Ground and refresher training should be provided by suitably qualified personnel.

(b)For non-EBT assessment and training: flight training should be provided by a flight instructor (FI), type rating instructor (TRI) or class rating instructor (CRI) or, in the case of the FSTD content, a synthetic flight instructor (SFI). The FI, TRI, CRI or SFI should satisfy the operator's standardisation, experience and knowledge requirements.

(c)Emergency and safety equipment training should be provided by suitably qualified personnel.

(d)CRM training should be provided by an EBT instructor or, for the classroom CRM training, a CRM trainer.

(e)Additional personnel requirements are described in ORO.FC.146 and ORO.FC.231 and in the associated AMC and GM.

RECURRENT CREW RESOURCE MANAGEMENT (CRM) Operators implementing EBT in accordance with ORO.FC.231 may demonstrate compliance with ORO.FC.115 by showing how the recurrent CRM requirements are integrated within the operator’s EBT programme. An example of how this may be done is provided in the safety promotion material of EASA (e.g. ‘EASA EBT manual´).

EBT PROGRAMME — TRANSITION FROM MIXED EBT The operator may agree with the competent authority the transition measures from mixed EBT to EBT baseline, which may include amongst others that the 3-year programme may include one or more modules in mixed EBT and one or more modules in EBT baseline, provided that all assessment and training topics in ORO.FC.232 are completed in the 3-year programme.

CUSTOMISATION OF THE EBT PROGRAMME (SYLLABI)

(a)Syllabi can be customised at three different steps:

(1)The first step would be a syllabus for the whole pilots’ population (customisation only at type rating level and/or aircraft generation level). At this step, the operator customises the example scenario elements based on relevant operational data (safety management system, state safety plan, OSD, occurrences, manufacturer data, etc.), and the training topics within the module are the same (same syllabus). At this level, it may be necessary to have a different example scenario element for the different crews within the same module to ensure that pilots are exposed to surprise and unexpected events and thus avoid pilots knowing all the details of the simulator session beforehand.

(2)The second step would be a different syllabus or part of it for the different populations of pilots. For example, some parts of the syllabus are different for the co-pilot and the captain, or the syllabus is different for the B747 pilots or for the Airbus pilots, etc. At this step, the module or part of the module is different for each population; this may include a different example scenario element for each population (or a different training topic; however, the customisation at training topic level is more difficult to control).

(3)The third step would be syllabi tailored to the individual pilot (pilot customisation — individual syllabus). This step is linked to the procedures established for the tailored training and the additional training of the pilots following the VENN model.

(b)The procedure to describe the customisation of syllabi must be described in the OM. Customisation is based on evidence that can be gathered on three different levels, two from the inner loop, one from the outer loop.

(1)Inner loop

(i)Individual evidence based on training data (e.g. grading metrics, training reports, questionnaires, etc.), analysed either for an individual pilot or a group of pilots (for example, all co-pilots, all B747 pilots, all pilots flying an Airbus model, etc.).

(ii)Operator-specific evidence gathered through the safety management process in accordance with ORO.GEN.200.

(2)Outer loop Evidence gathered from external sources such as authorities (e.g. state safety plan, etc.), OEMs (e.g. OEBs, OSD, safety documentation such as getting to grip, etc.

EBT PROGRAMME Further guidance on the EBT programme can be found in the EASA EBT manual.

EXPERIENCE IN MIXED EBT TO SUBSTITUTE ORO.FC.230

(a)The operator should have a minimum experience of 3 years of a mixed EBT programme. Note: More information on a mixed EBT programme is provided in GM1 ORO.FC.230(a);(b);(f) and in GM2 ORO.FC.A.245.

(b)The operator should demonstrate 2 years of an instructor concordance assurance programme.

(c)The operator should demonstrate 1 year of a valid equivalency of malfunctions.

(d)The operator should demonstrate 1 year of integration of the training data in the customisation of the EBT programme and SMS data for the contextualisation of the example scenario elements.

(e)The operator should demonstrate that there is a verification of the grading system and feedback is provided to the training system performance and to the instructor standardisation concordance assurance. SUBSTITUTION OF THE REQUIREMENTS OF ORO.FC.230

(f)One complete EBT module substitutes one operator proficiency check (OPC).

(g)The line evaluation of competence substitutes the line check.

EBT PROGRAMME AND ASSSESMENT AND TRAINING TOPICS — RESILIENCE

(a)Compliance with the table of assessment and training topics ensures that crews are presented with an array of realistic changing events that allow for resilience development purposes.

(b)The EBT programme should be designed observing the following principles for resilience development:

(1)Resilience, surprise, and unexpected events The EBT programme should be designed in such a way that in every cycle the simulator session (or part of it) allows variations so that the pilots are not familiar with the scenarios presented in the simulator session. Variations should be the focus of EBT programme design, and should not be left to the discretion of individual instructors, in order to preserve programme integrity and fairness.

(2)Resilience and decision-making (dilemma) The EBT programme should be designed in such a way that in every cycle the crews are exposed to a scenario where more than one possible and less than ideal solutions exist, with some unfavourable conditions attached to each solution.

VALIDITY OF THE EBT MODULE

(a)The validity period should be counted from the end of the month when the module was completed. When the module is undertaken within the last 3 months of the validity period, the new validity period should be counted from the original expiry date.

(b)In the context of ORO.FC.130 point (a), the pilot should have a valid module.

EBT PROGRAMME AND ASSSESMENT AND TRAINING TOPICS — RESILIENCE

(a)For resilience development, crews should be exposed to an array of realistic changing scenarios. The strategies developed by the crews whilst coping with different causes of action will create opportunities for resilience development.

(b)Resilience and surprise The operator may create a comprehensive list of scenarios to ensure that each crew is trained in different scenarios avoiding the same scenarios for all crews. This relates to training topic ‘surprise’ and to the customisation of the EBT programme.

(c)Resilience and unexpected events Exposing crews to rare, fortuitous, events may prepare crews to deal with other unexpected events. For instance, the table of assessment and training topics offers infrequent example scenario elements such as flying over ‘no fly zone’, etc. The operator may also take infrequent examples from occurrence reporting, or SMS, or manufacturer reports, etc. This relates to decision-making (PSD) — see OB 6.9 ‘Demonstrates resilience when encountering an unexpected event’.

(d)Dilemma The operator may create scenarios suitable for training of threat assessment, threat management processes and option generation, leading to an optimum decision-making process. At programme design, as in real life, one ‘correct answer’ should be avoided; instead, the EBT programme should offer the crews a number of less than ideal courses of actions; some with unfavourable conditions attached. This relates to decision-making (PSD) and to the contextualisation of the example scenario element.

EBT PROGRAMME —TRAINING PHASE — IN-SEAT INSTRUCTION (ISI)

(a)Effective monitoring and error detection are increasingly important when operating highly reliable automated aircraft.

(b)In-seat instruction may be used as a valuable tool to maintain and develop the training objectives of some of the training topics, such as skills of monitoring, cross-checking, error management, and recognition of mismanaged aircraft state.

EBT PROGRAMME — ORDER OF THE PHASES The order of the phases is intended as follows:

(a)First, the EVAL; and

(b)Second, and in a timely manner after the EVAL, the training phases. The training phases are the MT and the SBT and may be delivered in any order. Further guidance can be found in the EASA EBT manual.

EBT PROGRAMME — ENROLMENT

(a)Enrolment is when a flight crew member commences the first EBT module.

(b)A flight crew member is considered to leave the operator’s EBT programme (de-enrolled) when the operator is no longer responsible for the administrative action for the flight crew’s licence revalidation under an EBT programme.

(c)The operator should inform the flight crew members who fail to demonstrate an acceptable level of competence and leave the operator’s EBT programme (de-enrolled) that they should not exercise the privileges of that type rating.

MODULE SEPARATION BY A PERIOD OF NOT LESS THAN 3 MONTHS

(a)The separation begins when the first module finished (end of the training phase) and the second module begins (EVAL).

(b)When the operator decides to do more than two modules during the validity period of the type rating (approximately 1 year), the operator may count the 3 months of separation between the first and the third module if it so wishes.

(c)The separation of 3 months applies even between modules in different validity periods.

INSTRUCTOR CONCORDANCE ASSURANCE PROGRAMME (ICAP)

(a)The ICAP should be able to identify areas of weak concordance to drive improvement in the quality and validity of the grading system.

(b)The ICAP should be adapted to the size and complexity of the instructors’ group and the complexity of the operator’s EBT programme.

(c)Complex operators should include an ICAP-specific data analysis, demonstrating:

(1)instructor-group assessment homogeneity (agreement);

(2)instructor assessment accuracy (alignment).

(d)The operator should verify the concordance of the instructors:

(1)once every cycle;

(2)for a sufficient number of competency-grade combinations.

(e)The operator should establish procedures to address those instructors who do not meet the standards required.

(f)The operator should maintain a list with the EBT instructors qualified to deliver the EBT programme.

INSTRUCTOR CONCORDANCE ASSURANCE PROGRAMME (ICAP)

(a)Instructor concordance is a tool for continuous improvement of the EBT programme as data reliability results in a more accurate and effective training.

(b)The operator may have a more frequent, or even a continuous, assessment of concordance as it provides more opportunities to improve.

(c)Concordance standards are normally set by the operator; however, the competent authority may recommend criteria, as licences’ revalidation is performed under EBT.

(d)Individual instructor concordance may be verified:

(1)through uniform standardisation material where at least three different levels of performance are included and for all the competencies at a frequency of 72 months;

(2)by reference to the analysis of the data produced by the instructor every 12 months; normalisation may be necessary as there is no homogeneity of all EBT modules and the pilots that the instructor assessed; and

(e)Instructor-group assessment homogeneity (agreement) may be inferred from instructors who have observed the same content.

(f)Instructor assessment accuracy (alignment) may be inferred from comparing instructor assessments with an ‘assessment standard’ consisting of correctly identified competency(-ies) and correctly identified grade levels. Neither the competency(-ies) nor the grade level(s) may be communicated in advance to the instructors. The assessment standards may be set by consensus of a standards group, in order to guard against individual biases.

(g)When the operator uses a small group of instructors (e.g. 10), the data-driven concordance assurance programme may be directly integrated into the annual refresher training, removing the need for the above guidance.

(h)Operators with a complex group of instructors (e.g. a big rotation of instructors, subcontracted instructors, big number of instructors, many different fleets, etc.) may need to implement a more extensive concordance assessment system.

CONTINGENCY PROCEDURES FOR UNFORESEEN CIRCUMSTANCES THAT MAY AFFECT THE DELIVERY OF THE MODULE

(a)The operator should detail in the EBT programme the contingency procedures in the event of unforeseen circumstances that may affect the delivery of the module (e.g. long-term sick pilot).

(b)In case of unforeseen interruption of a module at any point, the missing parts of the module should be rescheduled.

(1)The pilot may continue line flying until the expiry of the validity period unless the performance observed was below the minimum acceptable level.

(2)If the interruption results in an instructor change, the operator should ensure that the instructor completing the module is provided with the details of the performance of the pilots.

(c)In case the pilot misses modules and does not meet the requirements of recent experience (FCL.060):

(1)when the pilot misses one module out of the two modules required, the EVAL of the missing module should be rescheduled before the pilot can resume line operations. The MT and SBT phases of the missing module should be completed 30 days after the EVAL or before the expiry date, whichever occurs first;

(2)when the pilot misses one module in the preceding 12 months but the pilot’s rating is expired by less than 3 months, the missing module should be rescheduled before the pilot can resume line operations;

(3)when the pilot misses one module in the preceding 12 months but the pilot’s rating is expired by longer than 3 months but shorter than 1 year, the missing module should be rescheduled. The evaluation should be delivered by an EBT instructor (or instructors) with examiner privileges before the pilot can resume line operations;

(4)when the pilot misses two modules and the pilot’s rating is valid:

(i)one module should be rescheduled before the pilot can resume line operations using an EBT instructor (or instructors) with examiner privileges; and

(ii)training topics B and C of the other module should be rescheduled before the expiry date. In such case, the 3-month separation requirement between modules may not apply;

(5)when the pilot misses two modules and the pilot’s rating is expired by less than 1 year:

(i)one module should be rescheduled using an EBT instructor (or instructors) with examiner privileges; and

(ii)training topics B and C of the other module should be rescheduled before the pilot can resume line operations. In such case, the period of 3-month separation between modules may not apply; and

(6)if the amount of time elapsed since the expiry of the rating is more than 1 year, the pilot is de-enrolled. AMC1 FCL.625(c) ‘IR — Validity, revalidation and renewal’ and AMC1 FCL.740(b)‘Validity and renewal of class and type ratings’ apply.

(d)In the case of other situations not covered by points (b) or (c), point (a) applies.

CONTINGENCY PROCEDURES — RATINGS RENEWAL

(a)The renewal of ratings (e.g. type rating or instrument rating) in EBT follows the Annex I (Part-FCL) to the Aircrew Regulation provisions (IRs and AMC) and is complemented with the provisions covered in AMC1 ORO.FC.231(a)(5). The ATO or the operator will determine the amount of training following Part-FCL; however, as EBT combines assessment and training, the following guidance is applicable:

(1)Expiry shorter than 3 months may not require additional training in Part-FCL. In EBT, the missing module is rescheduled with an EBT instructor. Following that, the EBT manager for the type rating may renew the licence without extra training, as the EBT programme is now completed (at least two modules in the last 12 months).

(2)In Part-FCL, when the expiry is longer than 3 months but shorter than 1 year, there need to be two training sessions. In EBT, there are two cases:

(i)One module is missing: the pilot must complete the missing module (two simulator sessions) before resuming line operations. Following that, the EBT manager for the type rating may renew the licence in accordance with Appendix 10 as the EBT programme is now completed (two modules in the last 12 months).

(ii)Two modules are missing: the pilot must complete one module (two simulator sessions) and training topics B and C of the other missing module (an extra simulator session) with a total of three simulator sessions. Training data is gathered in a short time period; therefore, an EBT instructor with examiner privilege is involved to ensure the proficiency of the pilot.

(b)In case of an expiry longer than 1 year, the requirements of Part-FCL will be followed and the proficiency checks will be performed in accordance with Appendix 9 as the EBT system may not have sufficient training data for the pilot. Expiry longer than 1 year but shorter than 3 years: a minimum of three training sessions in which the most important malfunctions in the available system are covered plus a proficiency check in accordance with Appendix 9 to renew the licence.

RECOMMENDED EBT COMPETENCIES (EASA COMPETENCY FRAMEWORK)

(a)The operator should include in its EBT programme at least the following competencies:

ADAPTED COMPETENCY MODEL

(a)An operator seeking to develop an adapted competency model under ORO.GEN.120 should:

(1)identify positive behaviours and use language that avoids ambiguity; and

(2)demonstrate equivalence to the recommended EBT competencies in AMC1 ORO.FC.231(b).

(b)In order to demonstrate equivalence, the operator should map the competencies and observable behaviours to the recommended EBT competencies.

(c)When the operator is translating AMC1 ORO.FC.231(b) into its common language, the application of ORO.GEN.120 may not be necessary. The translation may not be literal.

ADAPTED COMPETENCY MODEL/POSITIVE OBSERVABLE BEHAVIOUR

(a)OBs should describe behaviours that contribute to positive pilot performance.

(b)The indicators should clearly describe how a competency is expected to be demonstrated by a crew member in the context of the operational environment.

(c)If the operator makes small adjustments in the wording used to describe the OBs of the EASA competency framework in order to improve the understanding of the pilots while maintaining the same meaning, it may be considered as EASA competency framework and not as an adapted competency model.

TRAINING SYSTEM PERFORMANCE — FEEDBACK PROCESS

(a)Feedback process is the continuous process of collecting and analysing assessment and training data from an EBT programme.

(b)The feedback process should use defined metrics to collect data in order to:

(1)identify trends and ensure corrective action where necessary;

(2)identify collective training needs;

(3)review, adjust and continuously improve the training programme;

(4)further develop the training system; and

(5)standardise the instructors (when the standardisation and concordance assurance programme is integrated into the training system performance).

(c)The following defined metrics should be collected as a minimum:

(1)level 0 grading metrics (competent metrics): data metrics providing the information whether the pilot(s) is (are) competent or not;

(2)level 1 grading metrics (competency metrics): quantifiable data from the grading system — numeric grade of the competencies (e.g. 1 to 5);

(3)level 2 grading metrics (observable behaviour metrics): the instructors record predetermined OBs during the session;

(4)level 3 grading metrics (other metrics): the instructors may record other data (e.g. abstract, specific tasks, actions, questions, etc.).

(d)Alternatively, where a system for the measurement of training system performance already exists, the operator may use it and, if necessary, adapt it to meet the demands of EBT.

FEEDBACK PROCESS — DATA PROTECTION – GRADING SYSTEM

(a)The objective of protecting the EBT data is to avoid inappropriate use of it in order to ensure the continued availability of such data, to maintain and improve pilot competencies.

(b)The data access and security policy should restrict information access to authorised persons.

(c)The data access and security policy should include the measures to ensure the security of the data (e.g. information security standard).

(d)The data access and security policy (including the procedure to prevent disclosure of crew identity) should be agreed by all parties involved (airline management and flight crew member representatives nominated either by the union or the flight crew themselves).

(e)The data access and security policy should be in line with the organisation safety policy in order to not make available or to not make use of the EBT data to attribute blame or liability.

(f)The operator may integrate the security policy within other management systems already in place (e.g. information security management).

TRAINING SYSTEM PERFORMANCE — FEEDBACK PROCESS — METRICS

(a)Training metrics within the feedback process are a valuable source of data. Typical metrics may include but are not limited to:

(1)differences in success rates between training topics;

(2)the trainees’ feedback (e.g. surveys), which provides a different perspective as to the quality and effectiveness of the training;

(3)instructor concordance assurance: this system is important to measure the effectiveness of the instructor calibration process. It is important to remind that the purpose of this system is not to spy on instructors or to pressure individuals to change their grading;

(4)level 0 grading metrics (competent metrics): Metrics examples: distribution of pilots not competent after the SBT, distribution of pilots not competent in the EVAL and competent after the SBT;

(5)level 1 grading metrics (competency metrics): Metrics examples:

(i)distribution of level of performance within the range of competencies;

(ii)differences in grades between aircraft types;

(6)level 2 grading metrics (observable behaviour metrics): e.g. in specific example scenario elements. Metrics example: differences in displaying OBs between ranks of pilots;

(7)level 3 grading metrics (other metrics such as data based on tasks): for instance, did the pilot calculate the landing distance? Or, did the pilots make a call-out in a specific manoeuvre? This level is usually linked to data collection of the SMS or EBT feedback loop (e.g. was the call-out of the TCAS manoeuvre correct? ‘TCAS I have control’). Metrics example: distribution of errors for various training scenarios and aircraft types.

(8)during the simulator session, the operator may consider the level of grading metrics that the instructor needs to collect, taking into consideration the workload of the instructor.

(b)Training metrics are an invaluable component in supporting an EBT programme, but they must be placed in the context of operational data because only the latter can justify the importance of specific training. For this purpose, data from the line evaluation of competence is important to measure the effectiveness of the EBT programme in operations. It may include data from the process for the monitoring of line operations.

(c)Complex operators may, in the context of their safety management system, establish a safety action group dedicated to training: ‘training safety action group’. This may be a best practice to meet the implementing rule.

FEEDBACK PROCESS — DATA PROTECTION – GRADING SYSTEM

(a)The data access and security policy may, as a minimum, define:

(1)a policy for access to information only to specifically authorised persons identified by their position in order to perform their duties. The required authorised person(s) does (do) not need to be the EBT manager; it could be the EBT programme manager or a third party mutually acceptable to unions or staff and management. The third party may also be in charge of ensuring the correct application of the data access and security policy (e.g. the third party is the one activating the system to allow access to the authorised persons);

(2)the identified data retention policy and accountability;

(3)the measures to ensure that the security of the data includes the information security standard (e.g. information security management systems standard e.g. ISO 2700x-ISO 27001, NIST SP 800-53, etc.);

(4)the method to obtain de-identified crew feedback on those occasions that require specific follow-up; and

(b)When there is a need for data protection, it is preferable to de-identify the data rather than anonymise it.

GRADING SYSTEM

(a)The grading system should provide quantifiable data for the measurement of the training system performance.

(b)The grading scale should be 1 to 5, where:

(1)Grade 1 — NOT COMPETENT — determines that the minimum acceptable level of performance was not achieved for the conduct of line operations. An outcome of ADDITIONAL TRAINING REQUIRED and level 2 grading metrics should be recorded.

(2)Grade 2 to 5 determine an outcome of COMPETENT for the conduct of line operations.

(3)Grade 2 (below the average) determines that the minimum acceptable level was achieved for the conduct of line operations. Additionally, level 2 grading metrics should be recorded. Minimum performance indicates a need for training (e.g. tailored or additional) to elevate performance. It includes:

(i)a competency graded continuously with 2 in multiple modules, or

(ii)the majority of competencies graded with 2 in a module.

(4)Grade 3 is the average.

(5)Grade 4 determines that the pilot is above the average.

(6)Grade 5 (exemplary) determines that the pilot is above the average and the outcome is enhanced safety, effectiveness and efficiency.

(c)The operator should develop further grading guidance to the above points to help the instructors determine the grade of the pilots they assess.

GRADING SYSTEM — ALTERNATIVE SYSTEM

(a)An operator seeking to develop an alternative grading system under ORO.GEN.120 should:

(1)provide quantifiable data for the measurement of the training system performance; and

(2)demonstrate equivalence to the recommended grading system in AMC1 ORO.FC.231(d)(1).

(b)The grading scale for each competency should:

(1)determine the grade at which the performance is considered:

(i)NOT COMPETENT for the conduct of line operations. An outcome of ADDITIONAL TRAINING REQUIRED and level 2 grading metrics should be recorded; and

(ii)COMPETENT for the conduct of line operations; and

(2)determine for the pilot whose performance is considered competent for the conduct of line operations:

(i)if the pilot needs more training (e.g. tailored or additional training) to elevate their performance to the operator specified norm;

(ii)if the pilot is at the operator specified norm;

(iii)if the pilot is above the average (it can be one or more grades e.g. above the average and exemplary).

(c)The operator should develop further guidance to the above points to help the instructors determine the grade of the pilots they assess.

RECOMMENDED CONDUCT OF THE GRADING — ORCA

(a)Grading the performance of flight crew members during an EBT module should include the following steps:

(1)Observe performance (behaviours) during the simulator session.

(2)Record details of effective and ineffective performance (behaviours) observed during the simulator session (‘record’ in this context refers to instructors taking notes).

(3)Classify observations against the OBs and allocate the OBs to each competency (or competencies), using amongst others the facilitation technique.

(4)Assess and evaluate (grade): assess the performance by determining the root cause(s) according to the competency framework. Low performance would normally indicate the area of performance to be remediated in subsequent phases or modules. Evaluate (grade) the performance by determining a grade for each competency using a methodology defined by the operator.

(b)As a minimum, the instructor should grade all the observed competencies at:

(1)the end of the EVAL (de-briefing) by providing at least level 1 grading metrics;

(2)the end of the MT (de-briefing) by providing at least level 0 grading metrics; and

(3)at the end of the EBT module (de-briefing) by providing at least level 0 grading metrics (level 1 grading metrics are recommended).

RECOMMENDED GRADING SYSTEM METHODOLOGY — VENN MODEL

(a)To grade a competency, the instructor should assess the associated OBs of each competency against the following dimensions by determining:

(1)what was the outcome of the threat management, error management and undesired aircraft state management relating specifically to the competency being assessed;

(2)how well the flight crew member demonstrated the OB(s) when they were required. This includes:

(i)how many OBs the flight crew member demonstrated over the EBT phase (e.g. EVAL, MT, SBT) when they were required; and

(ii)how often the flight crew member demonstrated the OB(s) when they were required;

(b)Grades should be determined during each EBT module as follows:

(1)EVAL — overall performance of the phase for each competency at level 1 grading metrics.

(2)MT — overall performance of the phase at level 0 grading metrics. When the phase is graded ‘not competent’, it requires level 2 grading metrics. Note: Only a limited number of competencies may be observed and graded in this phase (e.g. PRO, FPA, FPM); the others are ‘to be left in blank’.

(3)SBT — overall performance of the phase for each competency at level 1 grading metrics. Unless just culture and the necessary non-jeopardy environment during training may be compromised. In that case, level 0 grading metrics. Note: In-seat instruction (ISI) should not be included in any assessment.

(c)Where any competency is graded below the minimum acceptable level of performance (grade 1 on a 5-point scale), an outcome of additional FSTD training is required.

(1)Additional level 2 grading metrics must be recorded.

(2)The flight crew member should not be released to unsupervised line operations until each competency is demonstrated at or above the minimum acceptable level of performance.

(d)Where all competencies are determined at or above the minimum acceptable level of performance (grade 2 on a 5-point scale), the outcome should be COMPETENT. Consistent grading below the average (2 on a 5-point scale) may indicate a need for training to elevate the performance to the average (grade 3 on a 5-point scale). As a minimum, the following conditions apply:

(1)Any competency graded with 2 requires level 2 grading metrics.

(2)Any competency graded with 2 in any simulator session of the 1st module followed by a grade 2 in the same competency in the EVAL of the 2nd module requires individual tailored training in the SBT of the 2nd module. (First example: 1st Module SBT graded with 2, 2nd Module EVAL graded with 2 in the same competency, thus the 2nd SBT should be an individual tailored training on that competency. Second example: 1s module EVAL graded 2, 2nd module EVAL graded 2 on the same competency, thus the 2nd module SBT should be individual tailored training on that competency).

(3)Any competency graded with 2 in three consecutive modules requires individual tailored training. If at the end of the tailored training (3rd SBT) the competency continues being graded with 2, additional FSTD training is required within the next 3 months. For instance, following the example above, the SBT in the 2nd Module was an individual tailored training. In the 3rd Module during the EVAL the same competency is graded with 2 and individual tailored training is applied. The SBT is graded with 2 again. The pilot may continue line operations but should receive additional FSTD training within the next 3 months.

(4)The operator should not release a flight crew member to unsupervised line operations when more than four competencies (the majority of the competencies — five competencies or above) are graded with 2 in any single simulator session of the module.

(5)Any EVAL graded with 2 in more than three competencies requires individual tailored training in the SBT. If at the end of the module more than three competencies continue being graded with 2, the pilot may continue line operations but should receive additional FSTD training within the next 3 months.

(e)‘Individual tailored training’ refers to a simulator session tailored to the pilot’s individual training needs, which may require a different programme or syllabus. Normally, it may be done during the SBT and normally there is not an increase of FSTD volume (no extra simulator session). It may require an increased volume of training such as CBT, additional briefings, etc. Any individual tailored training may be substituted by additional FSTD training before the start of the next module.

(f)‘Additional FSTD training’ refers to the fact that in addition to the requirements of tailored training, there is an increase of FSTD volume (extra simulator session). It normally happens after individual tailored training.

RECOMMENDED CONDUCT OF THE GRADING — ORCA

(a)At the end of the EVAL, after the facilitated de-briefing, the instructor may, as a minimum, record level 1 grading metrics.

(b)The instructor may conduct the simulator session of the EVAL following the principles of a summative assessment and the facilitated de-briefing following the principles of a formative assessment. The MT and SBT simulator sessions may be conducted as a formative assessment.

(c)At the end of each training phase, it is recommended to record level 1 grading metrics unless just culture and the necessary non-jeopardy environment during training may be compromised. In that case, the following alternative may be recommended: level 0 grading metrics for all competencies may be recorded (exceptionally ‘not observed’ or ‘left in blank’ may be recorded) and de-identified level 1 grading metrics may be recorded for the data collection and analysis purposes.

(d)A simple practice to classify the observations recorded during the simulator session is to classify the OB as positive, negative, neutral.

RECOMMENDED GRADING SYSTEM METHODOLOGY — VENN MODEL

(a)Grades may be determined during each EBT module as follows:

(1)For each assigned grade:

(i)the observed performance should be identified with one or more OBs; and

(ii)the OB(s) should simply link the observed performance to the competency; they are not to be used as a checklist.

(2)At the completion of the EVAL, the grade should be the overall assessment of the performance of each competency during the EVAL. Although it is not recommended, if the instructor performs an overall grade (additional to level 1), it should be at level 0 grading metric (competent or not).

(3)The underlying philosophy of the individual tailored training and additional FSTD training is the identification of the pilot’s individual training needs during the EVAL or EVALs. However, there may be cases in which such an identification may be complemented using other phases or combination of phases along the EBT programme. Nevertheless, when this happens consistently to a large number of pilots, it may indicate a problem of instructor standardisation.

(4)At the completion of the MT, only a limited number of competencies can be graded. The others are to be left in blank. Note: The grade of a competency as ‘not observed’ is a relevant set of data to be used in the EBT programme (e.g. may be used for instructor concordance assurance programme, programme design, etc.), while ‘competency left in blank’ is stating the obvious, which is that MT is a skill retention phase and therefore it focuses on only some of the competencies which may provide NO opportunity to observe all the competencies.

(5)At the completion of the module, grades should be assigned for each competency, based on the overall assessment of training during the SBT.

(6)In exceptional occasions, the instructor may have been unable to assess one or two competencies in the EVAL or SBT. A ‘not observed’ may be graded. The training system performance and concordance assurance system may use these metrics to improve instructors’ standardisation and the EBT programme design. When the operator grades the MT alone (instead of grading the MT and EVAL together), a ‘not observed’ grading may be frequent. It also occurs when the instructor grades each one of the manoeuvres.

(b)The word pictures are standardised according to the VENN model but may be simplified once instructors become familiar with the system.

VERIFICATION OF THE ACCURACY OF THE GRADING SYSTEM

(a)The purpose is to provide data to assess the accuracy of the grading system.

(b)The items defined below are based on Part-FCL Appendix 9. They should be included in the EVAL and MT of the applicable module. The minimum items to be included are: rejected take-off, failure of critical engine between V1 & V2, 3D approaches down to a decision height (DH) not less than 60 m (200 ft), engine-out approach & go-around, 2D approach down to the MDH/A, engine-out approach & go-around, engine-out landing.

(c)Instructors should record if the exercises are flown to proficiency using Appendix 9 references (define criteria). Note: Individual pilots’ grading and assessment remains according to the EBT grading system and Appendix 10.

(d)This verification should be performed once every 3 years.

VERIFICATION OF THE ACCURACY OF THE GRADING SYSTEM Items that may be included in a verification of the accuracy of the grading system:

VERIFICATION OF THE ACCURACY OF THE GRADING SYSTEM — FEEDBACK PROCESS The verification of the accuracy of the grading system provides valuable data for the training system performance and concordance assurance. Therefore, the verification is necessary from a systemic point of view and the intention is not to measure individual pilot against Appendix 9 criteria. Concordance agreement between instructors may be high; however, the whole community of instructors may be grading too low or too high (accuracy). The statistical result of the verification against Appendix 9 criteria can provide the operator with a criterion-referenced system to adjust the accuracy of the grading system. The verification does not require an examiner; EBT instructors may provide the necessary data. Example 1: For the last 36 months, the operator has a rate of 3 % of pilots scoring 1 (assuming the data is statistically relevant). In this example, the rate of 3 % of the pilots scoring 1 is maintained across all the technical competencies. When the operator performs a verification, the rate of failure would have been only 0,5 %. This may indicate that instructors are rating too low in EBT and therefore some of the pilots scoring 1 should have been graded with a score higher than 1. This may be economically negative for the operator. On the other hand, it could be that the operator has decided to implement higher standards. Example 2: The operator has an EBT programme with a negligible rate of pilots scoring 1 and a 1 % of pilots scoring 2 in two consecutive recurrent modules. The verification of the technical competencies against Appendix 9 criteria provides a rate of 5 % failure. The EBT manager should further investigate the reason behind this mismatch between EBT and Appendix 9 in the technical competencies. There may be factors influencing this mismatch (e.g. statistical issues, the events in the EBT modules are too benign compared to the events in Appendix 9), which may lead to a corrective action (e.g. redesign of the EBT modules). If the difficulty of the EBT scenarios is equivalent to Appendix 9 and the concordance is high between instructors, then the discrepancy in outcomes might be because the community of instructors are grading too high in the technical competencies (they are grading with 2 when they should have graded 1). Further instructor standardisation will be needed to address this. The implementation of mixed EBT following GM1 ORO.FC.230(a);(b);(f) provides a good opportunity to fine-tune and verify the accuracy of the grading system because an Appendix 9 licence proficiency check is carried out every year. The authority may not allow full EBT unless the accuracy of the grading system is demonstrated. Further guidance can be found in the EASA EBT manual.

VOLUME AND FSTD QUALIFICATION LEVEL

(a)The EBT programme has been developed to include a notional exemplar of 48 FSTD hours over a 3-year programme for each flight crew member.

(b)Subject to ORO.GEN.120, the operator may reduce the number of FSTD hours provided that an equivalent level of safety is achieved. The programme should not be less than 36 FSTD hours.

(c)Each EBT module should be conducted in an FSTD with a qualification level adequate to complete proficiency checks; therefore, it should be conducted in a full-flight simulator (FFS) level C or D.

EQUIVALENCY OF MALFUNCTIONS — PROCESS

(a)The equivalency of malfunctions process should be undertaken by subject matter experts (SMEs) who hold or have held a type rating on the aeroplane type.

(b)Steps of the equivalency of malfunctions Step 1: Look at (review) all aircraft system malfunctions provided by the OEM. For example, FCOM for Airbus, or AFM for other manufacturers, does not normally provide an exhaustive list of malfunctions. Step 2: Determine and retain in a list only malfunctions that place a significant demand on a proficient crew, in isolation from an environmental or operational context. Step 3: For each retained malfunction, determine the applicable characteristic or characteristics. Step 4: Develop the EBT FSTD programme to incorporate malfunctions at the frequency specified in the table of assessment and training topics.

(c)Malfunctions included in the equivalency of malfunctions but not included in the EBT FSTD programme require review and appropriate procedural knowledge training, conducted in a less qualified but suitable alternative environment (classroom, flight procedure training device, advance computer-based training, aviation blended learning environment (ABLE), etc.). Further guidance can be found in the EASA EBT manual.

(d)The operator should establish procedures to determine what malfunctions should be included in the FSTD. This may include a different malfunction difficulty between the EVAL and the SBT.

CREW EXPOSURE TO AT LEAST ONE MALFUNCTION FOR EACH CHARACTERISTIC

(a)Unless specified in the OSD, each crew member should be exposed to the characteristics of degraded control and loss of instrumentation in the role of pilot flying.

(b)Notwithstanding point (a), for aircraft types with a limited number of malfunctions in the characteristic of degraded control or loss of instrumentation, the operator may use an alternative means of compliance in accordance with ORO.GEN.120.

EQUIVALENCY OF MALFUNCTIONS — SIGNIFICANT DEMAND ON A PROFICIENT CREW

(a)The criteria to determine that a malfunction places a significant demand on a proficient crew are the following:

(1)The procedure includes one or more action items and not only a set of information for crew awareness.

(2)The flight crew’s cognitive load (resources required by the mental processes of perception, memory, judgement, and reasoning) significantly increases during or after the application of the associated abnormal or emergency procedure. The cognitive load is considered to be significantly increased when it is well above the cognitive load induced by the application of the normal standard operating procedures.

(3)The flight crew’s workload significantly increases during or after the application of the associated abnormal or emergency procedure. The workload is considered to be significantly increased when it is well above the workload induced by the application of the normal standard operating procedures.

(4)The aircraft handling perceived by the pilot when flying in abnormal conditions is different compared to the aircraft handling in normal conditions; e.g. the symmetry of the flight is affected.

(b)The criteria to determine that a malfunction places a significant demand on a proficient crew allow the identification of:

(1)the pilot competencies that are specifically challenged during the management of the related procedure, and

(2)the characteristic of the aircraft system malfunction procedure. Note: The identification of the pilot competencies allows a consistent assessment to determine the proficiency of the crew member.

(c)When a malfunction is placing a significant demand on a proficient crew, it means it has one or more of the malfunction characteristics (see more in GM2.ORO.FC.231(f)).

EQUIVALENCY OF MALFUNCTIONS — MALFUNCTION CHARACTERISTICS The following may be considered suitable definitions for each of the characteristics:

(a)‘Immediacy’: System malfunctions that require immediate and urgent crew intervention or decision (e.g. malfunctions with memory items, loss of pressurisation at high altitude, brake failure during landing).

(b)‘Complexity’: System malfunctions that require recovery procedures with multiple options to analyse and/or multiple decision paths to apply (e.g. multiple hydraulic system failures, smoke and fumes procedures).

(c)‘Degradation of aircraft control’: System malfunctions that result in significant degradation of flight controls in combination with abnormal handling characteristics, such as modification of the normal pitch attitude during approach and landing or reconfiguration of the flight control laws or modes (e.g. jammed stabiliser, flaps/slats inoperative)

(d)‘Loss of instrumentation’: System malfunctions that require monitoring and management of the flight path using degraded or alternative displays such as temporary or permanent loss of any flight-path-related parameter displayed on the primary flight display (PFD), head-up display (HUD) or navigation display (ND), including loss of any setting capability of one of these indications. It includes primary instrumentation to monitor and manage primary aircraft systems (e.g. FLAPS indication, loss of fuel indications, etc.).

(e)‘Management of consequences’: System malfunctions that affect significantly the flight crew standard task sharing and/or the workload management and/or the decision-making process during an extensive period after the management of the malfunction itself (e.g. fuel leak or fuel not usable, altitude/speed limitations, malfunctions with ‘deferred’ items in later flight phases). Note: Equivalency of malfunctions may be undertaken in consultation with the aircraft OEM. The objective of the OEM consultation is to review the operator analysis regarding the OEM operational certification (e.g. OSD) documents and the general OEM operation and training policy.

EQUIVALENCY OF MALFUNCTIONS — ISOLATION FROM AN ENVIRONMENTAL OR OPERATIONAL CONTEXT When considering significant demand on a proficient crew, SMEs may consider that there are no significant environmental and operational threats. For example, the aircraft is close to a suitable aerodrome with environmental conditions permitting all published approaches to be made, with no pre-existing malfunctions and sufficient fuel for several hours (e.g. A320 or B737 overhead Ibiza - Spain, at FL350 with visible moisture at 30 000 ft, at the aerodrome wind calm, CAVOK, ISA).

EQUIVALENCY OF MALFUNCTIONS PROCESS — DELPHI

(a)The operator reviews/looks at aircraft system malfunctions provided in the official documentation of the OEM — for example, FCOM for Airbus, or AFM for other manufacturers.

(b)Before launching the equivalency of malfunctions survey and when the aircraft system malfunctions list is very long, the operator may slightly shorten the list by removing the malfunctions that surely will not place a significant demand of a proficient crew (see GM on SIGNIFICANT DEMAND ON A PROFICIENT CREW).

(c)A group of EBT instructors statistically relevant will be selected to perform the equivalency of malfunctions survey. 50 % of the instructors’ community will be used as a reference. In small instructors’ communities, it may be necessary to refer to 100 %. In operators with large instructors’ communities, the number of instructors statistically relevant may be less than 50 %.

(d)The group of instructors selected in point (c) will rate each of the malfunctions listed in points (a) and (b)

(1)Each instructor will rate each one of the 5 characteristics in each malfunction listed in point (b).

(2)The rate will be 0 when the malfunction does not have the characteristic (the characteristic does not appear in the malfunction).

(3)The rate will be 1 to 5 when the characteristic appears in the malfunction. Rating 1 when the characteristic is not relevant for the malfunction and rate 5 when the characteristic is very relevant.

(4)The instructors will rate individually (e.g. home, classroom, etc.) to avoid exchange of opinions with other instructors.

(e)An average rate of the whole instructors’ community as a result of point (d) will be calculated for each characteristic of each malfunction.

(f)A second round of survey will be performed with the same instructors and the same list. This time the operator will provide the average calculated in point (e) and ask them if in light of the average they would like to change their rating. Group discussion may substitute or complement the second survey.

(g)When an instructor changes their rating, the old rate will be discarded.

(h)A new average will be calculated for each characteristic of each malfunction at the end of the second survey. The final average will be rounded to the closest integer number.

(i)The operator may select an average rate of the characteristics (e.g. rate 2 or 3) at which or above which the characteristic is considered to be present in the malfunction, thus it places a significant demand on a proficient crew.

(j)The operator may use the rates of the characteristics to determine the difficulty of the malfunction. As SBT is a developing phase, the operator may select a higher difficulty of the malfunctions selected in this phase. Further guidance can be found in the EASA EBT manual.

(k)The operator may refer to an aircraft OEM malfunction analysis to support all the steps of the session.

(l)A simpler version of the process may be acceptable provided that:

(1)the aircraft manufacturer provides equivalency of malfunction documentation;

(2)there is a minimum of three EBT instructors who have a deep knowledge of aircraft systems; and

(3)the instructors referred to in (2) above are properly standardised. The standardisation is based on the EBT programme design knowledge and in particular the concept, definitions and process of the equivalency of malfunctions. The simplified process may or may not use a survey and use either a two-point scale (0 and 1), three-point scale (1, 2 and 3) or five-point scale (1 to 5).

APPROACHES THAT PLACE AN ADDITIONAL DEMAND ON A PROFICIENT CREW

(a)In order to identify approaches that place an additional demand on a proficient crew, an operator should:

(1)review its operational network;

(2)select approaches with one or more of the following characteristics:

(i)unusual design;

(ii)low frequency of exposure; and

(iii)degraded approach guidance;

(3)select at least one approach of each type and method and include them in the EBT programme at the frequency given in the table of assessment and training topics; and

(4)ensure the approaches selected in (3) cover all the characteristics at the frequency given in the table of assessment and training topics. Note: The approaches listed within Section 2 of the table of assessment and training topics should be selected in this process.

(b)Any approach that is required to be flown in the PF role specifically should be classified as ‘skills retention’ and may be trained in the MT.

EQUIVALENCY OF APPROACHES RELEVANT TO OPERATIONS — SPECIFIC APPROVAL The operator may extend the interval for recurrent training and checking of approaches that require specific approval as defined in the AMC to Part-SPA (e.g. SPA.LVO) to the frequency given in the EBT programme.

EQUIVALENCY OF APPROACHES RELEVANT TO OPERATIONS — APPROACH CHARACTERISTICS The following may be considered suitable examples for each of the approach characteristics:

(a)Design

(1)Unusual approach design feature — for example, offset final approach track or steep approach, etc.

(2)Unusual runway design feature — for example, non-standard lighting or marking

(b)Frequency

(1)Infrequently visited airfields — for example, alternate airfields

(2)Infrequently flown approaches at commonly visited airfields — for example, circling approach, CAT 2, SA CATI

(c)Degraded guidance

(1)Degraded internal guidance or aircraft equipment — for example, head-up display (HUD) failure

(2)Degraded external guidance or ground equipment — for example, GPS signal failure

SELECTED APPROACHES AT THE FREQUENCY GIVEN IN THE EBT PROGRAMME The table of assessment and training topics for each generation provides the type of approach, flight method and frequency for the crew.

LINE EVALUATION OF COMPETENCE

(a)The purpose of the line evaluation of competence is to verify the capability of the flight crew member(s) to undertake line operations, including preflight and post-flight activities as specified in the operations manual. Therefore, the line evaluation of competence should be performed in the aircraft. The route should be representative of typical sectors undertaken in normal operations. The commander, or any pilot who may be required to relieve the commander, should also demonstrate their competency in the role.

(b)Each flight crew member should be assessed according to the competency framework and grading system approved for their operator’s EBT programme.

(c)Flight crew members should be assessed in duties as pilot flying and pilot monitoring; they should be evaluated in each role. Therefore, they should be checked on one flight sector as pilot flying and on another flight sector as pilot monitoring.

(d)The operator should maintain a list and inform the competent authority about the line evaluators suitably qualified to undertake line evaluations of competence.

(e)The person that conducts the line evaluation of competence should occupy an observer’s seat. For aeroplanes, in the case of long-haul operations where additional operating flight crew members are carried, the person that conducts the line evaluation of competence may fulfil the function of a cruise relief pilot and should not occupy either pilot’s seat during take-off, departure, initial cruise, descent, approach and landing.

(f)The validity period should be counted from the end of the month when the line evaluation of competence was undertaken. When the line evaluation of competence is undertaken within the last 3 months of the validity period, the new validity period should be counted from the original expiry date.

LINE EVALUATION OF COMPETENCE — LINE EVALUATOR

(a)The line evaluator should have a valid line evaluation of competence.

(b)The line evaluator should receive an acceptable training based on the EBT instructor training. The EBT assessment of competence is not required.

LINE EVALUATION OF COMPETENCE — EXTENSION OF THE VALIDITY In order to extend the validity of the line evaluation of competence to:

(a)2 years, in every cycle, one EVAL for each pilot should be conducted by an EBT instructor (EBT instructors) who has (have) a valid line evaluation of competence in the same operator;

(b)3 years, in addition to point (a) above, the operator should have a feedback process for the monitoring of line operations which:

(1)identifies threats in the airline’s operating environment;

(2)identifies threats within the airline’s operations;

(3)assesses the degree of transference of training to the line operations;

(4)checks the quality and usability of procedures;

(5)identifies design problems in the human-machine interface;

(6)understands pilots’ shortcuts and workarounds; and

(7)assesses safety margins.

LINE EVALUATION OF COMPETENCE

(a)Line evaluation of competence, route and aerodrome knowledge, and recent experience requirements are intended to verify the capability of the flight crew member(s) to operate safely, effectively and efficiently under line operating conditions, including preflight and post-flight activities as specified in the operations manual. Other EBT assessments, legacy checks and emergency and safety equipment training are primarily intended to prepare flight crew members for abnormal/emergency procedures.

(b)The line evaluation of competence is considered a particularly important factor in the development, maintenance and refinement of high operating standards, and can provide the operator with a valuable indication of the usefulness of its training policy and methods.

LINE EVALUATOR

(a)AMC1.ORO.FC.146(c) ‘EBT instructor training’ provides some learning objectives which may be used to qualify the commander nominated by the operator to perform line evaluation of competence. The training may be a minimum of 7 hours, where 1 hour may be done outside the classroom. The use of advance training environments such as advance computer-based training or ABLE may reduce further the need of classroom training. The assessment of competence may not be required. Further guidance can be found in the EASA EBT manual.

(b)The line evaluator training may be included in the EBT instructor standardisation and concordance programme. This option is however limited due to the limited number of line evaluations of competence that are required (every 2 or 3 years), the difficulties in observing the whole range of performance of competencies and the lack of control of the environment during a line evaluation of competence. Therefore, the operator may need to use EBT instructors to maintain an acceptable level of standardisation.

PERFORMANCE-BASED CONTINUOUS TECHNICAL GROUND TRAINING

(a)Technical ground training programme

(1)The objective of the technical ground training programme is to ensure that pilots have adequate:

(i)knowledge of:

(A)the aircraft systems; and

(B)the operational procedures and requirements; and

(ii)awareness of:

(A)the most significant accidents or incidents that could affect their operations following the ‘threat and error management model’ or an alternative risk model agreed with the authority; and

(B)the occurrences in the airline or occurrences from other airlines that may be relevant for their operations, accident/incident and occurrence review.

(2)The technical ground training should:

(i)be conducted as part of a 3-year programme;

(ii)allow a customisation of syllabi. The operator should describe in the operations manual the procedure to determine the customisation of syllabi. This customisation should be based on evidence both internal and external to the operator.

(iii)as a minimum, allow the pilot to receive technical ground training every 12 months. The validity period should be counted from the end of the month. When this training is conducted within the last 3 months of the validity period, the new validity period should be counted from the original expiry date.

(3)The technical ground training syllabi should be delivered using different methods and tools.

(i)The selection of the method and tool results from a combination of the learning objectives and the target group receiving the training (WHAT needs to be trained and WHO needs to be trained).

(ii)The selection of the appropriate method and tool should be driven by the desired outcome in terms of adequate knowledge.

(iii)The delivery of the technical ground training syllabi should include the methods or tools to verify if the pilot has acquired the objective of the technical ground training programme. This may be achieved by means a questionnaire, assessment of application of the competency ‘knowledge’ (KNO) or other suitable methods.

(4)The measurement and evaluation of the training system performance through the feedback process should include the performance of the technical ground training.

(b)Emergency and safety equipment training

(1)Training on the location and use of all emergency and safety equipment should be conducted in an aircraft or a suitable alternative training device.

(2)Every year the emergency and safety equipment training programme should include the following:

(i)actual donning of a life jacket, where fitted;

(ii)actual donning of protective breathing equipment, where fitted;

(iii)actual handling of fire extinguishers of the type used;

(iv)instruction on the location and use of all emergency and safety equipment carried on the aircraft;

(v)instruction on the location and use of all types of exits; and

(vi)security procedures.

(3)Every 3 years the programme of training should include the following:

(i)actual operation of all types of exits;

(ii)demonstration of the method used to operate a slide, where fitted;

(iii)actual firefighting using equipment representative of that carried on the aircraft on an actual or simulated fire except that, with Halon extinguishers, an alternative extinguisher may be used;

(iv)the effects of smoke in an enclosed area and actual use of all relevant equipment in a simulated smoke-filled environment;

(v)actual handling of pyrotechnics, real or simulated, where applicable;

(vi)demonstration in the use of the life rafts, where fitted; and

(vii)particularly in the case where no cabin crew is required, first aid appropriate to the aircraft type, the kind of operation and the crew complement.

(4)The successful resolution of aircraft emergencies requires interaction between flight crew and cabin/technical crew and emphasis should be placed on the importance of effective coordination and two-way communication between all crew members in various emergency situations.

(5)Emergency and safety equipment training should include joint practice in aircraft evacuations so that all who are involved are aware of the duties other crew members should perform. When such practice is not possible, combined flight crew and cabin/technical crew training should include joint discussion of emergency scenarios.

(6)Emergency and safety equipment training should, as far as practicable, take place in conjunction with cabin/technical crew undergoing similar training with emphasis on coordinated procedures and two-way communication between the flight crew compartment and the cabin.

(7)The emergency and safety equipment training should include a pilot’s assessment of the training received; as a minimum, by means of a questionnaire, or computer-based exercises, or other suitable methods.

(8)When the emergency and safety equipment training is conducted within 3 calendar months prior to the expiry of the 12-calendar-month period, the next emergency and safety equipment training should be completed within 12 calendar months of the original expiry date of the previous training.

(c)Emergency and safety equipment training — extension of period of training

(1)The emergency and safety equipment training programme should establish and maintain at least an equivalent level of proficiency achieved by complying with the provisions of (b). The level of flight crew proficiency in the use of emergency and safety equipment should be demonstrated prior to being granted approval to extend the period of training by the competent authority.

(2)The operator applying for an approval to extend the period of emergency and safety equipment training should provide the competent authority with an implementation plan, including a description of the level of flight crew proficiency to be achieved in the use of emergency and safety equipment. The implementation plan should comprise the following:

(i)A safety case which should:

(A)demonstrate that the required or equivalent level of proficiency in the use of emergency and safety equipment is maintained;

(B)incorporate the programme of implementation, to include controls and validity checks;

(C)minimise risk during all phases of the programme’s implementation and operation; and

(D)include oversight, including review and audits.

(ii)The measurement and evaluation of the training system performance through the feedback process should include the performance of the emergency and safety equipment training. The feedback should be used as a tool to validate that the emergency and safety equipment training is correctly implemented; this enables substantiation of the emergency and safety equipment training and ensures that objectives have been met.

(iii)Documentation that details the scope and requirements of the programme, including the following:

(A)the operator’s training needs and established operational and training objectives;

(B)a description of the process for designing and obtaining approval for the operator’s emergency and safety equipment training programmes. This should include quantified operational and training objectives identified by the operator’s internal monitoring programmes. External sources may also be used; and

(C)a description of how the programme will develop a support and feedback process to form a self-correcting training system.

(3)When the emergency and safety equipment training is conducted within 6 calendar months prior to the expiry of the 24-calendar-month period, the next emergency and safety equipment training should be completed within 24 calendar months of the original expiry date of the previous training.

PERFORMANCE-BASED CONTINUOUS GROUND TRAINING — INTERNAL AND EXTERNAL EVIDENCE

(a)Operator evidence (inner loop)

(1)Pilot data (individual or group);

(2)Population-based data according to the training metrics determined in the training system performance;

(3)Evidence identified or recognised through the safety management process covered in ORO.GEN.200.

(b)External evidence from the authority and manufacturers (external loop)

(1)Revision of existing rules and regulations, updated versions of the EBT data report, state safety plan;

(2)Training needs derived from updated OSD (if appropriate for ground training), etc.

(c)The evidence drives the selection of the methods and tools.

PERFORMANCE-BASED CONTINUOUS GROUND TRAINING — METHODS AND TOOLS This is a non-exhaustive list of methods and tools to deliver ground training: — classroom, presentations, — web-based training, — self-learning instructions, — advance CBT such as virtual reality, chatbots, interactive scenario trainers.