Appendix IX Assessment method for the multimedia-based training (MBT)

1.The purpose of this Appendix is to establish the requirements for the assessment and approval by a competent authority of any course that includes MBT in accordance with point 66.B.135. This Appendix may be used for the assessment of other training courses if the competent authority decides that the assessment method laid down in this Appendix are appropriate for such other courses. The assessment shall be conducted by the competent authority against all the criteria laid down in Table (A), grouped in four categories from (a) to (d). The competent authority shall clearly identify in the table the MBT product being assessed and its production and update versions.

2.The competent authority carrying out the assessment shall put itself in the position of the student or the end user and shall rate each criterion listed in Table (A) individually on a rating scale from 1 to 5, as follows: 1: Not acceptable. Does not meet the required criteria. 2: Partially acceptable, but improvement is needed to meet the required criteria. 3: Acceptable. Meets the required criteria. 4: Good. Meets the required criteria with enhancements made. 5: Excellent. Exceeds the required criteria.

3.If one or more of the criteria is rated below 3, an alternative learning process shall be requested by the competent authority in order to enhance the suitability of the product to an acceptable level.

4.Once the competent authority has rated each of the individual criteria listed in Table (A) , the following combined rating scale shall be used by the competent authority to determine the overall suitability level for each MBT learning resource: — 100–80: Excellent learning resource. It offers different functionalities and meets the required suitability criteria. — 79–60: The learning resource meets the required suitability criteria. — 59–40: The learning resource does not allow for a sufficiently worthy educational use. It can be used for ‘informal’ training only. — 39–20: The learning resource is below the average. It does not meet several required suitability criteria. Before approving the product, the competent authority shall check that the final score of the MBT is equal to or above 60, and that there is no single criterion that is rated below 3.

Table (A): Assessment for the multimedia-based training (MBT)

Notes: The following shall be taken into account by the competent authority when assessing the MBT against the individual criteria listed in Table (A): Categories:

(a)Academic quality The information presented in the multimedia resource shall have two characteristics: i. Reliability: the information is reliable, current, and relatively free of errors. The information complies with the current regulatory requirements. ii. Relevance: the information is relevant to the learning objectives defined for the course. It supports the student in achieving the learning objectives.

(b)Pedagogical quality The MBT emphasises the activities which promote the development of the required knowledge and skills. The main criteria for each product are related to three aspects: i. Pedagogical formulation/construction: it is characterised by the quality of simplification, the presence of summaries as well as the use of diagrams, figures, animations, and illustrations. It evaluates whether the structure of the learning resource promotes its use in a pedagogical context. This refers to the ease of orientation (summary, lesson plan), presence of appropriate interactions, usability (back, forward, scroll boxes, etc.), and communication resources (questions and answers, FAQs, forum, etc.) ii. Pedagogical strategies: teaching and learning styles should be based on active teaching approaches to build meaningful situations related to learning objectives and to learner motivation. iii. Student assessment methods: methods are implemented to measure the achievement of learning objectives.

(c)Didactic quality i. Learning activities: the content refers to real-life situations the student could possibly face in an actual maintenance environment. ii. Learning content: the content is adequate to meet the learning objectives.

(d)Technical quality This section assesses the design, browsing and technological aspects of the learning resources: i. Design: the content and organisation of the learning resource shall promote the appropriate use of colours, interactivity, graphic quality for selected images, animations, and illustrations. ii. Browsing: while navigating, the student should be able to find a plan, an index, or a detailed table of contents. The suggested choices or guidelines shall be clear and the groupings within the menus shall be consistent. iii. Technological aspects: multimedia techniques aim to combine and exploit the capacities of any new technology in education to enhance the transfer of knowledge. Therefore, the system shall favour the use of animations, simulations, or any other interactive elements.