ATM/ANS.OR.C.005 Safety support assessment and assurance of changes to the functional system

GENERAL

(a)The safety support assessment should be conducted by the service provider itself. It may also be carried out by another organisation, on its behalf, provided that the responsibility for the safety support assessment remains with the service provider.

(b)A safety support assessment needs to be performed when a change affects a part of the functional system managed by a service provider other than an air traffic services provider and it is being used in the provision of its services. The safety support assessment or the way it is conducted does not depend on whether the change is a result of a business decision or a decision to improve the service performance.

SAFETY SUPPORT ASSESSMENTS BY PROVIDERS THAT ARE ALSO ATS PROVIDERS

(a)Only air traffic services providers can perform a safety assessment. Service providers other than air traffic services providers can only perform a safety support assessment to determine that the new or changed service behaves only as specified in a specified context.

(b)A safety support assessment should be carried out for changes that cross the organisation’s boundary.

(c)An air traffic services provider may choose not to perform a safety support assessment of changes to its functional system when the changes do not cross the organisation’s boundary. In this specific case, the safety assessment of changes to the functional system should be performed.

SAFETY SUPPORT ASSESSMENT

(a)A safety support assessment is needed whenever the functional system of a service provider other than an air traffic services provider changes. This may be as a result of:

(1)the provider proposing a change to:

(i)its functional system;

(ii)the services it provides;

(iii)the context in which its functional system operates; or

(iv)the context in which the service is provided;

(2)the services used by the provider in the delivery of its services being planned to change; or/and

(3)a change to the context in which the service provider’s functional system operates as a result of a proposed change by another service provider, another organisation regulated by Regulation (EC) No 216/2008 or an unregulated body.

(b)The granularity of the safety support case report will depend on:

(1)the scope of the change;

(2)the nature and number of arguments; and

(3)the necessary and sufficient evidence needed to provide appropriate confidence that the safety support assurance is valid (complete and correct).

SCOPE OF THE CHANGE

(a)The description of the elements being changed includes the nature, functionality, location, performance, maintenance tasks, training and responsibilities of these elements, where applicable. The description of interfaces and interactions, between machines and between humans and machines, should include communication means, e.g. language, phraseology, protocol, format, order and timing and transmission means, where applicable. In addition, it includes the description of the context in which they operate.

(b)There are two main aspects to consider in evaluating the scope of a change:

(1)The interactions within the changed functional system.

(2)The interactions within the changing functional system, i.e. those that occur during transitions from the current functional system to the changed system. During such transitions, components are replaced/installed in the functional system. These installation activities are interactions within the changing functional system and are to be included within the scope of the change. As each transition can be treated as a change to the functional system, the identification of both the above has a common approach described below.

(c)The scope of the change is defined as the set of the changed components and affected components. In order to identify the impacted components and the changed components, it is necessary to:

(1)know which components will be changed;

(2)know which component’s (components’) behaviour might be affected by the changed components, although it is (they are) not changed itself (themselves); and

(3)detect indirectly affected components by identifying:

(i)new interactions introduced by the changed or directly affected components;

(ii)interactions with changed or directly affected components via the context. Furthermore, directly and indirectly impacted components will be identified as a result of applying the above iteratively to any directly and indirectly impacted components that have been identified previously. The scope of the change is the set of changed, directly impacted and indirectly impacted components identified when the iteration identifies no new components.

(d)The context in which the changed service is intended to be provided (see ATM/ANS.OR.C.005(a)(1)(iii)) includes the interface through which the service will be delivered to other service providers.

TRAINING If the change modifies the way people interact with the rest of the functional system, then they will require training before the change becomes operational. Care should be taken when training operational staff before the change is operational, as the training may change the behaviour of the operational staff when they interact with the existing functional system before any other part of the change is made, and so the training may have to be treated as a transitional stage of the change. For example, as a result of training, ATCOs may come to expect information or alerts to be presented differently. People may also need refreshment training periodically in order to ensure that their performance does not degrade over time. The training needed before operation forms part of the design of the change, while the refreshment training is part of the maintenance of the functional system after the change is in operation.

INTERACTIONS The identification of changed interactions is necessary in order to identify the scope of the change because any changed behaviour in the system comes about via a changed interaction. Changed interaction happens via an interaction at an interface of the functional system and the context in which it operates. Consequently, identification of both interfaces and interactions is needed to ensure that all interactions have identified interfaces and all interfaces have identified interactions. From this, all interactions and interfaces that will be changed can be identified.

FORM OF ASSURANCE Service providers other than air traffic services providers should ensure that the assurance is documented in a safety support case.

COMPLETENESS OF THE ARGUMENT The argument should be considered complete when it shows that:

(a)the safety support assessment of ATM/ANS.OR.C.005(b) has produced a service specification and context specification where:

(1)the service has been defined in terms of functionality, performance and the form of the interfaces;

(2)the specification of context correctly and completely records the conditions under which the specification of the service is true;

(3)the interaction of components, under failure conditions or failures in services delivered to the components, have been assessed for their impact on the service and, where necessary, degraded modes of service have been defined; and

(4)the specification encompasses the interaction with the environment;

(b)safety support requirements have been placed on the elements changed and on those elements affected by the change;

(c)the behaviour necessitated by the safety support requirements is the complete behaviour expressed by the service specification;

(d)all safety support requirements have been traced from the service specification to the level of the architecture at which they have been satisfied;

(e)each component satisfies its safety support requirements; and

(f)the evidence is derived from known versions of the components and the architecture and known sets of products, data and descriptions that have been used in the production or verification of those versions.

DETERMINATION OF THE SPECIFICATION OF THE CHANGED SERVICE When determining the changes in the service specification that have resulted from the change to the functional system, service providers other than air traffic services providers should ensure that:

(a)the properties specified for the service can be observed and measured either directly or indirectly with a degree of certainty commensurate with the level of confidence sought from assurance; and

(b)the specification of the changed service must cover everything that has changed in the service provided when operated within the declared operational context.

DETERMINATION OF THE OPERATIONAL CONTEXT FOR THE CHANGE

(a)When determining the operational context for the change, service providers other than an air traffic services provider should ensure that:

(1)the specification of the operational context can be shown to be true for all circumstances and environments in which the changed service is intended to operate;

(2)the operational context is completely and coherently specified; and

(3)the specification of the operational context is internally consistent.

(b)The operational context must be specified so that its adherence to (a)(1) and (a)(2) is observable and measurable either directly or indirectly with a degree of certainty commensurate with the level of confidence sought from assurance.

ASSURANCE — SOFTWARE

(a)When a change to a functional system includes the introduction of new software or modifications to existing software, the service provider should ensure the existence of documented software assurance processes necessary to produce evidence and arguments that demonstrate that the software behaves as intended (software requirements), with a level of confidence consistent with the needs of the required application.

(b)The service provider should use feedback of software experience to confirm that the software assurance processes are effective and, when used, the allocated software assurance levels (SWALs) and the rigour of the assurances are appropriate. For that purpose, the effects from software malfunctions (i.e. the inability of a programme to perform a required function correctly) or failures (i.e. the inability of a programme to perform a required function) reported according to the relevant requirements on reporting and assessment of service occurrences should be assessed in comparison with the effects identified for the system concerned as per the service specification demonstration.

ASSURANCE — SOFTWARE ASSURANCE PROCESSES

(a)The software assurance processes should provide evidence and arguments that they, as a minimum, demonstrate the following:

(1)The software requirements correctly state what is required by the software, in order to meet the service and safety support requirements, as identified by the safety support assessment (AMC2 ATM/ANS.OR.C.005(a)(2)). For that purpose, the software requirements should:

(i)be correct, complete and compliant with the upper level requirements; and

(ii)specify the functional behaviour, in nominal and downgraded modes, timing performances, capacity, accuracy, resource usage on the target hardware, robustness to abnormal operating conditions and overload tolerance, as appropriate, of the software.

(2)The traceability is addressed in respect of all software requirements as follows:

(i)Each software requirement should be traced to the same level of design at which its satisfaction is demonstrated.

(ii)Each software requirement allocated to a component should either be traced to an upper level requirement or its need should be justified and assessed that it does not affect the satisfaction of the safety support requirements allocated to the component.

(3)The software implementation does not contain functions that adversely affect the satisfaction of the service specification.

(4)The functional behaviour, timing performances, capacity, accuracy, resource usage on the target hardware, robustness to abnormal operating conditions and overload tolerance, of the implemented software comply with the software requirements.

(5)The software verification is correct and complete, and is performed by analysis and/or testing and/or equivalent means, as agreed with the competent authority.

(b)The evidence and arguments produced by the software assurance processes should be derived from:

(1)a known executable version of the software;

(2)a known range of configuration data; and

(3)a known set of software items and descriptions, including specifications, that have been used in the production of that version, or can be justified as applicable to that version.

(c)The software assurance processes should determine the rigour to which the evidence and arguments are produced.

(d)The software assurance processes should include the necessary activities to ensure that the software life cycle data can be shown to be under configuration control throughout the software life cycle, including the possible evolutions due to changes or problems’ corrections. They should include, as a minimum:

(1)configuration identification, traceability and status accounting activities, including archiving procedures;

(2)problem reporting, tracking and corrective actions management; and

(3)retrieval and release procedures.

(e)The software assurance processes should also cover the particularities of specific types of software such as commercial-off-the-shelf (COTS), non-developmental software and previously developed software where generic assurance processes cannot be applied. The software assurance processes should include other means to give sufficient confidence that the software meets the service and safety support requirements. If sufficient assurance cannot be provided, complementary mitigation means aiming at decreasing the impact of specific failure modes of this type of software, should be applied. This may include but is not limited to:

(1)software and/or system architectural considerations;

(2)existing service level experience; and

(3)monitoring.

SPECIFICATION ‘Continue to behave only as specified in the specified context’ means that assurance needs to be provided that the monitoring requirements are suitable for demonstrating that the service behaves only as specified in the specified context during operation.

ASSURANCE LEVELS

(a)The use of assurance level concepts, e.g. design assurance levels (DALs), software assurance levels (SWALs), hardware assurance levels (HWALs), can be helpful in generating an appropriate and sufficient body of evidence to help establish the required confidence in the argument.

(b)The term ‘software assurance level (SWAL)’ is understood to be the level of rigour of the software assurances throughout the software lifecycle. In this context, the software life cycle is understood to be:

(1)an ordered collection of processes determined by an organisation to be sufficient and adequate to produce a software item;

(2)the period of the time that begins with the decision to produce or modify a software item and ends when the item is retired from service.

SAFETY SUPPORT REQUIREMENTS The complete behaviour is limited to the scope of the change. Safety support requirements only apply to the parts of a system affected by the change. In other words, if parts of a system can be isolated from each other and only some parts are affected by the change, then these are the only parts that are of concern and so will have safety support requirements attached to them. The following list contains examples, not exhaustive, of safety support requirements that specify:

(a)for equipment, the complete behaviour, in terms of functions, accuracy, timing, order, format, capacity, resource usage, robustness to abnormal conditions, overload tolerance, availability, reliability, confidence and integrity;

(b)for people, their performance in terms of tasks (e.g. accuracy, response times, acceptable workload, resilience to distraction, self-awareness, ‘team-playerness’, adaptability, reliability, confidence, skills, and knowledge in relation to their tasks);

(c)for procedures, the circumstances for their enactment, the resources needed to perform the procedure (i.e. people and equipment), the sequence of actions to be performed and the timing and accuracy of the actions; and

(d)interactions between all parts of the system.

VERIFICATION The service provider other than the air traffic services provider should ensure that verification activities of the safety support assessment process include verification:

(a)that the full scope of the change is addressed throughout the whole assessment process, i.e. all the elements of the functional system or environment of operation that are changed or affected by the change and those unchanged elements that depend upon them and on which they depend are identified;

(b)that the way the service behaves complies with and does not contradict any requirements placed on the changed service by another part of the regulations or conditions attached to the providers’ certificate;

(c)that the specification of the way the service behaves and the safety support requirements are complete and correct;

(d)that the specification of the operational context is complete and correct;

(e)that the specification was analysed in the context in which it is intended to operate;

(f)of the completeness of the argument as per AMC2 ATM/ANS.OR.C.005(a)(2);

(g)that the safety support requirements are correct and complete by reference to the specification; and

(h)to the intended degree of confidence, that the implementation satisfies the safety support requirements and behaves only as specified in the given operational context.

DESCRIPTION OF THE SCOPE — ‘MULTI-ACTOR CHANGE’ In the case where the change is a ‘multi-actor change’ in reference to ATM/ANS.OR.A.045(e), the interfaces and interactions include the interfaces with the other service providers and/or aviation undertakings that are also affected by the change. Information related to cooperatively identifying the scope of ‘multi-actor changes’ may be found in EUROCAE ED-78A.

VERIFICATION This requirement is seeking verification because it is a simple cross-check of available material, i.e. that the specification reflects the requirements of other parts of this Regulation.

(a)Behaviour ATM/ANS.OR.C.005(b)(1)(ii) requires that the service meets its specification. Consequently, the specification must be complete and valid, i.e. it includes the behaviour addressed in ATM/ANS.OR.C.005(b)(1)(iii) and any additional behaviour in the specified context.

(b)Compliance with other requirements

(1)ATM/ANS.OR.C.005(b)(1)(iii) requires the service providers to identify all parts of this Regulation that impose behaviour on the changed service and also includes any conditions attached to the certificate. They have to identify only those parts of this Regulation that describe required behaviour relevant to the changed service. The identified behaviour shall be included in the specification of the changed service. Note that the Regulation or conditions attached to the certificate may render compliance with technical standards and ICAO SARPs mandatory.

(2)Compliance with other non-mandatory standards may also be a necessary condition for other reasons.

(3)ATM/ANS.OR.C.005(b)(1)(iii) does not state that the service only meets the requirements of the other parts of this Regulation. It may do other things as well, as described in (5) below.

(4)In ATM/ANS.OR.C.005(b)(1)(iii), ‘does not contradict’ is used to express the concern that behaviour beyond that required by a standard might cause the behaviour required by the standard to be undermined.

(5)The behaviour of a service is likely to include behaviour unspecified in standards; such behaviour may come from:

(i)the behaviour of degraded modes of operation;

(ii)additional behaviour not required by the standard, but put there for commercial purposes, e.g. competitive edge; or

(iii)other behaviour identified by the customer, e.g. an air traffic services provider.

(6)Consequently, the total behaviour should be specified.

MONITORING Service providers other than an air traffic services provider should ensure that within the safety support assessment process for a change, the monitoring criteria that are to be used to demonstrate that the safety support case remains valid during the operation of the changed functional system, i.e. that the changed service continues to meet its specification, are identified and documented. These criteria should be such that:

(a)they indicate that the assumptions made in the safety support case remain valid; and

(b)if the properties being monitored remain within the bounds set by these criteria, the service will be behaving as specified.

MONITORING OF INTRODUCED CHANGES

(a)Monitoring is intended to maintain confidence in the safety support argument during operation of the changed functional system. Monitoring is, therefore, only applicable following the entry into service of the change.

(b)Monitoring is likely to be of internal parameters of the functional system that provide a good indication of the performance of the service. These parameters may not be directly observable at the service level, i.e. at the interface of the service with the operational environment. For example, where a function is provided by multiple redundant resources, the availability of the function will be so high that monitoring it may not be useful. However, monitoring the availability of individual resources, which fail much more often, may be a useful indicator of the performance of the overall function.