Tower of Babel - Validation, Verification, Qualification, and Testing

By Daniel Riganati

Validation, verification, qualification, and testing – don’t they really mean the same thing? Well, yes. And no.
In Life Sciences, we use these terms to refer to different activities associated with computerized systems validation as shown in the figure below. To simplify the discussion, let’s start with these terms: validation, testing, and verification.

Validation

The FDA provides the following definition of validation that is used in the Life Sciences industry:

Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes.

This is a general definition of validation that has its roots in pre-computerized manufacturing processes, so it needs a little deconstruction in order to understand it in the context of information systems.
In manufacturing, a process is a sequence of actions or operations, performed by a person with or without equipment, to produce a product. In information systems, the sequence of operations is the application program, there may be equipment controlled by the computer system or the computer system itself may be the only equipment needed, and the product is information or data. So, one application of the FDA definition of validation to information systems is:

Establishing documented evidence which provides a high degree of assurance that a specific application program will consistently produce information or data meeting its predetermined specifications and quality attributes.

The two main activities that follow from this definition of validation are (1) testing of the computer system and (2) verification of the documentation associated with the computer system to control its operation and ensure its maintenance and also that the documentation is created, followed, and regularly reviewed for appropriateness.
Figure 1. Activities associated with computerized systems validation

Testing the Computer System (aka Verification and Qualification)

Testing is confirmation that a computer system meets its requirements and specifications. This is achieved through variously detailed levels of testing beginning with the installation of the hardware and system software, however, the term “qualification” is used because of the pre-computerized manufacturing roots of validation:

• Installation Qualification (IQ) – (FDA) Establishing confidence that process equipment and ancillary systems are compliant with appropriate codes and approved design intentions, and that manufacturer's recommendations are suitably considered. In other words: (1) installation of hardware and system software per the manufacturer’s instructions, or (2) in the cloud, the provisioning of a virtual machine per an approved procedure and the installation of system software per the manufacturer’s instructions

• Operational Qualification (OQ) – (FDA) Establishing confidence that process equipment and sub-systems are capable of consistently operating within established limits and tolerances. In other words: testing against the documented and approved requirements and specifications (unit, string, and integration testing per the documented and approved system design specifications; and system testing per the documented and approved functional requirements).

• Performance Qualification (PQ) – (1) (FDA) process performance qualification: establishing confidence that the process is effective and reproducible, or (2) (FDA) product performance qualification: establishing confidence through appropriate testing that the finished product produced by a specified process meets all release requirements for functionality and safety. In other words: user acceptance testing (UAT) against documented and approved user requirements.

Verification of Other Functional Controls (aka Validation and Qualification)

Verification concerns the confirmation that the documentation associated with the computer system to control its operation and ensure its maintenance is created, followed, and regularly reviewed for appropriateness. This is achieved through the creation, review, and approval of documents and records and the auditing of operational and QMS (quality management system) processes:

• URS, FRS, SDS, et al. – approved user, functional, and design documents to be used for testing computer systems and equipment

• SOPs (standard operating procedures) – documented and approved instructions to be followed for carrying out an operation or in a given situation

• Records produced by SOPs – provide objective evidence that procedures are being followed

• Training records – provide objective evidence that people are qualified to perform their roles, to operate computer systems and equipment, and to understand the procedures to be followed

• Periodic Review and Audits –to ensure documentation remains appropriate, to verify the existence of objective evidence (records) showing conformance to required procedures, and to assess how successfully procedures have been implemented

What’s the difference between qualification and validation?

Because of the manufacturing process roots of validation and its terminology, a distinction is sometimes made between qualification and validation.  Within the context of information systems the difference is that equipment and computer systems are qualified and computerized systems (including the controlled process) are validated.

Confused? There’s More

Adding to the confusion caused by these terms with similar and overlapping meanings, different organizations mix the terms and definitions.  Some organizations refer to verification as validation. Some define verification as dynamic testing and validation as static testing (i.e., peer review). Others refer to testing as verification or qualification. And others refer to qualification as validation. And because of the nature of custom software development, unit/string/integration testing is often referred to as IQ rather than OQ.
What’s important is not that we agree on terms, but that we understand all the activities associated with the validation of computerized systems and ensure that they are performed.
My strategy has been to understand the activities that are associated with computerized systems validation and then, with each consulting engagement, learn the terms the client organization assigns to those meanings.