CSV vs. CSA: What Are the Main Differences?

CSV and CSA both play the same role in the digitally matured life sciences companies. Yet, these approaches have some key differences. In this article, we will review the differences between CSV and CSA.

Until very recently, CSA was seen as a futuristic approach for software validation, but this perception changed as the FDA published its latest draft guidance, “Computer Software Assurance for Production and Quality System Software,” on September 13, 2022. This guidance will supplement the FDA's earlier published guidance, “General Principles of Software Validation,” and supersede Section 6 (“Validation of Automated Process Equipment and Quality System Software”) of the software validation guidance.

To compare and differentiate the CSV and CSA processes, let’s first look at the CSV and CSA processes separately.

What Is Computer System Validation?

As the concept of CSV came first, let’s start there. CSV is a software validation process that confirms the software specifications' adherence to user needs and intended uses. The CSV process also ensures that the software requirements are consistently fulfilled. The CSV process relies heavily on examination to provide objective evidence to prove conformance. 

The CSV approach has been there since the publication of the CSV guideline in 2003, in addition to the 21 CFR Part 11. The FDA’s “Guidance for Industry Computer Systems Used in Clinical Trials” applies to the computerized systems used to create, modify, maintain, archive, retrieve, or transmit clinical data intended for submission to the FDA because the clinical data have broad public health significance and must be of the highest quality and integrity. The common factor in CSV and CSA is that they are both software validation approaches, but the CSV approach requires extensive documentation of the scripted tests and test results in the form of evidence such as screenshots and documentation.

How to Plan the CSV Documentation Process

The CSV documentation process begins by answering questions such as:

1. 1. What will be validated?
      The software’s name and version number. 
      
      
   2. What will be the acceptance criteria?
      The anticipated test results for different types of specifications, such as user requirement specifications (URS), functional specifications (FS), and design specifications (DS).
      
      
   3. How will it be validated?
      It relates to the 3Qs of software validation. The written strategy and tests will be performed for installation qualification (IQ), operational qualification (OQ), and performance qualification (PQ). 
      
      
   4. Who will validate?
      This is the stakeholders' role and responsibility.

The sequence of the above questions is essential. First, we need to know the software or software-related parts to be validated and the acceptance criteria. Without knowing the acceptance criteria, we cannot define the tests that need to be performed; without knowing the tests, we cannot define the roles and responsibilities.

How to Define Acceptance Criteria for CSV 

The acceptance criteria depend on the user requirements (URS), functional specifications (FS), and design specifications (DS). Acceptance criteria are also met if the URS, FS, and DS requirements are met.

User Requirements Specification (URS)

As the name suggests, the user requirement specifications are the user needs the software must fulfill. It also contains critical constraints, such as regulations, safety, operations, etc. For example, the following is a list of a few user requirements that might be needed for a lab system:

• • • Track training data of lab analysts on lab methods/techniques
    • Track samples received in the lab
    • Automatically assign tasks to the lab analysts based on availability and training
    • Send sample test results to the ERP
    • 21 CFR Part 11 Compliance
      
      

Functional Specifications (FS)

The functional specification document describes how the software functions and intends to meet the user's needs. The document might include descriptions of how specific user interface screens and reports should look or describe data that needs to be captured.

The functional requirements can also include logic, calculations, and regulatory requirements. For example, passwords and the audit trail should work to comply with the 21 CFR Part 11 requirements.

Design Specifications (DS)

The design specification document contains all of the technical elements of the software or systems, including:

• • • Database Design – field definitions, file structures, entity relationship diagrams, data flow diagrams, etc.
    • Logic/Process Design – pseudo code for calculations and logic
    • Security Design – hacker protection, virus protection
    • Interface Design – what data transfer will occur from one system to another, with what frequency and how; failure handling
    • Architectural Design – required hardware support, operating systems, application versions, middleware, etc.
    • Network requirements
    • Specific peripheral devices, such as scanners, printers, etc.

How to Perform and Report the CSV Tests

Each test in the software validation process verifies specific pieces of the planning and specifications that were used to design the system. This approach is based on the classic V model used in the GAMP guidelines. The model's left side addresses the requirements and specifications to define and build a system. The model's right side addresses the associated testing required to verify the requirements and specifications.

As can be seen in the above graphic:

• • The IQ tests are performed to evaluate the DS
  • The OQ tests are performed to evaluate the FS
  • The PQ tests are performed to evaluate the URS

In other words, the number of tests performed in a CSV approach is equal to the number of specifications (URS, FS, and DS together). The report should conform to the validation plan and include results for each test against the corresponding specifications. The results should also include screenshots of the tested specifications. This makes CSV a highly documentation-oriented approach, resulting in vast volumes of information that may be burdensome.

What Is Computer Software Assurance?

Computer software assurance is a risk-based approach for establishing and maintaining confidence that software is fit for its intended use. This approach considers the potential risk of compromised safety and/or quality of the device to determine the level of assurance effort and activities appropriate to establish confidence in the software. Because the computer system assurance effort is risk-based, it follows the least burdensome approach, where the burden of validation is no more than necessary to address the risk. Such an approach supports the efficient use of resources, in turn promoting product quality. In addition, computer software assurance establishes and maintains that the software used in production or the quality system is controlled throughout its lifecycle, meaning the software is always in the validated state.

The four simple steps in establishing computer system assurance are as follows:

1. Identifying the intended use

The US FDA recognizes that the level of assurance needed for a computer system depends on the software’s intended use. If the software is part of a production or quality system, then it needs a high level of assurance compared to software used as support for the production or quality system. Similarly, software not part of the production or quality system needs a low level of assurance.

2. Determining the risk-based approach

The FDA recommends using a risk-based analysis to determine the appropriate assurance activities. Broadly, the proposed risk-based approach in the draft guidelines entails systematically identifying reasonably foreseeable software failures, determining whether these failures pose a high process risk, and systematically selecting and performing assurance activities commensurate with the medical device or process risk, as applicable.

3. Determining appropriate assurance activities

FDA suggests that heightened risks of software features, functions, or operations generally entail greater rigor, i.e., a greater amount of objective evidence. Conversely, relatively less risk (i.e., no high process risk) of compromised safety and/or quality generally entails less collection of objective evidence for the CSA effort.

4. Establishing an appropriate record

When establishing the record, the manufacturer should capture sufficient objective evidence to demonstrate that the software feature, function, or operation was assessed and performed as intended.

How to Identify the Intended Use of Software for CSA

The intended use of software can be identified with the help of some rules of thumb described in the CSA draft guidelines. For example, software with the following intended uses is considered to be used directly as part of production or the quality system:

• • Software is intended for automating production processes, inspection, testing, or the collection and processing of production data; and
  • Software is intended to automate quality system processes, collect and process quality system data, or maintain a quality record established under the quality system regulation.

Software with the following intended uses is considered to be used to support production or the quality system: 

• • Software is intended for use as development tools that test or monitor software systems or that automate testing activities for the software used as part of production or the quality system, such as those used for developing and running scripts; and 
  • Software is intended for automating general record-keeping that is not part of the quality records.

On the other hand, software with the following intended uses generally is not considered to be used as part of production or the quality system, such that the requirement for validation in 21 176 CFR 820.70(i) would not apply: 

• • Software is intended for the management of general business processes or operations, such as email or accounting applications; and 
  • Software is intended for establishing or supporting infrastructure not specific to production or the quality system, such as networking or continuity of operations.

How to Determine the Risk-Based Approach for CSA

The risk-based analysis for production or quality system software should consider factors that may impact or prevent the software from performing as intended, such as proper system configuration and management, system security, data storage, data transfer, or operational error. Thus, a risk-based analysis for production or quality system software should consider which failures are reasonably foreseeable (as opposed to likely) and the risks resulting from each such failure. 

The FDA considers a software feature, function, or operation to pose a high process risk when its failure to perform as intended may result in a quality problem that foreseeably compromises safety, meaning an increased medical device risk. Examples of software features, functions, or operations that are generally high process risk are those that: 

• • Maintain process parameters (e.g., temperature, pressure, or humidity) that affect the physical properties of product or manufacturing processes identified as essential to medical device safety or quality; 

• • Measure, inspect, analyze, and/or determine acceptability of product or process with limited or no additional human awareness or review; 

• • Perform process corrections or adjustments of process parameters based on data monitoring or automated feedback from other process steps without additional human awareness or review; 

• • Produce directions for use or other labeling provided to patients and users that are necessary for the safe operation of the medical device; and/or 

• • Automate surveillance, trending, or tracking data that the manufacturer identifies as essential to medical device safety and quality. 
    
    

In contrast, the FDA considers a software feature, function, or operation not to pose a high process risk when its failure to perform as intended would not result in a quality problem that foreseeably compromises safety. Examples of software features, functions, or operations that generally are not high process risk include those that: 

• • Collect and record data from the process for monitoring and review purposes that do not have a direct impact on production or process performance; 

• • Software used as part of the quality system for corrective and preventive actions (CAPA) routing, automated logging/tracking of complaints, automated change control management, or automated procedure management; 

• • Software intended to manage data (process, store, and/or organize data), automate an existing calculation, increase process monitoring, or provide alerts when an exception occurs in an established process; and/or 

• • Software used to support production or the quality system, as explained above.
    
    

Based on the above examples, the Scilife platform would come under not high process risk as it is used as part of the quality system for corrective and preventive actions (CAPA) routing, automated logging/tracking of complaints, automated change control management, or automated procedure management.

How to Determine Appropriate Assurance Activities for CSA

The US FDA suggests that depending on the level of risk associated with the system, the following types of assurance activities can be performed for the CSA:

• • Unscripted Testing
    Dynamic testing in which the tester’s actions are not prescribed by written instructions in a test case, including: 

    • Ad-Hoc Testing – A concept derived from unscripted practice that focuses primarily on performing testing that does not rely on large amounts of documentation (e.g., test procedures) to execute.
    • Error-guessing – A test design technique in which test cases are derived on the basis of the tester’s knowledge of past failures or general knowledge of failure modes.

• • Exploratory Testing

    Experience-based testing in which the tester spontaneously designs and executes tests based on the tester’s existing relevant knowledge, prior exploration of the test item (including results from previous tests), and heuristic “rules of thumb” regarding common software behaviors and types of failure. Exploratory testing looks for hidden properties, including hidden, unanticipated user behaviors, or accidental use situations that could interfere with other software properties being tested and pose a risk of software failure.

• • Scripted Testing

    Dynamic testing in which the tester’s actions are prescribed by written instructions in a test case. Scripted testing includes both robust and limited scripted testing.

    • Robust Scripted Testing – scripted testing efforts in which the risk of the computer system or automation includes evidence of repeatability, traceability to requirements, and auditability. 
    • Limited Scripted Testing – a hybrid approach of scripted and unscripted testing that is appropriately scaled according to the risk of the computer system or automation.  This approach may apply scripted testing for high-risk features or operations and unscripted testing for low- to medium-risk items as part of the same assurance effort.

For high-risk software features, functions, and operations, manufacturers may consider more rigorous methods, such as the use of scripted testing or limited scripted testing, when determining their assurance activities. In contrast, for software features, functions, and operations not high-risk, manufacturers may consider using unscripted testing methods, such as ad-hoc testing, error-guessing, exploratory testing, or a combination of methods suitable for the risk of the intended use. 

How to Establish an Appropriate Record for the CSA

The FDA suggests the record should include the following: 

• • The intended use of the software feature, function, or operation;
  • The determination of the risk of the software feature, function, or operation; 
  • Documentation of the assurance activities conducted, including a description of the testing conducted based on the assurance activity; 
  • Issues found (e.g., deviations, failures) and the disposition; 
  • Conclusion statement declaring acceptability of the results; 
  • The date of testing/assessment and the name of the person who conducted the testing/assessment;
  • Established review and approval when appropriate (e.g., when necessary, a signature and date of an individual with signatory authority) 

To illustrate, let’s take a look at the example in the draft guidelines:

Summary

Now that we have provided a detailed overview of the CSV and CSA processes individually, let’s summarize the similarities between the two approaches:

• The similarity between CSV and CSA is that both require some tests to be performed and objective evidence to be generated. However, the most crucial difference between the CSV and CSA is that CSV is an objective, evidence-based approach without risk assessment. Therefore, the CSV process results in more tests and test evidence. As a result, the CSV process generates larger volumes of data in the form of reports. This makes CSV a more burdensome approach compared to CSA, as in the case of CSA, the number of tests to be performed depends on the potential impact of the failure mode of the specific feature on the process or medical device.
  
  
• The number of steps in the CSV and CSA process differs as follows
  

• Another difference between CSV and CSA is that the CSV is a validation process itself, whereas CSA always remains in the validated state.

In a nutshell, CSA is a more critical thinking-driven and efficient approach compared with the CSV approach. However, the choice of CSV vs. CSA may also depend on your objective. For example, as a computerized system vendor, you may prefer to rely on the extensive testing and evidence to leave no stone unturned. But if you are a user, it might make sense to prioritize testing of the failure modes for high-risk features or systems.

Discover Scilife’s validation strategy using a GAMP5 approach!