Software assurance

Why does software assurance matter?

In today's digital world, software is used to control a wide range of devices and systems, including cars, medical devices, financial systems, and military equipment. Ensuring the reliability, safety, and security of software products is therefore critical. Without proper testing and verification, software can contain defects and vulnerabilities that can lead to system failures, security breaches, and other serious problems with negative consequences for individuals, businesses, and society as a whole.

The National Institute of Standards and Technology (NIST) defines software assurance as "the level of confidence that software is free from vulnerabilities, either intentionally designed into the software or accidentally inserted at any time during its life cycle, and that the software functions in the intended manner"^[22]. Organizations can reduce the risk of costly system failures, data breaches, and other negative outcomes by ensuring software assurance.

In addition to the potential risks associated with software defects and vulnerabilities, there are legal and regulatory requirements related to software assurance. Failure to comply with these regulations can result in legal and financial penalties. For example, organizations that develop software for certain industries may be subject to regulations that require them to ensure the safety and security of their products.

Many critical functions, such as national defense, banking, healthcare, telecommunications, aviation, and control of hazardous materials, depend on the correct and predictable operation of software. If the software-intensive systems that support these activities fail, they could be seriously disrupted. Therefore, it is essential for organizations to implement software testing and verification techniques and tools to reduce the risk of system failures and security breaches.

Execution

Software assurance is executed through a series of activities that aim to ensure the reliability, safety, and security of software products. These activities include requirements analysis, design reviews, code inspections, testing, and formal verification.^[1]

• Requirements analysis involves identifying and defining the functional and non-functional requirements of the software product. This includes understanding the purpose of the software, its intended users, and any relevant standards or regulations that must be met.
• Design reviews involve evaluating the software design to ensure that it meets the requirements and is implementable. This includes checking for design flaws, errors, or omissions that could impact the software's reliability, safety, or security.^[24]
• Code inspections involve reviewing the code to ensure that it conforms to industry-accepted standards and best practices. This includes checking for code quality, readability, and maintainability, as well as identifying potential vulnerabilities or security issues.^[2]
• Testing involves executing the software to identify defects or vulnerabilities that could impact its reliability, safety, or security. This includes functional testing, performance testing, and security testing, among other types of testing.^[3]
• Formal verification involves using mathematical or logical methods to prove that the software behaves as intended and does not contain any defects or vulnerabilities. This includes techniques such as model checking, theorem proving, and static analysis.^[1]

Techniques

Software testing and verification are techniques used to identify and address defects and vulnerabilities in software code. There are several types of testing and verification techniques, including functional testing, performance testing, and security testing.^[3]

• Machine learning is increasingly used in software assurance to detect software problems. With the ability to analyze large volumes of data, machine learning can identify patterns and anomalies that may go unnoticed by humans. The use of machine learning can ultimately improve the quality and security of software.
• Functional testing is used to ensure that the software performs the functions it is intended to perform. This type of testing involves running the software and testing its features and functionality to ensure that it works as expected. ^[4]
• Performance testing is used to measure the speed, responsiveness, and stability of the software. This type of testing involves simulating real-world scenarios to ensure that the software can handle the load and perform well under various conditions. ^[4]
• Security testing is used to identify vulnerabilities and weaknesses in the software that could be exploited by attackers. This type of testing involves using various techniques to attempt to breach the security of the software, such as penetration testing and vulnerability scanning.^[3]
• In addition to testing, verification techniques are used to ensure that the software code is correct and free of defects. This includes techniques such as code reviews, formal verification, and static analysis. ^[1]
• Code reviews involve checking the code to ensure that it conforms to industry-accepted standards and best practices. This includes checking for code quality, readability, and maintainability, as well as identifying potential vulnerabilities or security issues.^[2]
• Formal verification involves using mathematical or logical methods to prove that the software behaves as intended and does not contain any defects or vulnerabilities. This includes techniques such as model checking, theorem proving, and static analysis. ^[1]
• Static analysis involves analyzing the software code without executing it, to identify potential defects or vulnerabilities. This includes techniques such as code analysis tools and code inspections.^[2]

Tools

Software testing and verification tools are used to identify and address defects and vulnerabilities in software code. There are several types of testing and verification tools, including:

• Artificial intelligence (AI) is increasingly being used in software assurance to identify and address defects and vulnerabilities in software code. AI techniques such as machine learning and natural language processing can be used to automate testing and verification processes, making them faster and more efficient.
• Static analysis tools: These tools analyze the source code of a software application without executing it, to identify potential defects or vulnerabilities. Static analysis tools can be used to detect issues related to code quality, security, and compliance with coding standards.
• Dynamic analysis tools: These tools analyze the behavior of a software application while it is running, to identify defects or vulnerabilities that may not be apparent in the source code. Dynamic analysis tools can be used to detect issues related to performance, memory usage, and security.^[25]
• Fuzz testing tools: These tools generate random input to a software application, to test its resilience to unexpected or malformed data. Fuzz testing tools can be used to detect issues related to input validation, memory management, and security.^[25]
• Penetration testing tools: These tools simulate attacks on a software application, to identify vulnerabilities that could be exploited by attackers. Penetration testing tools can be used to detect issues related to security, such as SQL injection, cross-site scripting, and buffer overflows.
• Test management tools: These tools are used to manage the software testing process, including test case creation, execution, and reporting. Test management tools can help to ensure that all required tests are conducted, and that defects are tracked and resolved.^[24]

Alternate definitions

United States Department of Homeland Security (DHS)

According to the DHS, software assurance addresses:

• Trustworthiness - no exploitable vulnerabilities exist, either maliciously or unintentionally inserted;
• Predictable Execution - justifiable confidence that software, when executed, functions as intended;
• Conformance - planned and systematic set of multi-disciplinary activities that ensure software processes and products conform to requirements, standards/ procedures.

Contributing SwA disciplines, articulated in bodies of knowledge and core competencies: software engineering, systems engineering, information systems security engineering, information assurance, test and evaluation, safety, security, project management, and software acquisition.

Software assurance is a strategic initiative of the US Department of Homeland Security (DHS) to promote integrity, security, and reliability in software. The SwA Program is based upon the National Strategy to Secure Cyberspace - Action/Recommendation 2-14:

“DHS will facilitate a national public-private effort to promulgate best practices and methodologies that promote integrity, security, and reliability in software code development, including processes and procedures that diminish the possibilities of erroneous code, malicious code, or trap doors that could be introduced during development.” There are open-source software tools for software assurance that help identify potential security vulnerabilities.

United States Department of Defense (DoD)

For the DoD, SwA is defined as "the level of confidence that software functions only as intended and is free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software, throughout the life cycle. DoD is developing SwA as a sound systems engineering practice as demonstrated by two recent publications funded by JFAC with development led by the Software Engineering Institute (SEI) and expert practitioners within the Military Services and NSA. The Program Manager's SwA Guidebook shows how SwA should be planned, resourced, and managed while the Developer's SwA Guidebook recommends tailorable technical practices throughout the life cycle. Both of these documents are the first of their kind, and awarded. The two enterprise-scale organizations in DoD building SwA capability are the Joint Federated Assurance Center (JFAC) and the DoD SwA Community of practice which has operated as a quarterly collegial forum 32 consecutive gatherings. Both are open to other parts of the US Government. The JFAC Charter is available at its website. To develop wider situational awareness of the families of SwA tools commercially available, JFAC funded the Institute for Defense Analysis (IDA) to produce the State of the Art Resource (SOAR). A recent innovation in "engineering-in" SwA throughout the life cycle is coupling selected NIST 800-53 controls to engineering tasks so that the engineering results define the Risk Management Framework (RMF) and drive the Authority to Operate (ATO). A package including Data Item Descriptions (DIDs), machine-readable vulnerability report formats, and a brief overviewing application of the techniques is available at the JFAC website. Other disruptive innovations are in process.

Software Assurance Metrics and Tool Evaluation (SAMATE) project

According to the NIST SAMATE project, software assurance is "the planned and systematic set of activities that ensures that software processes and products conform to requirements, standards, and procedures to help achieve:

• Trustworthiness - No exploitable vulnerabilities exist, either of malicious or unintentional origin, and
• Predictable Execution - Justifiable confidence that software, when executed, functions as intended."

National Aeronautics and Space Administration (NASA)

According to NASA, software assurance is a "planned and systematic set of activities that ensures that software processes and products conform to requirements, standards, and procedures. It includes the disciplines of quality assurance, quality engineering, verification and validation, nonconformance reporting and corrective action, safety assurance, and security assurance and their application during a software life cycle." The NASA Software Assurance Standard also states: "The application of these disciplines during a software development life cycle is called software assurance."

Object Management Group (OMG)

According to the OMG, software assurance is “justifiable trustworthiness in meeting established business and security objectives.”

OMG's SwA Special Interest Group (SIG), works with Platform and Domain Task Forces and other software industry entities and groups external to the OMG, to coordinate the establishment of a common framework for analysis and exchange of information related to software trustworthiness by facilitating the development of a specification for a Software Assurance Framework that will:

• Establish a common framework of software properties that can be used to represent any/all classes of software so software suppliers and acquirers can represent their claims and arguments(respectively), along with the corresponding evidence, employing automated tools (to address scale)
• Verify that products have sufficiently satisfied these characteristics in advance of product acquisition, so that system engineers/integrators can use these products to build (compose) larger assured systems with them
• Enable industry to improve visibility into the current status of software assurance during development of its software
• Enable industry to develop automated tools that support the common framework.

Software Assurance Forum for Excellence in Code (SAFECode)

According to SAFECode, software assurance is “confidence that software, hardware and services are free from intentional and unintentional vulnerabilities and that the software functions as intended.”