In my blog Keeping Pace with Human Spaceflight: An Agile Approach to Independent Verification and Validation, I explored how Agile independent verification and validation (IV&V) emerged as a means to keep pace with the development of complex software for NASA’s Orion spacecraft. By applying concepts from Agile software development to the realm of evaluation, we are seeing a lot of success in quickly assessing whether things do what they are supposed to do and don’t do what they are not supposed to do.
Agile IV&V has been received very positively, earning praise from the NASA IV&V facility's director and the Orion software's architect. We have assisted NASA in developing briefs for applying this concept to other IV&V work.
In the best traditions of scientific collaboration, we offer some summary bullets on our findings:
1. Keep the main thing the main thing. Assurance is our primary product, in addition to issues and risks. With complexity comes distraction. Keep focused.
2. Capability-based assurance focuses on software in the context of mission capabilities, which requires system understanding captured in the IV&V Technical Reference while continuing to apply the IV&V “three questions”:
- Does the system and its software do what it is supposed to do?
- Does the system and its software not do what it is not supposed to do?
- Does the system and its software behave appropriately under adverse conditions?
3. Risk-targeted IV&V, also known as “Follow the Risk,” considers both functional and non-functional risk categories across the entire Orion risk landscape and addresses both mitigated risk and residual risk.
4. Adaptive IV&V incorporates critical thinking about what makes sense when performing analysis and what is the right thing to do. This includes leveraging existing assets such as the IV&V Technical Framework and the IV&V Catalog of Methods, but they do not constrain it.
5. Integration assurance integrates appropriate behaviors—especially emergent behaviors—and behaviors involving interfaces within and between software and systems.