Systems Readiness: Are We Really Ready?

In complex systems, integration and interfaces are mostly where things go wrong

10-20-2020
Don York
SYSTEMS ENGINEERING

When I started college, a few short decades ago, I commuted to Stevens Institute of Technology in Hoboken, N.J., just outside of New York City. The first time I took my father’s car for a solo journey to school, he asked, “Are you ready to drive the car alone? Do you think you’re mature enough to take the car into Hoboken?”

“Yeah, Dad, I’ll be OK. I’m ready,” I replied. My dad was an expert mechanic. Both he and I knew that the car was ready, and according to my driver’s license, the state of New Jersey felt I was ready too.

Later that morning, as I crossed “Old Ironbound”--the Pulaski Skyway--toward the Holland Tunnel, the driver of an oncoming southbound vehicle lost control. The driver jumped the low-profile center divider, striking my vehicle and three others traveling northbound.

Expect the unexpected

Let’s view my unfortunate incident from a systems standpoint. Although I was ready to drive my dad’s car to school, the system that was required to get me there was more complex than just my vehicle and I, the driver. The system boundary was bigger, which meant that there were more interfaces and integration scenarios to consider, such as another driver losing control.

Much more recently, my government customer asked if his critical system was ready to deploy. Yes, we conducted a technology readiness assessment (TRA); yes, we identified the critical technology elements (CTEs) of the system; yes, we performed performance reviews on the CTEs. The technology had been proven to work in its final form under expected conditions. However, I remembered the unforeseen car accident of my youth. I remembered the complex interfaces.

Integration and interfaces are where most things go wrong. Failures happen at the places where things come together. We had identified two CTEs. One had been used multiple times on other platforms but never in the customer’s type of system and not with the large number of interfaces. There were too many potential failure points.

System readiness assessment: a quantitative score

I’d been working with colleagues to develop and advance system readiness assessment (SRA) as a new systems engineering methodology to serve as a key element in transforming the government’s approach to complex systems integration. As a part of this methodology, new system metrics assess the readiness of systems for operation and help our customers to manage risk and reduce the total cost of ownership in an increasingly complex environment.

I convinced my customer to wait a few weeks. Using SAIC’s SRA User Environment, we assessed and evaluated not only the readiness of each technology and component but also each interface in the system. We scored the system with a readiness metric and determined how ready we actually were—not just the two CTEs but the entire system and all of its integrations. Using the Bayesian network modeling component of the environment, we actually arrived at a level of confidence in our decision.

My customer welcomed the surety, saying, “There’s too much at stake here to risk an integration failure in the field, especially in front of our partners. We need that confidence!”

Unlike my car scenario, our system deployed with foreknowledge of performance under expected and unexpected conditions. I told him, “We’ll be OK,” and I had the data to prove it.

Recent publications related to SRA:

  • York, et al., “Using Bayesian Networks to Validate Technology Readiness Assessments of Systems,” Conference on Systems Engineering Research (CSER), Redondo Beach, CA, March 23 – 25 2017.
  • Austin, York, “System Readiness Assessment (SRA), A Vade Mecum,” Complex Systems Design and Management (CSD&M) Conference, Paris, France, Nov 2015.
  • York, et al., “Applying Bayesian Networks to TRL Assessments – Innovation in Systems Engineering,” INCOSE International Symposium, Adelaide, Australia, July 2017.
  • “NSA Systems Readiness Assessment (SRA) Engineering Handbook,” Version 1.0, 27 June 2014.
  • “SRA, System Readiness Assessment, Planning for Success,” Trifold Brochure, Feb 2017.
  • Austin, Homberger, York, “System Readiness – A Look Beyond TRAs,” NDIA, October 2017.
  • Austin, York, “Bucketology and an Enterprise Approach to Systems Development: Meeting Your Strategic Objectives,” July 2016.
  • Austin, M.F., York, D.M., “System Readiness Assessment (SRA) – An Illustrative Example,” 13th Annual Conference on Systems Engineering Research (CSER), Hoboken, N.J., 18 – 20 March 2015.
  • Austin, York, et al., “A Systems Approach to the Transition of Emergent Technologies into Operational Systems – Herding the Cats, the Road to Euphoria and Planning for Success,” INCOSE International Symposium, Utrecht, Netherlands, 2008.

FURTHER READING: More from our systems engineering expert Don York.

Bayesian Networks Bring Order to Complexity

Systems Engineering Wisdom: Advice Versus Opinion

Posted by: Don York

Senior Systems Engineer

Don York is a Senior Systems Engineer at SAIC.

Dr. York is a SAIC Fellow with more than 40 years of experience in the design, development, engineering and productization of advanced technology systems.

He is recognized as the go-to professional for systems engineering, applying systems engineering expertise across various domains.

As a principal technical expert for system development metrics, Dr. York is part of a team that helped to develop the Government Accountability Office’s Technology Readiness Assessment Guide, published in 2019.

Dr. York has made contributions to the field of Systems Engineering in education, the profession, and the community. He has taught Systems Engineering at the graduate level and has co-authored and presented numerous technical papers.

He holds his Bachelors and Masters in Mechanical Engineering, a PhD in Systems Engineering and is a Certified Systems Engineering Professional.

Dr. York has been a member of the International Council on Systems Engineering (INCOSE) for over 25 years where he currently serves as the Chairman of INCOSE’s Corporate Advisory Board.

Read other blog posts from Don York >

< Return to Blogs