WHAT TO READ NEXT
What is machine learning & AI?
Technology Readiness Levels (TRL) provide the basic framework for incorporating new, innovative technology into an existing system. It should not be applied in a vacuum.
During the 1960s, NASA developed a novel framework for what is now called the Technology Readiness Levels (TRL) that would benchmark all technologies used for space applications against a standardized scale understood by all across the space program. In addition to providing a standardized measure, the TRL was intended to provide an assessment of the maturity level of a given technology. The need for the TRL was born from a desire to incorporate the latest technologies into space missions, while minimizing the potential risk that introducing new technology would bring to said missions.
Please note that this singular framework is one component within the larger 486-page “NASA Space Flight Program and Project Management Handbook” that outlines the considerations for program and project life cycle management at NASA. The purpose of this article is not to restate project or program management principles for high-technology industries. But rather to shed light on the more commonly understood technology development principles and their application within the context of the ongoing discussions regarding innovation in today’s technology landscape.
The nine-point scale starts with an idea and progresses from experimentation, environmental testing through to successful mission operations.
The TRL is used in communications regarding programs and projects with decision makers and engineers to provide the same frame of reference for all. The TRL provides information regarding relative maturity of multiple technologies, progress and status. It is used as a selection criteria for innovative technologies. However, because the TRL is a component (not system-wide) assessment, it is limited in that it:
- Does not provide information on integration risks or maturity of components within the system
- Does not provide insights into the impacts of alternative TRLs on the system as a whole
Complementing the TRL is a Technology Maturation Plan (TMP), which outlines the activities required to advance immature technologies to the desired TRL.
Adjacent Concepts Borne from the TRL
Given similar risk profiles for introducing new, untested technologies into existing systems within the defense, aerospace, energy and biopharma industries, it comes as no surprise that the TRL has been widely adopted across these sectors. However, through this adoption, it has undergone examination for its application and completeness with the most important acknowledgement that the TRL does not and must not operate in a vacuum. Consequently, over the last fifty years, many new models and concepts adjacent to the TRL have been introduced. Some of the more common principles surrounding the TRL include:
- Valley of Death
- Technology Readiness Assessment
- System Readiness Levels
- Integration Readiness Levels
- Manufacturing Readiness Levels
Valley of Death
The “Valley of Death” is perhaps the most commonly referred to concept discussed in and around the TRL. It refers to the failure to progress technology from TRL 4 to TRL7 due to the costs associated with testing new technology on the ground or in space. This concept refers explicitly to the technology development process.
Analogous to the inability to progress technology along the development roadmap is the inability to ramp up the commercialization growth curve or to cross over the chasm of the technology adoption curve first identified by Geoffrey Moore in the 1990s. However, it is important to note that while the Valley of Death is well understood in the context of technology development, similar concepts are well-established in discussions regarding the inability to commercialize technology or to successfully accomplish wide-spread adoption and those rather than the “valley of death” should be used in the commercial sense.
Technology Readiness Assessment (TRA)
Taken from the Department of Energy, the Technology Readiness Assessment (TRA) provides a “snapshot in time of the maturity of technologies and their readiness for insertion into the project design and execution schedule” across all the development programs in the pipeline. Because the TRL only provides an understanding of the maturity of a technology – not the likelihood of success or failure, the TRA is useful in providing the following missing information:
- Identify gaps in testing along with information needed to reach readiness level
- Identify at-risk technologies that need increased management
- Increase transparency of potential technologies that might put the project at risk
An example of a TRA assessment that could provide a snapshot in time is shared below:
System Readiness Levels (SRL)
The concept of a System Readiness Level (SRL) was first introduced by Sauser, Verma, Ramirez-Marquez and Grove (2006) and is a mathematical formula resulting from the products of the TRL and IRL matrices. While the idea of a system-wide view is a much-needed framework when considering the TRL, there is not enough research on whether this mathematical formula is an apt representation of the readiness of a system.
Integration Readiness Levels (IRL)
The Integration Readiness Level (IRL), a concept widely discussed by the Stevens Institute of Technology (located in Hoboken, New Jersey) and addresses the shortfalls of the TRL from an integration perspective with existing technologies and systems by providing information on the risk associated with integrating technologies. This seven-point scale provides information on:
- Identify gaps in testing along with information needed to reach readiness level
- Identify at-risk technologies that need increased management
- Increase transparency of potential technologies that might put the project at risk
Manufacturing Readiness Levels (MRL)
The Manufacturing Readiness Levels (MRL) are a measure of risk for manufacturing new technology and is related directly to the TRL. MRLs have ten scales and measure the shortfalls in manufacturing as technology progresses across the TRL and prepares it for alignment with a technology that meets TRL 9. The Department of Defense (DoD) has a deskbook that illustrates the parallel paths that are taken by both the MRL and the TRL frameworks.
Manufacturing is a complex and involved function – with multiple input parameters as well as infrastructure requirements. This MRL discussion is not intended to provide an SOP for setting up operations – rather it is being provided in the context of the TRL as a framework or guiding principle to help streamline the product development and manufacturing processes. The Department of Defense has indicated that the MRL is best used as a visualization of the overall progression of manufacturing readiness rather than a prescriptive linear progression from one MRL to the next. It also does not provide information on potential risks in achieving the next level of maturity, however it is useful in assisting to effectively and efficiently mature manufacturing capabilities.
Additionally, MRL is closely tied to the TRL and moves in parallel as demonstrated by the Figure below.
Analyzing Risk
Given that the TRL is a framework that is used to mitigate risk, it is important to also discuss several metrics that are used in conjunction with the TRL to measure and monitor the impacts of risk. In reviewing the “NASA Guidelines for Risk Management”, there are detailed perspectives on how to treat risk (attribution, classification, acceptance, etc.) – however the four concepts that stand out include:
A. Risk Likelihood Criteria
B. Risk Consequence Criteria
C. Impact Horizon
D. Risk Scoring
Risk Likelihood Criteria
The Risk Likelihood Criteria provides a certainty measure if a risk is likely to occur ranging from “Near Certainty” to “Not Likely”. The table below:
Risk Consequence Criteria
The Risk Consequence Criteria identifies several categories that should be evaluated for risk and scored on a scale (eg. 1 to 5). The risks identified by NASA are listed below in Table 2. The category with the highest score is the score that is used as the final consequence score.
Impact Horizon
The Impact Horizon provides a time frame for when the risk may occur and includes both a “Sunset” and “Sunrise”. The Sunrise is the earliest time the risk could become an issue whereas the Sunset is the latest. The impact horizon helps to further prioritize risk based on the time frame in which they are likely to occur.
Risk Matrix / Priority Score
Once each of the above risks have been identified (Likelihood, Consequence, Horizon), a score can be determined by calculating the “Priority Score” and input into a “Risk Matrix”. For example, a risk with a likelihood of 2 and a consequence of 5 would have a 17 Priority Score. This score should then be used for prioritization assessments of various projects and can help inform whether a mitigation plan is required and what (if any) resources would need to be allocated to support such a plan. The purpose of this scoring mechanism is to identify which risks are accepted, controlled, tracked or monitored.
Insight
Bottomline, the TRL remains an inherently useful indicator for assessing innovative, new technology but should never be taken in a vacuum. Also, applying only the nine-point scale instead of a fully validated project & program guideline is a recipe for disaster (reference NASA’s 486-page handbook on its Space Flight Program and Project Management guidelines). The complexity of technology development requires a complex approach to managing innovation and risk. Consequently, we recommend that organizations that blithely reference the TRL as their development program and fail to take into consideration the other factors and frameworks that help to inform the innovation roadmap, should approach this article as the “tip of the iceberg”.
Finally, one word of caution. The TRL should be used as a framework for innovation and not as sometimes occurs when the “valley of death” is used to refer to failure to ramp or “crossing the chasm”. There are commercial frameworks that should be applied to the commercial processes just as there are developmental frameworks that are applied to the development process. To use one in the context of the other without acknowledging the other exists creates unnecessary confusion.
FEATURED INSIGHTS
Risk assessment & mitigation planning for advanced therapies
CLIA, CMS & the FDA
Market positioning & launches
Connect with our experts today