Maturity Framework

The Maturity Framework

APHL’s Global health team created a rating tool to evaluate global laboratory vendors and eTools. The rating is based on the concept of a maturity model that assesses the capabilities of laboratory eTools based on listed criteria. This rating tool was sent to participating vendors to do a self-assessment.  The results of the assessment were compiled and ranked according to the vendor’s responses. The rating tool ranks tools on a 5-point scale.

It is important to note that providers of tools do not receive the maturity framework and therefore do not score themselves.  Instead, they receive a questionnaire and, based on the responses provided, the tools receive a maturity score.  This enables many more tools and providers to be included than would otherwise have been possible due to the large number.

Domains

There are 10 domains that comprise the maturity framework.

 

Domain 1.

System Infrastructure

This domain examines the technology the application is built upon. This includes everything from the operating systems at the user level, the server operating systems, server software for web servers, applications server architecture, database servers, development tools, deployment tools and any technology required directly or peripherally to the software. The ability for local staff to support and maintain and generally work with the software components is also important as well as the ability to find human resources locally who are familiar with the components. The ability for local staff to work with and support the technology is also an importan consideration.

Domain 2.

Utilization

This domain examines the breadth of the user base and utilization in the target countries. Some tools are utilized heavily in a low number of settings and other might many countries utilizing the tool but one installation per country. Some tools may only need small number of installations in each country if the tool is used at a central level by the Ministry of Health. Another tool might have dozens of installations in each country.

Domain 3.

Country eHealth Strategy

Tools may be recognized as part of the country's great health needs and overall health ecosystem, guided by a strategy that integrates leadership, financial, organization, human, and technological resources. The tool may be integrated into the country's action plan and budgets with coordination among multiple stakeholders.

Domain 4.

Roadmap

Roadmap provides detail understanding of intent and direction in which the company/tool provider plans to go, serving as a strategic communication tool. Due to evolving technologies and user expectations, determining future needs is challenging.

Domain 5.

Technical Support & Language

The day-to-day users who interact with the tool will have questions and encounter problems. There are different models to support those users. One convenient method is to have a small number of staff who receive extra training and become "super users". The regular staff should have easy access to these super users to ask questions and resolve issues. This is a convenient model because the super users are typically staff at the same location or nearby, speak the same language and are familiar with the specifics of the installation. Another method for supporting the end users is a help-line they can call which is staffed by the tool provider or other the implementing organization. For this model consideration should be made for the languages spoken, the ease with which someone can be contacted, how easy they are to communicate with and any language issues. Some tools are easier than others to support remotely. Other considerations are the ability to support the tool by remote login or even the availability and ease/cost of on-site visits for troubleshooting more advanced issues. It is also more common to have online forums or knowledgebases that can be searched for information about the tool.

Domain 6.

Documentation – Technical and End User

The ability to make highly technical changes to the system needs to have some level of documentation. In the case of configurability, this documentation is extremely valuable. Where source code is accessible the documentation focuses less on specifics of the underlying tools and languages and focuses more on how to setup the development environment, specific versions of tools and libraries, descriptions of where specific types of changes should be made and how to create the final product utilizing all those tools. Strong end user documentation can support use, long term adoption, maintenance and support for the tool.

Domain 7.

Implementation and Training

Tools very greatly in the level of effort required to get the tool installed and integrated into the workflow of the laboratory. Some tools will be able to be installed remotely by the provider or require very little site specific configuration or training. Some systems may be so critical to the the laboratory workflow they require detailed requirements gathering and gap analysis to determine small changes to make sure they conform to the local laboratory's workflow as well as also take time and expertise to make sure all of those requirements are captured. In general, the more dependent the staff are on the tool as part of their workflow, a similar level of configuration and time is likely required for implementation.

Domain 8.

Interoperability and Data Accessibility

Standards apply both from a technical standpoint and the health industry standpoint. Tools which utilize standards can be more efficient for people to use because it allows for a common terminology. Data standards allow systems to communicate more easily because each system uses the same values. Standardized data exchange formats and mechanisms help lower the time it takes to make systems communicate. Some tools will benefit more than others by adopting the industry standards. Tool providers need to stay on top of the industry changes from many different aspects and doing that means putting processes in place to monitor changes and adopt standards and changes.

Domain 9.

Security

Data security is related to protecting digital information from unauthorized access, and measures that ensure confidentiality, integrity and availability of the data.

Domain 10.

Installation Scalability

This domain refers to the ability of the tool to scale from a small to a large number of users, from low-level infrastructure to well-equipped laboratories and from peripheral levels of the laboratory to the national laboratory and Ministry of Health. Scalability may be viewed as:

 a) Vertical b) Horizontal c) Geographic and d) Software performance.

Scale

The maturity of each tool is rated on a 5-point scale, similar to the Measure Evaluations Maturity Model.  While the laboratory tools maturity model is not limited to determining the maturity of interoperability, the 5 levels have been applied across all domains except cost.

 

Level 1 Nascent Does not follow a systematic process, isolated ad hoc efforts
Level 2 Emerging Defined structures but not systematically documented. No formal performance monitoring
Level 3 Established Documented structures, structures are functional, metrics for performance monitoring are used systematically
Level 4 Institutionalized Stakeholders use the product and follow standard practices
Level 5 Optimized Developers/system providers regularly review needs and modify functionality to adapt to changing conditions