NESTcc’s 20 Test-Cases explore how medical device ecosystem stakeholders can use data from real-world settings to generate high-quality evidence for device evaluation.

The premise of the work is straightforward. NESTcc connects organizations that have research questions with organizations that can answer them, and supports research execution processes from contracting to project management and the dissemination of results. In addition to testing study feasibility across NESTcc’s Network Collaborators, the ultimate aim of the Test-Cases is to help ensure the availability of safe, effective, and innovative technologies for patients.

Test-Case concepts span organizations across the ecosystem, including health systems, government organizations, non-profit patient organizations, and medical device manufacturers. They include projects across nine disease areas, along the premarket approval and postmarket regulatory pathways, and throughout the medical device Total Product Life Cycle (TPLC), using multiple data sources including electronic health records (EHR), registries, claims and patient-generated data (PGD).

From announcing the first slate of Test-Cases in November 2018 to preparing for the results of many of those projects this year, NESTcc has learned a great deal about the conduct and management of real-world evidence (RWE) studies. We have also identified opportunities to use these learnings for improved efficiency and value across Test-Cases and future research projects conducted through NESTcc.

Early Learnings

Some of the most important lessons learned have been gleaned from a review of collaborative initiatives in the health care space, as well as from the inititial stage of executing Test-Case contracts.

• Standardizing agreements. Delays in contracting cause delays in research, highlighting the importance of establishing clear legal guidelines early on. The development of standardized contracts and sub-agreements has been effective in mitigating delays, and contracts now in place with NESTcc Network Collaborators will serve to streamline future projects.
• Reducing legal complexities. With a number of Network Collaborators participating in more than one study, legal review can be complicated. It is necessary to identify any conflicts early in the contracting process, as well as connect programmatic and legal teams to ensure alignment.
• Finding additional time-saving contracting opportunities. Existing relationships between Network Collaborators and registries can help circumvent the need to set up new data use agreements (DUAs) for efficient sharing of de-identified data across multiple institutions.

Later Learnings

In executing the Test-Case projects, we have uncovered important best practices with our Network Collaborators in the areas of protocol development, the collection of quality data, and data sharing.

• Matching needs to capabilities. When a study sponsor submits a research concept to NESTcc, the optimal Data Network capabilities can be identified more efficiently when device data can be provided early. For example, if a manufacturer has access to sales data or unique device identifiers (UDIs), Network Collaborators can use that data to determine whether they are best suited to answer the question based on their particular patient population, geography, volume of device use and relevant procedures, and other factors over a time period of interest.
• Sharpening the focus. Sharing information up front also helps set the right course for a project between a study sponsor and Network Collaborators. For a comparative analysis study, the identification of devices that are reasonable alternatives to the device of interest is key. For label expansion, the definition of data elements that support a regulatory submission aligns preliminary work with the data collected. From there, forming project workgroups helps ensure alignment on project goals and methodological approaches.
• Protocol pre-specification. As noted in the NESTcc Methods Framework, the pre-specification of a study protocol is key for the rigorous design and execution of research and to support regulatory purposes in the future. This concept holds true not just for the quality of a study, but also for its timeliness.
• Centralizing IRB review. Test-Case protocol edits were found to cause delays in contract agreements and Institutional Review Board (IRB) approval. Review and approval time could be reduced through the creation or use of a centralized IRB review process, which NESTcc plans to explore this year.
• Creating and refining templates for repeatable processes. In developing protocols, the use of NESTcc templates has been highly beneficial in standardizing projects through the execution phase. For Network Collaborators, the creation and validation of data extraction algorithms in the Test-Case process has enhanced their efficiency for future projects with NESTcc.
• Collaborating to identify the best data. UDIs are valuable for device research and essential for the viability of certain studies. Network Collaborators with UDI implemented into workstreams can save considerable time and effort in identifying populations accurately and ensuring data quality. In the absence of UDI documentation, Network Collaborators can work together to create and validate datasets through the review of device model numbers, barcode scanning system data, inventory management system data, scheduling logs, encounter information, patient charts, transaction tables, charge codes for device billing, and other sources.
• Finding common ground. The identification of commonalities can help improve data quality and methods across projects. The use of a common EHR system across Network Collaborator sites can offer similar (though not identical) databases and data flows at each location. Moreover, common data models, dictionaries, and ontologies can be used to develop acceptable value sets and standardize the representation of certain data elements.

As we continue to advance our Test-Case projects, our learnings and value opportunities will likewise continue to grow. NESTcc anticipates sharing final results of its first completed Test-Cases in the months ahead, and we look forward to continuing this important work with Test-Case partners to streamline processes and ensure optimal quality data for the purposes of answering important questions across the medical device ecosystem.

Robbert Zusterzeel, MD, PhD, MPH
Data Network Director, NESTcc