How Data Engineering Helps Health Systems Reach Their Most Critical Goals

Data engineering in healthcare is constantly evolving to create solutions for the industry’s most pressing issues: ensuring the safety of frontline workers while maintaining adequate staffing levels during the pandemic; moving toward value-based care models; health equity; and revenue cycle management. In each case, data engineering can play a significant role in developing solutions by integrating data from electronic health records, payer claims and tools that present a comprehensive view of patient information. The following use cases illustrate how data engineering applications are transforming healthcare.

Data Engineering Supports Population Health Strategies

Creating an effective population health strategy requires an organization to take a data-driven approach and study both internal and external data, especially from other providers and payers outside the organization that are ready to share data through clinically integrated network (CIN) contracts. However, integrating external data from multiple sources presents several challenges related to data quality, data feed format, cadence, standard codes, onboarding data from a new payer into the organization’s enterprise data, and more.

Our health system’s team developed data engineering solutions to overcome these challenges, including:

Data quality services that helped to standardize the data and bring payer data into a uniform structure Data integration mappings developed to feed the payer data into the standard structure and inject them into the enterprise data warehouse Quality checks on each layer of data driven through rule sets configured in the database for each payer’s data

This enterprise data was later used to analyze and bundle various clinical procedures to reduce the cost to patients without affecting the quality of service.

The data engineering solution enabled our health system to achieve strategic goals for improving population health. Armed with data, hospital leaders were able to analyze and compare different providers’ costs for procedures for specific diagnoses. Their analysis allowed for certain costs to be reduced, making healthcare more affordable for the population. In some cases, it also generated rebates to providers who were part of the CIN. 

RELATED: What are digital twins and how can they be used in healthcare?

The solution also made it possible to process payer data rapidly and enabled the parallel processing of multiple sets of payer data. The uniform structure that was created to store all payer data helped to onboard a new payer faster and made it easier to merge that new payer data with existing enterprise data to make it available for dashboards and analytics solutions. Furthermore, the quality checks identified issues well ahead of any potential problems, expediting the injection of quality data into the analytics system. With this design, onboarding any new payer data into the population health system can be achieved in two weeks.

Another benefit of the population health solution is how it builds and feeds critical data from other programs into our internal analytics system. With the ability to integrate this information and create analytics internally, we eliminated the need for external vendors and reduced data costs to the organization.

Data Engineering Was Key for Staff Management During the Pandemic

One of the many challenges hospitals faced during the pandemic was keeping clinicians safe as they worked on the front lines to care for patients. As soon as the first COVID-19 vaccine became available, our organization found that it needed a quick and efficient way to organize and prioritize clinical staff and to develop a system to administer the vaccine to those who were most responsible for patient care.

Data integration was used to assist in providing the vaccine by integrating cloud and on-premises systems with external systems. Using Informatica Cloud tool, we built extract, transform and load (ETL) mapping to connect to the Workday human resources program. This allowed us to obtain and prioritize a list of eligible clinicians and critical administrative staff to build a data repository. 

ETL also checked the designated personnel files for vaccine consent forms and scheduled vaccine administration in accordance with the prioritization. The ETL was quickly designed with a change data capture feature, which checked the system for new consent forms and added those personnel into the scheduling list. Once they were approved by the system, these individuals were automatically notified about their vaccination schedule through email and text messages using the command tasks built in the Informatica Cloud system.

Additionally, as illustrated in Figures 1 and 2, a data integration service can further integrate with the absentee system to keep track of staff who had tested positive for COVID-19, feeding their quarantined days into a reporting layer that populated vaccine data dashboards. This functionality helped management monitor our clinicians’ health and make quick decisions on workforce availability so they could recruit frontline workers as needed.