Transforming operations to support healthcare of the future

A Fortune 50 global leader in health services depends on data and analytics to gain the insights necessary to deliver highly impactful healthcare outcomes. When they realized their existing on-premises, legacy ETL and data engineering infrastructure was no longer able to support the future needs of their customers they looked to the scalability and flexibility of the cloud. But with over 2,000 data pipelines to migrate off their legacy system, the engineering effort needed was massive and was going to severely stretch the capabilities of their data team.

The company receives data from a wide variety of sources that help to drive their business decision making including their own enrollment, billing, and policy management systems, as well as pharmacy, and even patient generated data. All of this must comply with different privacy regulations such as HIPAA and GDPR. The inefficiency of their legacy ETL tools were exposed when data engineering needed to quickly integrate and analyze this diverse data while also maintaining compliance, often taking an entire day to generate insights.

The team were able to use Prophecy’s high-performance, low-code, cloud native data engineering platform to not only support the successful move of their 2,000 data pipelines to their Databricks Lakehouse environment on Azure cloud, but it also gave a broad cross section of their data team a visual environment for building high-quality data pipelines without having to understand complex data engineering concepts or how to code with Spark. This allowed the data team to improve the performance of some of their most important data transformation jobs from almost 2 hours to 10 minutes.

Thanks to Prophecy’s ability to work seamlessly across their existing on-premises legacy data warehouses as well as their new data architecture rooted in Databricks Lakehouse, their migration has been highly successful. The concerns about ensuring data privacy have also been alleviated as Prophecy was able to integrate with their enterprise access provisioning system, maintaining their highly sensitive data compliance requirements. This allowed the entire data team to stay productive throughout the transition.