It’s that time of the year again! Holiday trimmings are out. The year-end retrospectives are going up. And data folks around the world are snuggling up to their data lakes, with visions of ubiquitous, well-governed data to power the next generation of analytics and AI dancing in their heads.

But what’s that we’re hearing? There’s trouble on the way to the data wonderland? Not everyone can access the clean, trusted, and timely data they need? 

How can this be? 

You’ve made your data cloud adoption lists and checked them twice. Your central data teams have been good, adopting AI and tackling new data types. And yet the backlog keeps growing. And valuable pipelines are still locked up in legacy ETL tools. 

How can your data team avoid getting a lump of coal this year? The Prophecy Data Transformation Copilot is here to help!

Back and forth no more

One data shop was stuck. It was taking too long to deliver critical supply chain data from Databricks and their most in demand team resources were oversubscribed. 

Things had started out well. Data requirements from subject matter experts were sent over to the data team, and the data team would get to work. But at each check-in, there were new issues, tweaks, and revisions. Soon it was a 26-step process that took more than 3 months to develop a single pipeline!

The team took a page from data wonderland and brought the data team and subject matter experts together on the same Prophecy platform.

The result? The process went down to 9 steps and the team delivered 10x more data to the business in the first 6 months. 

One more data team off to the data wonderland.

‍

Data engineers and data analysts skating together into the data wonderland

Tapping the cloud to deliver critical data

At the Databricks Data and AI Summit in June the theme was data and AI together. To get there, data teams needed to be able to fully leverage their data cloud.

Healthcare analytics provider Aetion showed us just how important this can be. To onboard customers to their new Discover product, they needed to quickly process 200M records. With traditional data engineering, it was taking too long to create the pipelines and the team was limited to only the most highly skilled data engineers. Taking a new approach with Prophecy, they unleashed data cloud scale and speed. The result: reducing data set onboarding by 50% and tapping data cloud capabilities.

Over in the Netherlands, healthcare provider CZ’s move to Databricks included speeding the modernization by tapping experts from around the business. The result: 2000 pipelines migrated.

Two more data shops, gliding into the Data Wonderland!

Delivering data for the holidays with the Delta Lake Lodge

No data pipelines left behind

Meanwhile over at Amgen, where every team works to deliver innovative, life-changing treatments, data from the finance team is a critical resource. But with data pipelines locked in Alteryx, it was hard to scale to meet peak demands and required a lot of manual, duplicate effort to keep all the data in order.

The team made their move to Databricks to tap the data cloud speed and scale. But recreating the pipelines was proving trickier and taking more time than they anticipated.

Getting into the data wonderland spirit on a recent webinar, the team shared how they adopted Prophecy to speed pipeline creation to bring all their pipelines over to Databricks. In addition, to get through the backlog of complex pipelines, they leveraged Prophecy Migration Copilot to automate parts of the migration. 

The result? 20% faster data processing, capacity to run critical KPIs twice as often. 

No pipelines get left behind in the data wonderland!

Turbocharging pipelines for the holidays!

Delivering some pre-holiday cheer, the Prophecy product team capped the year off with a review of some of the AI innovations behind the data wonderland magic. 

On our webinar “New features to turbocharge pipeline development,” the team previewed ways to further speed pipeline development, tools to enhance observability, and how Copilot AI is transforming how we handle data.

Happy Holidays

The best gift of this holiday season: helping data teams deliver the clean, trusted, and timely data that every data user needs.

Happy holidays everyone! We hope that you and your data teams can glide along with us  into the data wonderland!