Unlocking Databricks for Data Analysts: Simplify Data Preparation with Prophecy

Prophecy and Databricks are redefining how teams build and manage data pipelines on Databricks SQL. Built on a lakehouse architecture, the Databricks Data Intelligence Platform provides the best foundation for data processing and governance. Databricks continues to expand its platform, making powerful data capabilities accessible to a broader range of users, including data analysts who rely on Databricks for BI and reporting. To maximize the value of Databricks, these users must also be able to prepare and transform data for analytics—quickly and efficiently. That’s where Prophecy comes in. Prophecy empowers analysts to prepare data seamlessly on Databricks while ensuring governance and best practices are followed. 

‍

With an intuitive, drag-and-drop interface Prophecy Data Integration Platform enables analysts to build pipelines efficiently while ensuring consistency and compliance. Deep integration with Databricks SQL means organizations can fully leverage its processing power while putting trusted data into the hands of the business faster.

Prophecy and Databricks drive a paradigm shift in data analytics

Data analysts play a crucial role in shaping the data products that drive business decisions. Data products are curated, reusable datasets or pipelines designed to deliver clean, trusted data for AI, analytics, and operational decision-making. For example, a data product might be a continuously updated customer segmentation dataset used by marketing and sales teams to tailor campaigns and optimize outreach. To create these data products, analysts spend significant time cleaning, transforming, and preparing raw data—ensuring it is structured, complete, and ready for downstream consumption. However, without the right tools, their ability to contribute effectively is often limited.

‍

The faster an organization can build and iterate on data products, the quicker it can make data-driven decisions, deploy AI models, and gain competitive advantages. Speeding up data product development means business teams can react to market trends in real time, optimize operations, and unlock new revenue opportunities. On the flip side, delays in building data products can mean missed opportunities, slower AI adoption, and decision-making based on outdated or incomplete data. Bottlenecks in data transformation and pipeline development hold organizations back from realizing the full value of their data.

‍

One of the biggest challenges in accelerating data product development is that data analysts often rely on data engineers for coding transformations in Scala or Python, which can introduce delays as requirements evolve, edge cases arise, and data nuances come to light. Additionally, many data analysts work in legacy tools that are neither cloud-native nor fully integrated with Databricks. This creates silos where central IT teams can’t share or enforce standards. Without a unified environment, data analysts–and all data teams–end up preparing data inconsistently, leading to quality and compliance issues.

‍

Prophecy removes these obstacles by empowering data analysts to independently build production-ready data pipelines in Databricks SQL while adhering to standards set by central IT. With Prophecy’s visual, drag-and-drop interface and seamless integration with Databricks, analysts can prepare data with greater autonomy and efficiency. Prophecy’s built-in features and integration with Databricks Unity Catalog enhance governance by ensuring secure, appropriate data access while maintaining enterprise-wide data management standards.

‍

For data analysts, this means they can finally clean, prepare, and transform data directly in Databricks. This accelerates the development of high-quality data products, reduces the back-and-forth between teams, and ensures data consistency across the organization. For the business, this translates into faster insights, more reliable AI models, and the ability to leverage data for strategic decision-making without unnecessary delays.

‍

Analysts can easily get started with Prophecy. Using Prophecy via Databricks Partner Connect with OAuth and SQL warehouse integration, access is both secure and effortless—data analysts gain immediate access to the data they are already entitled to in Databricks. This means they can start building and deploying data products on day one, helping the organization unlock the full potential of its data.

‍

Step 1: Navigate to Marketplace > Partner Connect Integration and select Prophecy

Step 2: Click on Connect on the upper right corner

Step 3: Select the SQL warehouse that you want to utilize when using Prophecy and choose the Catalog and Schema that your SQL warehouse has access to. If you do not select a Schema, it will be set to Default. Click on Next > Next > Connect to Prophecy. 

Note: The Catalog and Schema are the default settings on Prophecy. You can make changes on Prophecy if needed. If you would like to make changes to your connection in the future, delete the existing connection and create a new one. 

‍

‍

Step 4: Enter your information and password to create a new Prophecy account. If you already have a Prophecy account, enter your password to authenticate.

Step 5: Navigate to the HelloWorld Project > orders pipeline  and select the SQL warehouse connection from Partner Connect. Click on Attach Cluster. Voila!! You’re ready to explore the pipeline and transform your data! 

You can use the drag-and-drop UI to build your own transformation.

‍

Get started with Prophecy on Databricks

Get in touch with your Prophecy or Databricks rep and schedule a demo to unlock Databricks for data analysts.