Data preparation for business & platform teams
Visual designer is key to make every step - development, interactive execution, deployment, observability and fixing errors - accessible to the widest number of users.
AI is integrated deeply into the visual designer and recommends next operations, translates text into business logic, fixes bugs and explains business logic in text.
Compilers turn visual design, tests and text generated by AI to native Spark or SQL code. They ensure code correctness and performance. Extensibility relies on compiler magic to bring new code based plugins into the visual and AI layers.
Prophecy provides a simplified visual interface to commit changes to git, with collaboration operations such as merge done in the visual designer.
Automatic test generation increases test coverage and ensures that changes are high quality when committed and when pushed to production.
Multiple data users will build, enhance and fix data pipelines over time. AI generates documentation of the pipelines and summary of each change made by a user for higher quality and productivity.
Built by data engineers
Plugins enable data engineers to code standards that extend out of the box capabilities. New operations connect to your custom internal systems, standardize ops such as encryption, and make common business logic reusable across transformation jobs.
Use software best practices
Plugins in Prophecy are developed with software best practices. They are projects of standard Spark or SQL code that are deployed as packages with dependency management.
