Kestra is an open source data orchestration platform for complex workflows
Summary
Meet Kestra, a startup that has been working on an open source project focused on data orchestration across several services, databases, files, repositories and warehouses. At some point, big companies start to have data spread out across several storage locations. If you are familiar with Apache Airflow, Kestra can be used as an alternative to Airflow with some major differences.Instead of using Python code, Kestra is based on YAML configuration files and if youve played with Docker images at some point, you may already be familiar with YAML.Kestras API is treated as a first-class citizen, meaning that it integrates well with other tools and systems. In other words, Kestra has been designed to be language agnostic thanks to YAML usage for orchestration and the ability to change workflows and create tasks using Kestras API.And the reason why Kestra is bringing the infrastructure-as-code model to data orchestration is because the startup thinks data management should be handled by all engineers and business users instead of a specific team of data engineers.On top of this opinionated approach, Kestra has a solid library of integrations with official plug-ins for major cloud providers (AWS, Azure and Google Cloud), data warehouses (Snowflake and BigQuery), dbt for data transformation, Airbyte for data integration, and more.Kestra also has a user interface that makes it easy to create scheduled and event-driven workflows (if this happens, do that). With this UI, business users can also rely on Kestra to create SQL queries and build internal tools for internal reporting.Originally from France and co-founded by Emmanuel Darras and Ludovic Dehon, Kestra has raised $3 million in a seed round co-led by ISAI and Axeleo Capital.