| _layout |
|---|
landing |
Arcane is a data streaming platform based on Akka.NET that focuses on providing a simple, reliable, and scalable solution for data streaming. It is designed to be Kubernetes-native and is built to be cloud-agnostic.
- Get your data updates in real-time from various sources to a cloud warehouse or data lake for further processing and analysis.
- Use built-in observability features to monitor the streaming jobs and get insights into the data flow.
- Extend the platform with custom source and sink plugins to fit your specific use case.
- Use the built-in backfilling mechanism to ensure that data is not lost in case of data source schema changes.
Unlike other data streaming solutions like Kafka, Arcane is not based on any kind of consensus algorithm. This absence of consensus-based architectures eliminates their limitations, allowing Arcane to scale both horizontally and vertically. Additionally, it enables Arcane to benefit from the use of inexpensive spot instances in any cloud.
Arcane is implemented as a Kubernetes Operator that manages independent streaming applications isolated in the Kubernetes jobs.
Arcane comes with built-in observability features powered by Datadog platform.
Arcane utilizes a plugin architecture that allows extending its functionality with custom source and sink plugins.
{width=1076 height=1076, style="display: block; margin-left: 15%"}
Concepts overview.
Quickstart deployment guide.
Plugins development guide.
Operator Architecture overview.
<iframe width="560" height="315" src="https://www.youtube.com/embed/x0d-5VEL8eE?si=gRPLn8rjq2EIRAVJ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
Arcane is under active development. Check the roadmap for the upcoming features.