- As an embedded database, DuckDB does not inherently provide the same isolation for ETL and serving workloads that other OLAP databases offer.
- We have observed significant degradation in query performance during data ingestion.
- In a Kubernetes environment, it is recommended to use local disks instead of network disks, necessitating separate local disk backups.
- Utilizes separate DuckDB handles for reading and writing, each with distinct CPU and memory resources.
- Automatically backs up writes to GCS in real-time.
- Automatically restores from backups when starting with an empty local disk.
- Refer to
examples/main.gofor a usage example.
- Enable writes and reads to be executed on separate machines.
- Limit read operations to specific tables to support ephemeral tables (intermediate tables required only for writes).