Apache Druid
Alternative

Data professionals use Druid for its speed, but miss out on the benefits modern platforms have introduced. Firebolt combines the best of both worlds – query performance that is as fast or even faster than Druid, coupled with a modern decoupled storage and compute architecture, delivered as SaaS and is ANSI-SQL friendly.

Additionally, the complexity of deploying, managing and scaling Druid requires significant architectural planning and resources with specialized expertise. In contrast, Firebolt eliminates operational complexity, allowing you to focus on data analytics.  

As you build your next generation data apps, consider the ease of delivering sub-second response times, high concurrency with the simplicity of SQL, enabled by Firebolt.

Architecture

Druid is a customer managed processing engine designed primarily for OLAP use cases. Druid requires multiple role specific servers, external metadata store and deep storage, making it complex and resource intensive to size, deploy and manage.

Firebolt is a columnar Data Warehouse architected from the ground up for low-latency analytics workloads at TB++ scale with built-in storage optimization. As a SaaS offering, there are no instances or role specific servers to manage.

Decoupled Compute and Storage

Druid does not leverage decoupled compute and storage. While it uses deep storage for persistence, it does not use this data for queries. Due to the reliance on memory and internal storage, Druid deployments may need larger number of nodes.

Decoupled compute and storage is a key design element in Firebolt, allowing independent scaling of compute and unlimited storage. Provides control and visibility of resources for easy, cost effective scaling and workload isolation.

Operational Overhead

Installation, management, upgrades and scaling of a Druid cluster require extensive amount of resources. Requires careful planning and execution around server and storage sizing.

As a SaaS offering, Firebolt abstracts away the complexity of managing infrastructure. Scaling with cloud based resources is performed through Firebolt UI, SQL or APIs.

SQL and Ecosystem Integration

Druid supports a native query language and Druid SQL (limited SQL). Not all features available in the native query language are supported with Druid SQL. Druid offers limited third party ecosystem integration.

Firebolt leverages ANSI-SQL with built-in Query Optimizer and SQL IDE for SQL queries and scaling. Orchestrate through APIs and SDKs or through a growing set of modern integrations including the likes of dbt, Airflow, Superset, Cube and others.

Building Next Generation Data Apps

Data Apps extend analytics beyond traditional Business Intelligence and embedded analytics. Engineering and Developments teams collaborate to create Data Apps.  Firebolt is designed for Data Apps with the following capabilities.

1. Leverage decoupled compute and storage to not only scale but provide workload isolation as well. This ensures quality of service.
2. Ability to spin-up / spin-down environments rapidly using SQL, enhancing developer productivity.
3. Low latency and High concurrency are must have attributes for data apps. Firebolt achieves this with efficiently sized infrastructure.
4. ANSI-SQL and strong database fundamentals to simplify development of Data Apps.

While Druid provides low latency and high concurrency, the cost of delivering this requires sizable infrastructure investments combined with significant skilled resources to operate. While managed versions of Druid promise de-coupled compute and storage capabilities, they do not offer workload isolation or ease of environment management through SQL.

Sub-second Analytics and Concurrency

As customers build data warehouse based data apps, the need for sub-second performance and high number of queries per second are challenging. Firebolt consumes less resources, delivers sub-second analytics and high concurrency at the same time.  While Druid delivers fast performance, this performance comes at a cost. The cost is in the number of components that need to be sized, configured, maintained and not to mention the skills required for continuous upkeep.

Firebolt leverages various types of indexes to deliver fast, high concurrency analytics. Additionally, customers have the option of adding multiple engines, each with scale-out capabilities, to increase concurrency, resource and workload isolation.

Data Model

Druid was not designed to natively support Joins and denormalized data model is considered a best practice. Similarly, nested JSON needs to be flattened prior to loading into Druid.

Firebolt does not require a denormalized data model and executes joins in sub-seconds. Additionally, broad set of choices, including Lambda expressions, are available when working with JSON data.

Figure 5a. Sample JSON.

‍

Physical Data Layout & Ingestion

Druid data partitioning is based on timestamp only. While secondary partition can be specified, if queries are based on the secondary index, it will require the scanning of all time series data. From a data ingestion standpoint, Druid allows ingest of batch and streaming data, however, complex JSON based ingestion specs are required.  

‍Firebolt uses sparse indexes for highly efficient data pruning. Partitions are flexible and optional.  Firebolt ingestion process is simplified through SQL. As simple as “Insert into … Select * from …”. No need for complex JSON specs.

Aggregations and Roll-ups

Druid roll-ups summarize data based on an ingestion spec. To address lack of granularity in roll-ups, multiple independent roll-ups will need to be defined and maintained. 

Firebolt’s aggregating indexes are defined once, incrementally auto-synced at ingest and data can be queried at index speed immediately. All with online access to raw data.

Pricing Model

Druid complexity, in terms of infrastructure sizing and lack of decoupled compute and storage, results in unpredictable costs especially as your data volume grows.  

Firebolt lowers TCO through granular choice of instances without the complexity of cluster configuration, scale to zero with auto-stop and optimized object storage format.

Quick Benchmark Snapshot

Below is a quick comparison of performance on Apache Druid and Firebolt. This is a sample data set consisting of 100 million records on a single flat table. It should be noted that Druid consumes more disk storage and takes longer to ingest. 

Want to try it for yourself? Contact us to get started in Firebolt.

Response time comparison of various queries on Firebolt and Druid:

Our Customers Say

The Firebolt Advantage

Cloud-native architecture

• SaaS with operational simplicity
• Native decoupling of storage & compute
• Easy Workload Isolation & Elastic Scaling

Performance

• Sub-second Latency for Big Data
• High Concurrency Queries 
• Vectorized Processing

Integration with Modern Data Stack

• ANSI-SQL compliant
• REST APIs / SDK 
• Ecosystem Integrations

Cost Efficiency

• Highly efficient and granular Compute 
• Save costs with Auto Stop
• Lower TCO with scalable object storage

Hit us with your most challenging use case

‍