Search & Investigate

One engine for search, SQL, metrics, and traces.

The Data Engine stores, indexes, searches, and analyzes everything you send. Telemetry lands as open Parquet in object storage you own, and compact inverted indexes are written beside it as data arrives — so a needle query reads an index and a handful of row groups rather than scanning the lake.

One engine, every signal

Search, SQL, metrics, and traces — one query surface

Logs, metrics, and traces land in the same lake of open Parquet. Full-text search, standard SQL, metric math, and trace lookups all run over the same files — no hot store to keep in sync, no second copy to query.

One engine · four query surfacesopen Parquet · your bucket
level:error AND service:checkoutrun
ERRORcheckout timeout after 30s upstream=payments-api
WARNretry budget exhausted for payments-api region=us-east
ERROR499 client closed request path=/api/checkout

3 of 214 matches · 380 ms · scanned 41 MB

One lake of open Parquet. No hot store to replace.

The engine

The index that reads almost nothing

Speed here is not a bigger cluster. It is a sparse index that throws away almost the entire lake before a single byte moves. Petabytes go in, a sliver comes out, and the answer arrives in half a second.

One query · point lookup on trace_idanswer in 500ms
the bucket1.4 PBafter file pruning812 filesrow groups opened2,140 groupsbytes read41 MBOpen Parquet, your bucket500ms queries

41 MB read out of a 1.4 PB lake. That is roughly 3 bytes in every 100 million.

Stage 01

1.4 PB

Everything you sent, in open Parquet, in your own bucket. Nothing sampled, nothing dropped.

Stage 02

812 files

The sparse index prunes 46.2M files down to the handful that can possibly match.

Stage 03

2,140 groups

Inside those files, only the row groups whose ranges cover the predicate are opened.

Stage 04

41 MB

Bytes that actually leave storage. This is why a read costs us close to nothing, and costs you nothing.

Indexes 100× smaller

A point lookup touches a handful of blocks instead of a partition scan. High-cardinality fields such as trace_id, span_id, and user_id are indexed by default, not by request.

#1 combined on ClickBench, untuned

Top of the combined board with no per-query tuning and no hand-built schema. The default configuration is the benchmark configuration.

Reads are never metered

A standing query that re-sweeps a year of history every hour is a cron job here, not a billing event. The economics come straight out of the funnel above.

What the engine gives you

  • Full-text search over raw log bodies, with no fields to declare in advance.
  • Standard SQL for aggregation, joins, and windows across every signal.
  • Metrics stored as columns rather than series, so cardinality has no ceiling.
  • Stateless query nodes that fan out on demand and scale to zero when idle.
  • Schema-less ingestion: new fields index the moment they appear.
  • First place on the combined ClickBench result, untuned.

Other core components

All four run over the same lake of open Parquet in object storage you own.

Back to platform overview

Query a real lake before you send anything.

The playground runs against live data. Type a query, watch what it scans, and see the latency for yourself.