Highlights
- Zepto handles about 2 PB of WAF, CloudTrail and other security logs per month, stored in S3 bucket and used for security investigations and operational troubleshooting.
- Searching historical security logs with Amazon Athena was slow. A typical query took around 5 minutes, and some timed out after 30 minutes.
- CtrlB replaced Athena for this workload, bringing typical query times down to 500 milliseconds directly on the Parquet data in Zepto's own S3 bucket.
- With no migration required, the security team was up and running within hours and saw value in less than a week.
- CtrlB's 2X better compression allowed Zepto to store twice as much data in S3, further reducing storage costs.
See it on your own data
Behind Zepto's Lightning-Fast Security Investigations with CtrlB
Zepto pioneered grocery delivery within 10 minutes and the YC-backed unicorn is now a $4.5 billion company. Making that possible required innovating every step of the grocery delivery supply chain, including how goods are picked, sorted, packed, and delivered, then building an operating model that could deliver on that promise at scale. Today, hundreds of millions of urban households use Zepto to get goods ranging from a quick snack to the latest iPhone delivered to their doorsteps in minutes.
Operating at that speed and scale means that telemetry data piles up quickly. Zepto cannot compromise on reliability and data security. Every production incident means not only financial loss, but also poor customer experience, which is non-negotiable for its engineering team. A cyberattack can mess up the proprietary data models that make the right pricing possible.
That makes fast access to historical logs critical. During incident response or threat hunting, infrastructure teams need to look through petabytes of logs, correlate activity, and run successive queries to establish what happened. When each query takes several minutes, investigation workflows slow down while customers keep wondering why the app is not working as expected.
This is why improving the speed of searching logs was important for Zepto.
This is where CtrlB came in.
The challenge: Several minutes for each query on telemetry during incidents
Zepto handles 1.3 PB of WAF and CloudTrail logs per month. These data sources provide visibility into web traffic and activity across its AWS environment, supporting security investigations and operational troubleshooting.
Zepto used Amazon Athena to query this data. Typical queries took around five minutes to return results and some even timed out after 30 minutes, significantly slowing down the engineering team.
For security and infrastructure teams, query latency directly affects the pace of an investigation. They often need to refine queries, pivot across events, and correlate findings to narrow down the scope of an incident.
The challenge was to accelerate queries over large volumes of security data while retaining the benefits of storing that data in Amazon S3.
The solution: Replacing Athena with CtrlB
CtrlB replaced Athena for this workload, bringing typical query times down from around five minutes to 500 milliseconds.
Typical query time
But that did not mean that Zepto had to remove anything else from its stack. Rather than requiring Zepto to move its underlying data into a separate search data store, CtrlB could work with the Parquet data in Zepto's own S3 bucket.
So there was no migration required. The security team was able to get up and running within hours and see value in less than a week.
See it on your own data
The results: Lower query latency, better compression
The impact is measurable across query performance, compression, and data ownership.
Query latency: From around five minutes to 500 milliseconds
CtrlB reduced typical query times from around five minutes with Athena to 500 milliseconds, helping security and infrastructure teams move through investigations with less waiting between queries.
Data stays in Zepto's own S3 bucket
Zepto continues to store its underlying telemetry in its own S3 bucket, while CtrlB provides the search layer without requiring the data to be moved to a separate search data store for this workload.
More data in object storage, thanks to 2X more compression
Zepto was able to store 2X more data in its S3 bucket, allowing the security team to save further on storage cost.
Before CtrlB, historical security log queries typically took 5 minutes with Amazon Athena, and some timed out after 30 minutes. Now, the same data in our own S3 bucket is searchable within half a second. It's changed how quickly Zepto can investigate incidents.