Technology

GCP logs on your buckets, searchable in seconds.

Cloud Logging's retention pricing and log-sink exports mean most teams already ship their high-volume GCP logs to GCS and lose the ability to search them. CtrlB indexes those exports where they land.

Cloud Loggingacme-prod
Audit LogsAudit Logs
GKEGKE
VPCVPC
Cloud RunCloud Run

LOG ROUTER · SINKS

Log bucket
30 days
retention priced
BigQuery
per scan
priced by bytes
Cloud Storage
years
storage priced
Cloud Storage
gs://acme-log-archive
the bucket your sink already writes to
cloudaudit/
JSON · Parquet
_ctrlb_index/
written in place

no second destination · no re-export

Log sinks
Index what they already write
In place
Indexes land in your GCS bucket
Data access
Audit logs affordable to keep on
GKE
Per-pod dimensions preserved

Priced per question

An investigation is never one query

The export is queryable — that was never the problem. The problem is that a scan-priced engine charges for the whole window every time you narrow the search, so the sixth refinement costs exactly what the first one did.

who changed the IAM policy on that bucket?18 months · 3.8 TB exported
BigQuery on the export0.0 TB scanned
principalEmail = "svc-deploy@acme.iam"3.8 TB
+ methodName : "SetIamPolicy"3.8 TB
+ resource.type = "gcs_bucket"3.8 TB
− drop the read-only methods3.8 TB
+ widen the window to 18 months3.8 TB
+ callerIp = "203.0.113.9"3.8 TB

every refinement reads the window again — the table does not remember the last question

CtrlB on the same bucket0 full scans
principalEmail = "svc-deploy@acme.iam"index read
+ methodName : "SetIamPolicy"index read
+ resource.type = "gcs_bucket"index read
− drop the read-only methodsindex read
+ widen the window to 18 monthsindex read
+ callerIp = "203.0.113.9"index read
The index sits in the same bucket as the export it describes
Narrowing a search costs less than the search before it
Nothing is loaded, copied, or moved to be queried

Worked example, not a benchmark: one investigation of six refinements against an 18-month audit export of 3.8 TB.

Cloud LoggingCloud Logging sinks

The export to GCS was the right cost call. Search shouldn't require a second destination.

Data access audit logs are the highest-value source — and the first ones teams sink to cold storage because retention pricing makes keeping them in Logging untenable. CtrlB indexes what your sink already writes.

log router · sink destinations
Audit Logs
GKE
VPC
→ Log Router →
Log bucket
30-day retention
BigQuery
priced per scan
Cloud Storage
cheap · cold export
GCS + CtrlB
storage priced · indexed
high-volume logs leave Cloud Logging every day
gs://logs-export/
index beside data
export prefixes
📄audit/2026/08/
📄vpc/2026/08/
📄gke/containers/
.ctrlb/indexes/
audit.idx
principal.idx
ip.idx
BigQueryBigQuery · full scan
minutes
bytes scannedpriced per TB
CtrlB · index lookup
bytes readindex + row groups
org-wide sinks
billing-prod
data-plane
security
gke-host
Cloud Storage
one bucket · one history
projectfolderorg

one query spans every project that feeds the bucket

What the sink costs you

01

Data access audit logs are the highest-value and highest-volume source, and retention pricing makes keeping them a difficult argument.

02

The usual answer is a log sink to GCS, which solves cost and removes searchability in the same move.

03

Querying the export means BigQuery loads or full scans, priced by bytes and measured in minutes.

04

GKE container logs are voluminous and repetitive, so they get sampled at exactly the wrong time.

What changes

The GCS bucket your sink already writes to becomes the searchable lake — no second destination and no re-export.

Point queries against audit and flow logs return in seconds instead of minutes, without a per-scan bill.

Data access audit logging stays on, because keeping it is storage rather than retention pricing.

GKE logs and metrics keep per-pod identity, since dimensions are columns rather than billable series.

How it works

Leave your sinks exactly as they are. Indexes write into the same bucket, and queries span every project that feeds it.

Step 01
Audit Logs
GKE
VPC
Cloud Logging · Log Router

existing sink config untouched

Cloud Storage
gs://logs-export/
sink unchangedCtrlB attached
1

Keep your existing sink

Cloud Logging sinks already export to GCS. Leave them exactly as they are and point CtrlB at the destination bucket.

Step 02
BigQuery load
scanfull export

minutes · priced per TB

index lookup
readindex + row groups

sub-second

gs://logs-export/
├── audit/2026/08/28/
├── vpc/2026/08/28/
└── .ctrlb/indexes/audit.idx
2

Index where it lands

Compact inverted indexes are written into the same bucket, so a needle query reads an index rather than scanning the export.

Step 03
billing-prod
data-plane
security
gke-host
Cloud Storage
one bucket · one history
projectfolderorg

one query spans every project that feeds the bucket

3

Query across projects

Sinks from many projects into one bucket become one searchable history, with project as a dimension.

In the product

The shape of a GCP log survives the trip

Audit entries arrive deeply nested, with repeated blocks and ajsonPayloadthat differs per service. A destination with a declared schema has to flatten that or drop it. Here the structure is the index.

Audit log recordcloudaudit.googleapis.com%2Factivityas it left the sink
63 fields · nothing flattened
protoPayload
serviceName"storage.googleapis.com"
methodName"storage.setIamPolicy"
authenticationInfo
principalEmail"svc-deploy@acme.iam.gserviceaccount.com"
authorizationInfo[0]
permission"storage.buckets.setIamPolicy"
grantedtrue
requestMetadata
callerIp"203.0.113.9"
callerSuppliedUserAgent"google-cloud-sdk/458.0.1"
resource
type"gcs_bucket"
labels
bucket_name"acme-log-archive"
project_id"acme-prod"
location"europe-west4"
severity"NOTICE"
Filterable as it stands
protoPayload.authenticationInfo.principalEmail1.2K
protoPayload.methodName38
protoPayload.authorizationInfo[].permission214
protoPayload.requestMetadata.callerIp907
resource.labels.bucket_name62
resource.labels.project_id9
severity5

Repeated fields stay repeated and nested objects stay nested, so a leaf nobody planned for — a user agent, a permission, a label added last quarter — is a filter the moment you think of it.

Coverage

Anything the Log Router can reach

There is no per-service connector to wait for. If a service writes to Cloud Logging, a sink can export it and CtrlB can index it where it lands — including services you turn on after the bucket already exists.

Audit LogsAudit Logs
GKEGKE
VPCVPC
Load BalancingLoad Balancing
Cloud RunCloud Run
Compute EngineCompute Engine
Cloud FunctionsCloud Functions
App EngineApp Engine
Cloud SQLCloud SQL
SpannerSpanner
FirestoreFirestore
Pub/SubPub/Sub
Cloud ArmorCloud Armor
Cloud CDNCloud CDN
IAMIAM
Cloud DNSCloud DNS
Cloud NATCloud NAT
Secret ManagerSecret Manager

Icons are Google’s official Google Cloud product icons, shown unmodified.

What you get

Audit logs in full

VPC flow logs

GKE workloads

Organisation-wide sinks

Your project, your bucket

Multi-cloud

Production proof
GoComet
Events as they unfold

“CtrlB gives us real-time visibility into our logistics systems — I can see critical events as they unfold. It’s now woven into how we operate every day at GoComet.” — Ayush Lodhi, CTO & Co-Founder.

See what changed for GoComet

Questions

No. If a sink already exports to GCS, CtrlB indexes that bucket. Collection stays exactly as configured.

BigQuery is a strong analytical engine but log search is a different access pattern. CtrlB's inverted indexes make point lookups sub-second without loading or scanning the whole export.

That is usually the first thing teams turn back on, since the cost becomes GCS storage rather than log retention pricing.

Yes. Aggregated sinks across many projects land as one searchable history with project and folder as dimensions.

Put it on your own bucket and see.

Point a source at CtrlB, keep everything else where it is, and compare the same investigation side by side.