--- title: "Your Observability Stack Was Built for Humans. AI Agents Don't Work That Way." description: "A developer asks an AI coding agent, “Why did checkout errors spike at 14:20?” The agent doesn't run one query and wait for an answer. It checks the checkout logs. Then traces. Then a downstream service. Then the deployment history. Then the database errors. The result of each query determines what…" canonical: "https://ctrlb.ai/blogs/ai-coding-agents-your-observability-stack-was-built-for-humans" publishedTime: "2026-09-01" modifiedTime: "2026-09-03T13:25:13+0000" author: "Adarsh Srivastava" tags: [] --- # Your Observability Stack Was Built for Humans. AI Agents Don't Work That Way. ![](https://images.prismic.io/ctrlb-new/dUIGzqs2q880cHlR_Screenshot2026-09-01at6.41.40PM.png?auto=format,compress) A developer asks an AI coding agent, “Why did checkout errors spike at 14:20?” The agent doesn't run one query and wait for an answer. It checks the checkout logs. Then traces. Then a downstream service. Then the deployment history. Then the database errors. The result of each query determines what it asks next. That's a very different workload from a human opening a dashboard and running one or two queries. Observability systems were largely built around the second workflow. AI agents are quickly making the first one much more common. [Gartner](https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027?utm_source=ctrlb.ai&utm_medium=referral&utm_campaign=blog&utm_content=ai-coding-agents-your-observability-stack-was-built-for-humans) expects a significant share of agentic AI projects to be abandoned because of cost, unclear ROI, and operational risk. The infrastructure underneath these systems is part of that equation: agents need to repeatedly access the data they are reasoning over. AI coding tools are increasing how quickly teams can write and ship software. That also means observability systems may have to handle more frequent changes and more automated investigation. ## How AI Agents Move From Logs to Root Cause Human telemetry pattern: A human SRE gets an alert, opens a dashboard, and runs 1 to 2 targeted queries to check a single hypothesis. Agentic telemetry pattern: A coding agent gets a prompt (something like "Why did checkout error rates spike at 14:20 UTC?") and immediately kicks off an automated loop of roughly 50 queries. It works through logs, traces spans across microservices, correlates metric anomalies, and checks security events across the whole estate, all on its own. logs → traces → metrics → deployment → database → logs again Each answer creates the next question. That's the important difference. It's not that an agent asks exactly 50 queries every time. It's that it can keep querying until it has enough evidence to form a conclusion. ![](https://images.prismic.io/ctrlb-new/P75BfrsY8REE3wOd_Screenshot2026-09-03at5.10.00PM.png?auto=format,compress) At enough agent concurrency, repeated investigation can turn a small interactive workload into a significant query and telemetry workload. Most observability platforms were designed around humans asking occasional questions. Agents can continuously query, refine, and retry. ## The problem isn't always the model Most discussions about agent reliability focus on the model. In production, the problem is often simpler, the evidence the agent needs is no longer there. And that's really the problem. A human engineer who opens a dashboard to an empty panel knows right away there isn't enough to go on. An LLM doesn't always draw that same line. Give it a thin, sampled, or cut-down set of telemetry, and it will often produce an answer anyway instead of flagging that the evidence is incomplete. That answer can come out sounding perfectly confident, even though it's built on very little. A human engineer usually recognizes an empty dashboard as a dead end. An LLM may still produce an explanation from incomplete evidence. [Chroma's Context Rot](https://www.trychroma.com/research/context-rot?utm_source=ctrlb.ai&utm_medium=referral&utm_campaign=blog&utm_content=ai-coding-agents-your-observability-stack-was-built-for-humans) research found that model performance can degrade as irrelevant information accumulates. Their results show that more context isn't automatically better; focused context can outperform a much larger input containing distractors. ![](https://images.prismic.io/ctrlb-new/QNoM-inmseFBrh2h_Screenshot2026-09-01at6.36.11PM.png?auto=format,compress) Observability has a related problem in reverse: if important events were sampled out months ago, the agent doesn't have enough context to investigate the incident. A sampled log stream is basically the distractor problem playing out at production scale. Sampling doesn't just reduce volume. It can remove the exact event an agent needs to connect two parts of an incident. ## Keeping everything is expensive. Searching everything is harder. Faced with $250,000-a-month observability bills, SRE teams often end up dropping data, sampling logs, or capping retention at 7 days just to keep costs under control. ![](https://images.prismic.io/ctrlb-new/4DC44uUE2sgAljVF_Screenshot2026-09-03at5.21.24PM.png?auto=format,compress) The trouble is, this sets off a pretty destructive feedback loop: Agent hallucinations: AI agents lean on historical context to spot regressions. Delete the 90-day-old logs and the agent has nothing to compare current runtime metrics against, so it either gets the root cause wrong or gets stuck in retry loops. The latency trap: Cheap object storage solves the retention problem, but not necessarily the search problem. A large full-text search over a conventional data lake can take minutes or longer. That becomes painful when an agent is running an iterative investigation and each query depends on the result of the previous one. ## So what does an agent-ready observability stack look like? The underlying architecture has to solve two problems at the same time: Keep telemetry affordable to retain. Make that telemetry fast enough to query repeatedly. That's what CtrlB is built for, agent-scale telemetry and agent-scale queries. A 50+ query investigation over a petabyte-scale lake should feel as responsive as a handful of queries over a small dataset. ### Keep the raw telemetry CtrlB stores telemetry in open Parquet on the customer's object storage rather than requiring all data to live inside a proprietary storage layer. ### **Make historical data searchable** Rather than building a large index over every value, CtrlB uses a lightweight indexing layer to narrow the amount of data that needs to be scanned. The goal is to keep the index small while still making large historical searches practical. CtrlB keeps the index ratio down around 0.6%. Queries fan out across serverless workers, so a full sweep over 125 TB of logs takes about a minute instead of an hour. ### Let the agent query the same data Agents can query full telemetry directly from IDEs like Claude Code or Cursor using natural language, no dashboard hopping, no manual SQL, and no per-detection vendor tax. **$0.20 per GB, charged once at ingest:** That $0.20 is CtrlB's platform fee on data as it arrives. The data itself sits as Parquet in your own S3, GCS, or Azure bucket, and you pay your cloud provider its normal object-storage rate to keep it there. Nothing is billed again for age, so moving from 7 days of retention to 365 costs you object storage and nothing else. Totals land up to 90% below what legacy tools charge for a fraction of the window. ![](https://images.prismic.io/ctrlb-new/IwZxQbbXqeqdbgDI_CtrlB-one-data-layer.png?auto=format,compress) The underlying system needs to do three things well: retain the data, search it quickly, and make it accessible to both people and machines. ## _An agent is only as good as the evidence you kept around. _ If your observability stack is struggling with the demands of AI coding agents, **[let's tal**k](https://ctrlb.ai/contact) or **[explore our doc**s](https://docs.ctrlb.ai/) to see how CtrlB handles the workload.