Nikhita works with data and platform teams building high-concurrency, real-time systems—everything from telemetry and event pipelines to customer-facing analytics and AI-enabled experiences
Traditional data warehouses are great at what they were built to do: batch-oriented analytics. Load data on a schedule, run large analytical queries, publish dashboards, repeat.
The problem is that hi-tech workloads today no longer behave that way.
In practice, I keep seeing the same pattern play out. Teams invest heavily in a warehouse, dashboards look fine, leadership feels confident—until real users, APIs, and services start hitting the same data at the same time. That’s when latency spikes, costs jump, and engineers start adding “just one more system” to keep things running.
Today, data is created continuously. It’s queried by humans and machines. It’s monetized directly inside products. And increasingly, it’s consumed by AI agents that expect fresh context now—not after a nightly job finishes.
In short: modern hi-tech workloads break the core assumptions warehouses were built on.
Let’s unpack what’s changed, why “just scale the warehouse” usually fails, and what the most successful teams do differently when real-time performance and concurrency actually matter.
Traditional warehouses, whether on premises or in the cloud, were built around a fairly stable set of assumptions. Data arrives in batches, queries are mostly analytical, concurrency is manageable, schemas do not change too often, and waiting a few seconds or even minutes for results is acceptable. A data warehouse excels at retrospective analysis, but struggles when pressed into real-time operational service.
Even the modern cloud versions of these platforms still carry that DNA. They are much easier to operate and far more powerful than earlier generations, but they still assume that analytics happens after data has landed, been transformed, and been modeled into something suitable for reporting.
That workflow made sense when most questions were about what happened yesterday or last week.What I see now is very different. Hi tech companies increasingly operate on what is happening right now.
In modern hi-tech environments, “data” usually means:
Continuous streams of events, logs, clicks, and telemetry
Operational + analytical queries hitting the same dataset
High concurrency from thousands of users, APIs, services, and background jobs
Wide, fast-evolving schemas with hundreds of attributes, and with new fields added weekly
Sub-second SLAs powering customer-facing experiences
And increasingly: AI-driven queries: vector search, retrieval, feature lookups, and agentic workflows
If you’ve built a modern product analytics pipeline, a real-time monitoring dashboard, an ad-tech bidding system, or a hardware manufacturing quality loop, you already know this pattern.
What’s changed is that teams aren’t just observing systems in real time anymore. They’re expected to act in real time—automatically, continuously, and at scale.
That’s where many warehouse-centric architectures start to strain.
Agentic real-time applications require data access across transactional, structured, and semi-structured layers at high concurrency. Unlike traditional analytics, the semantic layer itself must be available in the real-time execution path so AI agents can retrieve, reason, and act without delay.
When teams try to push warehouses into real time, high concurrency roles, I tend to see the same problems show up over and over again. It does not usually fail in one dramatic moment. Instead, things slowly get more complicated, more expensive, and harder to operate.
The symptoms are familiar:
more and more copies of the same data
sudden latency cliffs that are hard to predict or explain
fragile pipelines held together by “one more job” fixes
costs that climb quietly until someone finally flags them
engineering teams spending more time moving data than actually using it
In my experience, these problems almost always trace back to a few structural mismatches between how warehouses were built and how hi tech systems now operate.
Warehouses are designed for large scans and aggregated analytics. That is exactly what you want when you are asking, “What happened last night?”
Many hi tech businesses now live in what is happening right now. A feature flag rollout starts causing errors, a manufacturing line begins producing defects, a funnel suddenly drops conversion, or an AI agent is making decisions and needs fresh context in the moment. These are not situations where yesterday’s data is good enough, and they are not problems teams can afford to diagnose hours later.
When your data flow still follows the pattern of ingest, then land, then transform, then load, and only then serve, you have already lost critical decision time before anyone can react. In environments where speed directly affects revenue, reliability, or customer trust, those minutes or hours of delay quickly become part of the problem instead of just an inconvenience.
In many environments I work with, that delay is not just inconvenient. It shows up directly in lost revenue, degraded customer experience, and increased operational risk. Teams know something is wrong, but the data they need to act on it is always a step behind.
Most conversations about scale focus on how many rows you can store or how fast a single query runs. In practice, what breaks systems much faster is how many things are trying to access the same data at the same time.
One dashboard is easy.A hundred internal analysts is usually manageable.
But thousands of concurrent API calls, dashboards, background jobs, customer queries, and now AI agents all hitting the same datasets is where warehouse backed architectures start to struggle, especially when they were never designed to behave like operational serving layers.
This kind of pressure usually shows up as:
queues forming and tail latency becoming unpredictable
systems that feel stable right up until they suddenly are not
expensive scaling strategies that still fail to deliver consistent SLAs
And this is the part teams rarely plan for. Once concurrency becomes a first class requirement, most warehouse architectures push you toward adding more systems to handle operational traffic. At that point, you are no longer just tuning queries. You are redesigning the entire stack, often while the business is already depending on it.
When teams finally accept that the warehouse is not going to work as an operational serving layer, the reaction is rarely a full architectural rethink. Much more often, they start adding systems around it to make things work. A cache here, a stream there, another database for fast reads. Each addition solves a real problem at the time, so it feels reasonable. Over months and quarters, though, the stack gets wider and harder to reason about.
In the environments I work with, the hot path usually ends up running through a long chain of systems: a warehouse for analytics, a transactional database for application reads and writes, a cache for the most time sensitive requests, a streaming system to move events, ETL or ELT jobs to keep datasets aligned, a vector database for semantic retrieval, a feature store for machine learning and AI, and then some form of governance layer trying to keep everything consistent. Each system becomes a separate consumer of the same underlying data sources, increasing duplication and drift.
None of these choices are bad in isolation. They even make perfect sense when you look at the immediate problem they were meant to solve.
What concerns me is what happens once all of them are tied together. This is when pipelines become fragile, business logic starts to get duplicated in multiple places, and teams spend more time arguing about which numbers are right than improving the product or the customer experience. The complexity is no longer hidden. It becomes part of day to day operations.
This fragmentation shows up even more clearly once AI driven workflows enter the picture. Real applications are not just running vector similarity search and calling it a day. They are trying to retrieve similar items, apply tenant and entitlement filters, join back to relational metadata, add recent behavioral context, and return results fast enough to feel instantaneous to users, all while handling heavy concurrent traffic. This requires joining multiple data sources—structured, semi-structured, and vector-based—in a single execution path.When that chain of logic is spread across several systems, latency budgets disappear much faster than most teams expect. Every extra hop and sync job adds uncertainty, and performance tuning turns into a constant game of whack a mole. At that point, the problem is not about whether you picked the right index or the right cache. It is that the architecture itself is working against you.
Warehouse pricing models were designed around analytical workloads. You pay for spinning up compute, running large scans, refreshing materialised views, and keeping multiple copies of data in object storage. That works fine when you are running dashboards or scheduled reports a few times an hour or a few times a day. When a data warehouse is used as an operational system, those pricing assumptions quickly break down.
Production systems look very different. Data is arriving all the time, not in neat batches. Large portions of it need to stay hot and queryable, not pushed off to cheaper storage. On top of that, you are serving traffic from users, services, background jobs, and increasingly AI driven workflows, all hitting the same datasets at once. Updates are not just inserts at the edge of the pipeline either, they are happening throughout the system.
What I see in practice is that costs do not usually spike immediately. They creep up, quarter after quarter, until someone finally asks why the warehouse bill is now part of board level discussions. At that point, teams realise they are paying analytics pricing to support operational behaviour, and unwinding that architecture is much harder than it would have been to design for it up front.
The strongest teams make a fundamental shift. They stop treating real-time operational data as something to warehouse later. They treat it as strategic data—and they build a data layer that can handle both operational and analytical workloads on the same data, at scale.
Three patterns consistently show up.
This is not about ripping out everything overnight. In most cases, the real goal is simply to reduce how many systems sit directly in the hot path.
The architectures that tend to work best are the ones where data can be ingested continuously and queried immediately, using the same engine for point lookups, joins, and analytical queries. That also means concurrency behaves predictably, and teams are not forced to keep operational and analytical copies of the same data in sync across multiple platforms.
This is why HTAP style designs have become so relevant for hi tech workloads. They allow teams to work with fresh operational data and still run analytical queries on it, without first pushing everything through a chain of pipelines and transformations. This is the defining capability of a real time analytical database: operational freshness with analytical power in a single system.
If you want a practical starting point, SingleStore lays out the core HTAP concepts in its beginner guide, along with how those ideas map to real time operational analytics in production systems.
High performing teams do not treat operational data as something to archive first and analyze later. They analyze it as it is generated, while it is still relevant.
I see this across very different industries. Semiconductor teams analyze test and yield data directly from manufacturing logs. Ad tech platforms recalculate segments and signals in real time as auctions and impressions happen. Sales tech companies run complex search across billions of records while users are actively working in the product. Mar tech platforms continuously update personalization as behavior changes.All of these use cases rely on the same underlying capabilities: fast ingestion, high concurrency, and the ability to query live operational data without waiting for it to be reshaped into warehouse friendly models. This is also why telemetry ingestion, device and hardware testing, and real time monitoring loops tend to expose architectural limits very quickly.
AI workloads are no longer something teams are planning for next year. In many products, they are already becoming part of the core execution path.
Support assistants need access to live account and interaction data. Anomaly detection runs directly on operational telemetry. Personalization models update continuously. Agent workflows retrieve information, reason over it, and then trigger actions.
All of this changes the concurrency profile in ways that are easy to underestimate. Humans tend to issue one query at a time. Agents do not. They fan out, retry, and run multiple retrieval steps in parallel. Even systems that feel comfortable under dashboard and API traffic can struggle once agent driven access patterns are introduced. This shift exposes whether your analytics database was built for dashboards, or for continuous, machine-driven access.
This is also where keeping vector retrieval close to relational context becomes important. When semantic search and structured filtering happen in different systems, latency and complexity both increase. Platforms that can handle both together make it easier to keep AI workflows fast and predictable under load. This is exactly why SingleStore supports indexed vector search alongside relational queries, so retrieval and filtering can happen in the same execution path rather than being stitched together across services.
One of the more memorable explanations I have heard came from a semiconductor team working on fault isolation. They described the problem as trying to spot a single hot dog on a street while sitting on the International Space Station. That is roughly what it feels like to find one faulty transistor in a chip with billions of components.
In one case, by leveraging SingleStore to query scan chain data directly instead of pushing it through multiple analytical layers, search times dropped from weeks to seconds. Engineers could run thousands of iterations in a short session instead of waiting minutes for each attempt. If you have ever been stuck in slow debug cycles on manufacturing data, you know how much that changes the pace of development.
A similar pattern shows up in product analytics. Many stacks rely on heavy rollups to make data queryable, which means giving up granularity and accepting slower feedback. When teams can ingest billions of events and query them at full fidelity, analysis happens at the level of individual interactions and up to the current moment. Questions do not need to be turned into modeling projects before they can be answered.
Heap is a public example of this approach in production, and IEX Cloud is another case where high volume, real time data is served at consistently low latency to external users.
If you’re building or refactoring your data stack for modern hi-tech workloads, here’s the playbook I come back to.
- Identify your “hot path” use cases
Not every workload needs millisecond freshness, but some do.
Customer-facing search and recommendations, telemetry and anomaly detection, real-time personalisation, and manufacturing quality loops are common examples. Be honest about where latency creates real business risk. - Design for concurrency first
Most systems don’t fail on data volume; they fail on concurrency. Measure concurrent requests, burst patterns, p95/p99 latency, multi-tenant effects, and how AI agents amplify traffic. Observability is how you catch problems before they escalate. - Collapse the stack where it matters
You don’t need a dozen systems in the hot path. Every extra system adds latency, cost, failure modes, and governance overhead. Fewer copies, fewer sync points, and less glue almost always win. - Keep operational and analytical truth aligned
The advantage isn’t just faster queries, it’s fewer arguments about what’s true. When operational and analytical views are unified, teams trust live metrics and move faster. - Build with AI in mind
Even if you’re not shipping AI today, you likely will be. If your architecture can’t support retrieval with relational joins, high concurrency, fast updates, and consistent semantics, you’ll end up bolting on another system later.
Traditional warehouses did not fail. They are simply being asked to do something they were never built for: act as the real time, high concurrency data plane of a modern hi tech business.
If your business runs on now, your data layer needs to run on now too.
If you are trying to figure out whether your current stack is starting to work against you, I usually suggest starting with a few simple questions. Which workloads truly sit on your critical path, and how fresh does that data need to be to make good decisions? How many systems does a single user request or API call actually touch before it returns a result? What happens to latency and cost when traffic spikes, not when data grows? And where are you already copying the same data just to make different systems usable?
You do not need to rip everything out to move forward, but you do need to be intentional about where consolidation will actually reduce risk instead of adding more layers. In many cases, collapsing the hot path and keeping operational and analytical access closer together delivers bigger gains than any amount of query tuning.
In the next post in this series, I will go deeper into what a unified, real time architecture looks like in practice and how teams evaluate it without getting pulled into yet another multi system maze. But if you are already wrestling with concurrency limits, real time SLAs, or AI driven retrieval on operational data, that is usually a strong signal that it is time to start simplifying, not adding more pieces.
.jpg?width=24&disable=upscale&auto=webp)
