Scale Testing with Locust: How We're Going to Crush Our Own API (Before Users Do)

Modern cloud systems are dangerously good at hiding their limits. Auto-scaling dashboards stay green. Serverless promises "infinite scale." Latency looks fine in staging. Then one day, a feature gets shared, a reel goes viral, a push notification lands at the wrong time, and that's usually when teams learn, painfully, where their system actually breaks.

At Hoomanely, we don't want that moment to come from users, so we're going to manufacture it ourselves. Here's how we're building a realistic scale-testing setup using Locust, what exactly we plan to test, which metrics matter, and how the results will shape infrastructure limits, product decisions, and even LLM cost controls.

Why we're doing this before we "need" it

Our stack looks modern and robust on paper: mobile clients built in Flutter, APIs running on AWS Lambda behind API Gateway, data stored in DynamoDB, conversational intelligence powered by Bedrock. Each component scales well on its own. The risk lives in how they interact under simultaneous load.

The goal is having hard data on system limits before user growth accelerates, not after. We have one core question: what actually happens when 100 real users talk to our system at the same time? Not synthetic traffic, not benchmark requests, real flows, real delays, real cost. Instead of guessing, we're going to answer this with data.

What we mean by "realistic" load testing

A lot of load tests fail before they begin because they test the wrong thing: firing the same endpoint in a tight loop, ignoring user think time, skipping authentication, measuring only requests per second. That's not how users behave. We're going to simulate complete user sessions, not isolated API calls, with each virtual user behaving like an actual person using the app.

The user flow we're going to simulate

Every virtual user follows the same journey: authenticate, start a new chat session, pause for a few seconds like a human thinking, then ask a question that triggers a full conversational response. That single question is deceptively expensive, behind the scenes it can involve writing conversation metadata, storing chat history, invoking an LLM, recording analytics events, and updating user context. This is the exact moment where latency, throughput, and cost collide, which is why we're designing the test around this flow.

Why we chose Locust for this

Locust lets us think in terms of users, not requests. It lets us create stateful virtual users, maintain authentication across requests, introduce natural waiting periods, increase concurrency gradually, and observe percentile-based latency. Most importantly, it helps answer the question we actually care about: what does the slowest 5% of users experience as concurrency increases?

How we're going to structure the test

We'll ramp up deliberately, starting with a small number of concurrent users and slowly increasing load in controlled steps, holding traffic steady at each level long enough for the system to stabilize. Our ramp plan: start at 10 concurrent users, add 10 every 2 minutes, hold each level for 5 minutes, and stop when either error rates exceed 1% or p95 latency crosses 5 seconds.

At every level we'll record median conversation latency, 95th percentile latency, error rates, throttling signals, and timeout frequency, watching these in real time through CloudWatch, Datadog, and Locust's own dashboard. The combination gives us infrastructure-level signals like Lambda throttles and DynamoDB rejections alongside user-facing latency. The goal isn't to pass the test, it's to find the edge of failure.

The one metric that matters most

We're intentionally ignoring flashy metrics like raw throughput. The metric we care about is conversation latency, specifically the time between a user submitting a question and receiving the full response. It matters because it maps directly to user experience, captures downstream dependencies, and degrades before outright failures occur. In most systems, errors show up late; latency spikes show up first, and we want to catch the system in that uncomfortable middle ground.

What we expect to break first

We're not going into this blind. Based on our architecture, we expect pressure to build in three places.

DynamoDB write capacity: each conversational turn generates multiple writes touching conversation state, message history, user context, and analytics streams. Under concurrent load, write capacity units can be exhausted quickly, and when that happens DynamoDB doesn't fail loudly, it throttles. Retries pile up, latency climbs, and user experience quietly degrades. This test will show us how quickly we hit write limits, whether adaptive capacity keeps up, and how retry behavior amplifies delays.

Lambda concurrency limits: every chat request triggers orchestration logic, and even if each execution is fast, concurrency multiplies fast when many users arrive together. We want to see when cold starts become visible, when concurrency limits get hit, and how throttling shows up at the API layer. Lambda failures are rarely catastrophic at first, they're slow, subtle, and confusing, exactly what we want to surface.

Bedrock inference pressure: LLMs introduce a new kind of scaling problem, they don't just cost compute, they cost tokens. Every additional user increases inference time, token usage, and billing exposure. This test is about performance, but it's also about financial sustainability.

Why cost is part of the load test

Traditional load testing stops at stability. We're going further, estimating what happens if this traffic were real. If 100 users are chatting concurrently, each generating multiple LLM responses, token count adds up fast. We'll use this data to answer questions like: how expensive is one active user per minute? What happens if engagement doubles? What does "going viral" actually cost? That directly informs rate limiting policies, token budgets, response verbosity, and feature gating. In an LLM-powered system, cost is a first-class metric.

What we'll do with the results

This isn't an academic exercise. Based on what we learn, we expect concrete changes. On infrastructure: adjusting DynamoDB capacity strategies, introducing write batching or queues, reserving Lambda concurrency, splitting synchronous and asynchronous paths. On product: limiting rapid consecutive messages, caching repeated questions, shortening default responses, introducing daily or per-session limits. Load testing informs product design just as much as architecture.

Why we're doing this before growth

Most teams run load tests when something already hurts. We're doing this because we want predictability, control, and confidence, we want to know where the system bends before it breaks, design safeguards intentionally rather than reactively, and make sure users experience reliability even when demand spikes unexpectedly.

Key takeaways

• We're going to load-test real user flows, not isolated API calls.
• Conversation latency is the metric that best reflects user experience.
• Serverless systems have limits, they just show up under concurrency.
• Latency spikes before errors, making it an early warning signal.
• LLM features must be tested for cost impact, not just stability.
• Breaking the system intentionally is safer than learning in production.