We Chose Cassandra for device_events. Here's What It Cost Us.

The workload, before the database.

Every smart bowl and collar in our fleet is a small, relentless firehose. A single device emits roughly twenty distinct event types: camera1 and camera2 frames, thermal and segmented images, audio clips, ocular temperature readings, plus a continuous drizzle of humidity, barometer, imu, proximity, weight, food_level, water_level, bowl_order, pedometer, and a gps fix every thirty seconds. None of it is ever updated after the fact. None of it is ever read outside the context of one specific device. And the read pattern is startlingly narrow - the dashboard wants this collar, last 24 hours; the nightly report wants this bowl, yesterday; the mission-control page wants this device, day by day, for two weeks. That's the entire access surface. Before choosing a datastore, it's worth writing that sentence down, because a workload this lopsided doesn't need a general-purpose database - it needs a specialized one.

Why Cassandra fits an append-only, partition-scoped workload.

Cassandra is built on an LSM-tree storage engine, which means a write is an append to an in-memory structure and a sequential flush to disk. There is no read-before-write, no page to locate and mutate, no index to rebalance. For a workload that is 100% inserts with zero updates, that's not a minor optimization - it's the difference between a write path that scales linearly with node count and one that eventually contends on a single primary. Just as importantly, Cassandra's data model forces you to declare your access pattern as the schema. Our device_events table is partitioned on device_id and clustered on (event_ts DESC, event_type, pet_id). Each device gets one wide row, physically sorted newest-first on disk. A query for a time window is a contiguous disk slice inside a single partition, served by a single replica set - no scatter-gather, no coordinator fan-out, no cross-node join. And because the clustering order already is descending time, the most common read in our system requires no sort at all.

The payoff shows up as an absence of code.

The clearest evidence that a schema fits is how boring the query layer becomes. Across a backend with dozens of endpoints - raw event feeds, pedometer aggregation, GPS trails, per-day capture counts, fleet-wide data-flow stats, the public showcase page - essentially every Cassandra read is the same statement: SELECT ... FROM device_events WHERE device_id = ? AND event_ts >= ? AND event_ts <= ?. Pagination is Cassandra's own opaque paging_state, base64-encoded and handed back to the client as a nextToken, so a cursor is a resumable position in a physical partition rather than an OFFSET that gets slower the deeper you scroll. There are no indexes to tune, no query planner to coax, no N+1 surprises. We also never issue a DELETE - soft deletes set an is_deleted flag instead, because in an LSM engine a delete is itself a write (a tombstone) that every subsequent read of that partition must skip past. A flag column costs one more write; a range full of tombstones costs every read forever.

What it cost us: the queries the partition key won't answer.

Here is the part most Cassandra posts leave out. A schema tuned for one access pattern is actively hostile to every other one. We cannot ask "show me all audio events across the fleet" - event_type is a clustering column after event_ts, so filtering on it without a device and a time bound means a full-table scan. We tried ALLOW FILTERING on event_type and it reliably timed out, so today we range-scan the time slice and filter event types in Python, pulling more rows off the wire than we keep. We also hit sharper edges once we moved to AWS Keyspaces: ORDER BY is rejected outright when range predicates are present, so we depend on the clustering order being correct rather than asking for it explicitly. And because a small limit combined with a high-volume imu stream would burn the entire result budget before a single low-volume audio row surfaced, filtering has to happen during the fetch loop, not after it. None of these are bugs. They are the bill for a schema that made the hot path free.

The honest answer: we don't use only Cassandra.

Cassandra owns the write path and the device-scoped read path, and nothing else. For the access patterns the partition key structurally cannot serve, we mirror events into DynamoDB with a global secondary index keyed the other way around - partition on event_type, sort on device_id - which is exactly the query Cassandra refuses. Log search and per-device activity histograms go to OpenSearch, because "find every ERROR in the last six hours across the fleet" is a text-and-aggregation problem, not a time-slice problem. Derived feeding events live in their own store. The pattern worth naming is this: Cassandra is not a replacement for a general-purpose database, it's a specialized layer you adopt for one shape of data and then deliberately route around. Teams that get burned are usually the ones that tried to make it the only store.

The decision rule.

So when is Cassandra the right call? Reach for it when your writes vastly outnumber your updates, when every read is naturally scoped to a single known key, when the data is time-ordered and you want it back in that order for free, when you need linear write scaling without a primary bottleneck, and when you're willing to pay for that with an inflexible query surface. Reject it when you need ad-hoc queries, joins, cross-partition aggregation, or transactions - because none of those get better with tuning; they were priced out at schema-design time. For device_events, the trade was obvious: our writes are append-only sensor data, and our reads are always this device, this window. That's not a database choice so much as a recognition that the workload had already made the choice for us.