Designing Audio Anomaly Detection for Pets: A Real‑World Engineering Blueprint

Knowing when a pet's mealtime "sounds wrong" seems like an easy problem, until you actually try to build it. Kitchens and living rooms are loud: background noise, human speech, clinking utensils, bowls sliding across the floor, humming fans, and every pet has its own eating style. Inside all that noise, we need to catch small anomalies: hesitation, discomfort, choking-like patterns, sudden stops, or gulps that are louder than usual.

At Hoomanely, audio plays a quiet supporting role in devices like EverSense and EverBowl. We don't talk about the hardware much, but it's doing real work behind the scenes, running edge-deployed models that have to hold up in kitchens, balconies, farms, and apartments, wherever a pet actually eats.

This post lays out the engineering behind a production audio anomaly detection pipeline for pets. The approach generalizes to most consumer IoT audio problems, but it's shaped by what we learned deploying real systems in real homes.

The real problem: audio is messy

Anomaly detection looks clean in a lab. In someone's kitchen, it becomes an exercise in survival engineering.

• Heterogeneous noise: fans, TV, speech, traffic, utensils.
• Pet variability: jaw size, eating speed, chewing rhythm.
• Room acoustics: echoey kitchens versus carpeted living rooms.
• Hardware limits: small microphones, low-power processors, small models.

The goal isn't eliminating noise. It's pulling out embeddings consistent enough to tell normal behavior from abnormal.

Our approach: a two-stage system

The blueprint that held up best in practice is a two-stage pipeline: an eating activity classifier that decides whether a sound is eating at all, and an anomaly scorer that learns what normal eating looks like for that specific pet. Plenty of systems fold both into one model, but keeping them separate makes debugging, personalization, and scaling much easier to reason about.

Stage 1: classifying eating vs non-eating

Step 1, audio to mel-spectrogram: short windows (roughly 20-40ms hop) get converted into 64-128 mel bins, giving us a time-frequency image.

Step 2, CNN for local acoustic patterns: eating sounds have bursts, crunches, and rhythmic texture, and a small CNN picks those up well while staying light enough for the edge.

Step 3, output is P(eating): only high-confidence windows move downstream.

Stage 2: embeddings and personalized anomaly detection

Once we've isolated eating windows, we embed them. Embeddings compress acoustic behavior into a vector that captures chewing force, frequency distribution, rhythm consistency, and crunch texture, which makes them well suited to anomaly detection.

Step 1: extract a 64-128 dimensional embedding z_t, using the last hidden layer of the CNN or a small GRU.

Step 2: build a pet-specific baseline. For each pet, we collect 10 to 20 normal meals and compute a mean vector and covariance, which defines "normal eating" for that pet.

score = (z_t - μ).T @ Σ^-1 @ (z_t - μ)  # Mahalanobis distance

Step 3: that gives a smooth anomaly estimate per window. Step 4: aggregate the per-window scores into a final meal score using percentile thresholding, max-pooling, or temporal smoothing, and use that to decide whether the meal was normal or unusual.

Why personalization matters

Dogs vary more than people do, sometimes by a lot. A large-breed dog can eat ten times louder and crunchier than a toy breed. A single global model against that range of variation produces frequent false alarms and erodes user trust fast. Personalization solves it, and it's a pattern we lean on across Hoomanely's pipelines, from thermal calibration to identity recognition to food volume estimation. Audio anomaly detection fits the same approach.

The full pipeline, end to end

On-device: preprocessing, eating detection, embedding extraction. Cloud: baseline storage, anomaly scoring, dashboards. App: alerts and timeline visualization. Keeping the heavy computation on-device protects privacy and keeps latency low, while cloud storage lets us track trends over weeks and months.

Practical considerations that actually matter

• Windowing strategy: short windows capture detail, long windows capture rhythm, and mixing both works well.
• Noise suppression: you don't need perfect denoising, you need consistency. Simple band-pass filters, log-mel compression, and per-window energy normalization go a long way.
• Bowl movement: dragging a bowl produces strong low-frequency noise. Treat it as its own class, or filter it out.
• When to alert: only alert on sustained anomalies, and hold off during the first two or three meals while the system is still personalizing.
• False positives hurt trust: anomaly detection needs to stay conservative, especially with pet parents watching closely.

How we use this internally

Inside Hoomanely, this pipeline supports early detection of unusual eating events: hesitation, rapid eating, sudden pauses. The blog keeps things generic on purpose, but internally we combine audio anomaly scores with EverSense's depth sensing and EverBowl's eating logs to build a multi-modal health profile. That feeds a broader goal: pet health systems that read behavior, nutrition, and wellness the way someone who knows the animal well would.

A simplified timeline

Window 0-3s: high P(eating), embeddings look normal. Window 3-6s: sudden silence, high anomaly score. Window 6-9s: weak chewing, moderate anomaly. Aggregate the three, apply the threshold, and the parent gets a notification.

Key takeaways

• Real-world anomaly detection is more about robustness than clever modeling.
• A two-stage approach (activity, then anomaly) simplifies the engineering considerably.
• Embeddings plus personalization get you high accuracy across very different pets.
• Careful windowing, normalization, and conservative alerting build trust.
• Combining audio with thermal and depth data produces a stronger health signal than any one sensor alone.