Transformer-Guided Noise Cancellation: AudioSep as Teacher, ConvTasNet as Student

Introduction

At Hoomanely, we only record audio when the proximity sensor detects a dog near the bowl, so the sound we capture is tied directly to feeding or drinking rather than hours of unrelated background noise.

Even in those few seconds, though, real homes are chaotic: TVs, conversation, clanking bowls, outside noise, room echo. Traditional DSP filters or Mel-band masking help somewhat, but they still let speech leak through and suppress the subtle transients we actually care about, like chewing spikes, micro-gulps, and lapping patterns.

That pushed us toward a two-stage, learning-based noise-cancellation pipeline: a transformer separation model called AudioSep acts as the teacher, and a lightweight ConvTasNet model becomes the student we deploy on-device. This post covers how we built it, why AudioSep beats handcrafted band-pass filters, and what it does for dog-sound understanding at Hoomanely overall.

Why classical filtering wasn't enough

Our early attempts combined high-pass filtering around 100 Hz, low-pass filtering around 5 to 6 kHz, and Mel-band suppression that reduced speech-dominant bins. These cut steady-state noise but failed against human speech mixed with chewing, TV dialogue overlapping mid-bands, sudden metal impacts, and reverberant bowls amplifying noise. Worse, these DSP methods often removed parts of the dog's actual eating signal, which confused our downstream classifiers. We needed a cleaner supervisory signal.

AudioSep: transformer-based separation with text queries

AudioSep is a transformer-based audio separation model that isolates sounds based on a natural-language query. Instead of telling it how to filter sound, we tell it what we want, like "dog chewing kibble," "dog drinking water," or "dog eating food," and it extracts only the audio matching that description.

Compared with DSP or Mel-band masks, AudioSep gives much cleaner separation of dog sounds from background speech, keeps the tiny transients like crunches and lapping spikes, doesn't depend on frequency overlap, and produces stable pseudo-labels across different rooms, mics, and bowls. Mel band-pass cares about frequency; AudioSep cares about meaning, which is exactly what makes it a good teacher model.

Building the dataset from AudioSep pseudo-labels

Training ConvTasNet needs paired data: noisy real bowl audio (X_noisy) and clean dog-only audio (Y_clean). We can't record true clean signals, so AudioSep generates the pseudo-ground truth instead.

We record proximity-triggered audio during eating or drinking, run AudioSep with a relevant query like "dog drinking water," and use its clean dog-only output as Y_clean. We then mix that clean audio with various noise samples, speech, fan hum, utensils, to create additional X_noisy variations, and each resulting pair becomes a training example. That gives us a large, high-quality dataset without hand-annotating anything.

We considered using Mel-filtered output as ground truth instead, but Mel filtering suppresses dog transients, speech leakage is unpredictable, and the resulting artifacts would confuse ConvTasNet. AudioSep avoids all three problems.

ConvTasNet: the lightweight student for on-device inference

Once we have clean pseudo-labels, we train ConvTasNet, which is fast, small enough for on-device inference, and good at separating non-stationary, transient sounds. Its architecture has a 1D convolutional encoder, stacks of dilated depthwise convolution blocks, and a 1D decoder that reconstructs clean waveforms. Dog chewing and lapping sounds are transient, broadband, rhythmic, and non-stationary, and ConvTasNet captures those patterns far better than a generic RNNoise-style denoiser.

Why AudioSep beats Mel band-pass filtering

Frequency overlap isn't a limitation for AudioSep. Speech overlaps heavily with dog chewing in linear and Mel frequency scales, which filters can't fully separate, but AudioSep works from semantic cues instead.

Transient preservation is much better. Mel suppression often kills the tiniest crunches and licks, the parts carrying the most useful information, while AudioSep preserves them almost perfectly.

Consistency across rooms and bowls holds up better too. Mel filtering shifts with microphone gain, room acoustics, and background volume, while AudioSep gives stable, reproducible output regardless.

And cleaner supervision means a better ConvTasNet: training on clean AudioSep signals lets the student model learn correct dog-sound patterns, suppress irrelevant noise, and generalize better on-device, which directly improves detection accuracy.

Results on real devices

After deploying the AudioSep-to-ConvTasNet pipeline, speech suppression improved by 10 to 12 dB, chewing detection accuracy went up by roughly 20%, drinking onset detection became noticeably more stable, background hum and utensil sounds dropped substantially, and dog transients came through much better preserved.

How this strengthens Hoomanely's work

We want to understand dogs at a deeper sensory level: what they eat, how fast they drink, how those patterns shift over time. Audio quality has a direct effect on how good those insights are. This pipeline helps us detect eating and drinking events accurately, catch anomalies earlier such as reduced appetite or gulping irregularities, build health signals we can trust, and run efficient models on edge hardware. Better sound leads to better insight, and better insight leads to healthier pets.

Key takeaways

• DSP and Mel filtering alone weren't enough for real home noise.
• AudioSep acts as a strong semantic teacher for separating dog sounds.
• ConvTasNet is a lightweight, accurate student model for on-device inference.
• AudioSep-generated pseudo-labels noticeably improve model quality.
• This pipeline directly strengthens Hoomanely's feeding-related health insights.