Motion Wakes the Mic: Fusing IMU and Audio on a Collar

Motion Wakes the Mic: Fusing IMU and Audio on a Collar

By Vaishak C

A barking dog tells you something. A lunging, barking dog tells you a lot more. On a battery-powered pet collar, the two most informative senses are motion and sound, but you can't run a microphone flat-out all day, and a motion sensor alone can't tell a bark from a yawn. So our tracker treats them as one system: the inertial sensor watches for movement continuously and cheaply, and when something happens, it wakes the microphone to record a short, context-rich clip. The two streams then get stitched back together by a shared timestamp so the backend sees a single event, the dog moved, and here's exactly what it sounded like. This post walks through how that motion-triggered, preroll-buffered, time-correlated audio capture actually works in firmware.

The problem: two senses, one tiny power budget

A collar's inertial measurement unit (IMU) is happy to run all day, it's low-power and its motion data feeds the pedometer regardless. A microphone is a different beast. Recording, compressing, storing, and shipping 48 kHz audio is expensive in CPU, flash, and radio time, so it can't be always-on.

The naive fixes both fail. Record continuously and you flatten the battery and drown the wireless link. Record only on a fixed timer and you miss the moments that matter while capturing hours of silence.

There's a subtler trap too. Even if you trigger audio on an event, a microphone that starts recording when it detects the event has already missed the event's beginning, the sharp onset of a bark is over in tens of milliseconds, long before any trigger fires. And once you have a clip and a motion record, you still have to prove they describe the same moment. Getting all of this right is what turns two raw sensors into one behavioral signal.

The approach: let motion be the trigger

The design principle is simple: the cheap sensor gates the expensive one. The IMU runs continuously and acts as the always-on sentinel; the microphone stays idle until motion arms it. That keeps average power low while ensuring the mic is recording whenever there's something worth hearing.

To make that robust, the motion detector can't just look at raw acceleration magnitude, a collar at rest still reads about 1 g of gravity, and a tilt isn't motion. Instead we track the peak-to-peak swing of the acceleration magnitude over a short rolling window, which captures dynamic movement while ignoring static orientation. A glitch guard then requires the threshold to hold across consecutive reads, so a single noisy IMU sample can't fire the mic on its own:

extern "C" bool accelorometer_is_moving(void)
{
    float p2p = accelorometer_motion_p2p();
    if (p2p < 0.0f) return false;
    /* Glitch guard: require the threshold to hold for >=2 consecutive reads so a
     * single duplicate/garbage IMU sample (the stream has occasional ones) can't
     * arm a clip on its own. is_moving() is polled ~once per audio block. */
    static int over = 0;
    if (p2p > ACCEL_MOTION_P2P_G) { if (over < 2) over++; }
    else                         { over = 0; }
    return over >= 2;
}

This single boolean, "is the dog moving," is the bridge between the two subsystems. The audio capture loop polls it roughly once per encoded audio block and arms a clip the moment it goes true, or when the hub forces a one-shot capture for a labeled dataset.

Preroll, arm, and correlate

Catching the onset with a preroll ring. To solve the "the bark is over before we trigger" problem, the encoder is always writing its most recent compressed audio blocks into a small circular look-back buffer, even while idle. When motion finally arms a clip, we flush that ring into the front of the recording, so the clip begins a couple of seconds before the trigger:

static int preroll_flush_to_clip(void)
{
    if (!s_preroll) return 0;
    int oldest = (s_preroll_head - s_preroll_count + PREROLL_BLOCKS) % PREROLL_BLOCKS;
    for (int i = 0; i < s_preroll_count; ++i) {
        clip_write_block(s_preroll[(oldest + i) % PREROLL_BLOCKS]);
    }
    return s_preroll_count;
}

The cost is tiny, a few kilobytes of heap holding pre-compressed blocks, and the payoff is that the onset that triggered capture sits at the head of the clip, not lost to trigger latency.

Arming and the shared correlation key. When motion (or a forced command) fires, the capture loop opens a clip and stamps a capture id (cid) derived from the wall-clock millisecond timestamp, the same clock the IMU stamps onto its own sample frames. It then tells the sensor-frame writer to tag every concurrent IMU frame with that id:

ESP_LOGI(TAG, "clip armed by motion (%.2f g) cid=%llu",
         motion_g, (unsigned long long)clip_id);
/* Bound the IMU tagging window to the requested duration: clear
 * the capture_id at arm + N s even if clip_close is delayed by
 * transfer contention, so the IMU tagged with this cid spans
 * exactly the N-second interval, not the record+transfer window. */
cid_deadline_ms = ts_ms + (uint64_t)(target_blocks / AUDIO_BLOCKS_PER_SEC) * 1000ull;
clip_arm_ms = ts_ms;
file_operations_set_capture_id(clip_id);  /* tag concurrent IMU frames */

From that moment, the IMU producer writes the cid into every motion frame it emits, so the backend can join the audio clip with the exact window of motion that accompanied it. Notice the discipline around the tagging window. The cid gets cleared after exactly the clip's intended duration, even if shipping the clip over the radio is delayed, so the motion tagged with a given clip spans precisely the recorded interval, never the longer record-plus-transfer window.

The results

The combined behavior is a collar that hears the right things at the right times. Average power stays low because the mic sleeps through stillness; clips are dense with signal because they only open on movement; and each clip arrives with its bark onset intact and a window of synchronized motion attached. A still-but-vocal dog is the one tricky case, which is exactly why the trigger was designed as a hybrid that can also arm on sound, motion is the primary gate, but the architecture leaves room for an acoustic onset.

Just as important, the fusion costs almost nothing. The bridge between subsystems is a single boolean and a 64-bit id; there's no shared buffer to lock, no second clock to reconcile, and no heavyweight message bus. One sensor polls a flag, the other stamps an integer, and the join happens later on a far larger machine. That's the kind of cheap, robust coupling a milliwatt budget demands.

Why it matters at Hoomanely

Behavior is where many health issues first surface, and behavior is inherently multimodal. Separation anxiety looks like restless motion and sounds like whining; pain can show up as a change in gait and a change in vocalization. A motion stream and an audio stream analyzed in isolation each tell half the story; correlated by a shared cid, they let the Biosense engine reason about a complete event, what the dog did and what it sounded like, in the same moment.

Key takeaways

Let the cheap sensor gate the expensive one, since an always-on IMU as a motion sentinel keeps the power-hungry microphone idle until there's something worth recording. Detect motion, not tilt, triggering on peak-to-peak swing over a short window with a consecutive-read glitch guard. Preroll captures the onset, since continuously buffering recent compressed audio and flushing it on trigger puts the bark's beginning at the head of the clip. Correlate with a shared key, since stamping audio clips and IMU frames with the same timestamp-derived cid lets the backend join the two streams into one event. And bound the tagging window, since clearing the correlation id after the clip's true duration keeps the fused motion window precise, independent of transfer delays.

Una comparación útil de camisetas de fútbol para el día de partido comienza por la temporada, el diseño y los detalles visibles. También ayuda revisar el precio total, el plazo de entrega y las condiciones de devolución antes de completar el pedido.

Read more