When a sensor hub captures images, thermal frames, audio, and telemetry and pushes them over a 5 Mbps CAN FD bus, the software on the receiving end has exactly one job it cannot fail: don't drop frames. An image alone is thousands of CAN FD frames arriving back-to-back in a burst; miss a handful and the whole picture is corrupt. The classic mistake is to read a frame and process it in the same loop — decode, decompress, write to disk — because the moment processing takes a beat too long, the kernel's receive buffer overflows and frames vanish silently. Our hub's receiver is built around a different rule: receive as fast as the wire delivers, and process somewhere else. This post is about how that lossless receiver works on Linux.

The Problem: The Wire Doesn't Wait

CAN FD is fast and bursty. A single image transfer is a dense wall of frames with almost no gaps, and several data sources can burst at once. The receiving program has no way to slow the wire down at the frame level — the bytes are already arriving.

The naive architecture is a single loop: read one frame, handle it, repeat. It works in a demo and fails in production, because "handle it" is variable-latency. Decoding a message, touching the filesystem, or waiting on a lock can each take longer than the inter-frame gap. While the loop is busy, incoming frames pile up in the kernel's socket buffer, and once that fills, the kernel drops — quietly, with no error at the application layer.

So the receiver has two hard requirements. It must never block the read path on slow work, and it must have a way to tell the sender "ease off" before anything overflows. And when loss does become unavoidable, it must be counted, not hidden — a measured drop is a bug you can fix; a silent one is a mystery corrupting data downstream.

The Approach: Two Threads and a Big Ring

The design splits the work across two threads connected by one large in-memory ring buffer. A receive thread does nothing but pull frames off the socket and drop them into the ring as fast as possible. A processing thread pulls from the other end at its own pace, doing the slow decode-and-store work. The buffer absorbs the mismatch between a bursty wire and variable-latency processing.

The ring is deliberately huge — a hundred thousand frames — so a long image burst fits with room to spare. On top of it sits flow control: when the buffer crosses a high-water mark the receiver tells the sender to pause, and when it drains back below a low-water mark it says resume. That hysteresis gap keeps the sender from oscillating.

We also give the kernel a lot of slack. Before any threading matters, the raw socket itself is configured with large receive buffers so brief scheduling hiccups don't cause loss below the application:

450	    // Set socket receive buffer size for high throughput
451	    int socket_buffer_size = 16777216; // 16MB (doubled for maximum throughput)
452	    if (setsockopt(s_canSocket, SOL_SOCKET, SO_RCVBUF, &socket_buffer_size, sizeof(socket_buffer_size)) < 0) {
453	        perror("[CM4 CAN_HANDLER] setsockopt SO_RCVBUF");
454	    }

There are three layers of buffering here — the kernel socket buffer, the 100k-frame ring, and the flow-control back-pressure — and each catches a different timescale of burst. Together they turn a firehose into a stream the processor can sip.

The Process: Put, Get, and Push Back

The producer never waits. The receive thread's put operation adds a frame under a short lock and returns immediately. If the ring is genuinely full it increments a drop counter and moves on — it will not block the read path and let the kernel buffer overflow behind it. Loss is measured, loud, and rare:

116	    if (can_buffer.count >= CAN_BUFFER_SIZE) {
117	        can_buffer.buffer_overflows++;
118	        can_buffer.frames_dropped++;
119	        printf("[CM4 CAN_HANDLER] WARNING: Buffer overflow, dropping frame (total drops: %lu)\n", 
120	               (unsigned long)can_buffer.frames_dropped);
121	        pthread_mutex_unlock(&can_buffer.mutex);
122	        return false;
123	    }
124	    
125	    // Add frame to buffer
126	    can_buffer.frames[can_buffer.head] = *frame;
127	    can_buffer.head = (can_buffer.head + 1) % CAN_BUFFER_SIZE;
128	    can_buffer.count++;

Back-pressure fires from inside put. The instant the buffer crosses the high threshold, the receiver sends a pause command to the sender — before the ring is anywhere near full — so the burst is throttled at the source rather than dropped at the destination:

139	    // Flow control: send pause if buffer is getting full
140	    if (can_buffer.count >= FLOW_CONTROL_HIGH_THRESHOLD && !can_buffer.flow_control_paused) {
141	        pthread_mutex_unlock(&can_buffer.mutex);
142	        send_flow_control_pause();
143	        pthread_mutex_lock(&can_buffer.mutex);
144	    }
145	    
146	    pthread_cond_signal(&can_buffer.not_empty);

Notice the lock is released before sending the pause frame and reacquired after — sending on the socket must never happen while holding the buffer lock, or the two threads would serialize on it. The not_empty signal then wakes the processor if it was idle.

The consumer blocks politely and releases pressure. The processing thread's get waits on a condition variable when the ring is empty — burning zero CPU — and, symmetrically, sends the resume command once it has drained the buffer back below the low-water mark:

170	    // Flow control: send resume if buffer has space
171	    if (can_buffer.count <= FLOW_CONTROL_LOW_THRESHOLD && can_buffer.flow_control_paused) {
172	        pthread_mutex_unlock(&can_buffer.mutex);
173	        send_flow_control_resume();
174	        pthread_mutex_lock(&can_buffer.mutex);
175	    }

The gap between the 80%-full pause and the 20%-full resume is intentional hysteresis: without it, the buffer would hover at the threshold and flip pause/resume dozens of times a second. With it, the sender pauses once, the processor drains a big chunk, and the sender resumes once — smooth and predictable.

Duplicates get filtered cheaply. Bursty buses occasionally deliver the same frame twice. Rather than pay that cost downstream, the receiver drops a repeat of the same identifier seen within a short time window — while deliberately skipping this check for image frames, where the overhead isn't worth it:

321	        if ((current_id & 0x7FF) != 0x402) { // Skip duplicate detection for image frames
322	            long delta_us = (tv.tv_sec - last.last_time.tv_sec) * 1000000 + (tv.tv_usec - last.last_time.tv_usec);
323	            if (delta_us < 5000 && // 5ms duplicate window
324:                current_id == last.last_can_id &&

That one-line exemption for image traffic is the kind of pragmatic tuning production demands: dedup protects low-rate telemetry, but you don't want to run it on the thousands of back-to-back frames of an image where duplicates aren't the failure mode.

The Results

The combined effect is a receiver that keeps up with the bus under sustained bursts. The read path stays lean, the kernel buffer rarely fills because the ring drains it continuously, and when a genuine burst outpaces processing, the sender is throttled before anything is lost rather than after. Batch draining on the consumer side lets the processor pull many frames per lock acquisition, so the mutex is never the bottleneck.

And when loss is truly unavoidable, it shows up as a number — an incrementing drop counter with a warning — instead of a corrupt image nobody can explain. Alongside frames-received and frames-processed counts, that gives a clear, continuous picture of whether the receiver is healthy or falling behind. Boring, measurable, and honest: exactly what an always-on ingestion path should be.

Why It Matters at Hoomanely

Hoomanely is reinventing healthcare for pets — replacing reactive, imprecise care with continuous, clinical-grade monitoring that catches problems early. Our devices form a Physical Intelligence ecosystem: sensors fused at the edge, feeding the Biosense AI Engine that turns raw signals into personalized, preventive insights.

This CAN FD receiver is the spine of that ecosystem on the hub. Every image, every thermal frame, every burst of telemetry from the sensor nodes flows through it on the way to becoming a health insight — and a dropped frame there is a corrupted vital, a missing data point, a gap in the record the AI reasons over. Making the ingestion path lossless and self-throttling is what lets everything downstream trust that what it receives is what was sent.

Our guiding principle is that every signal matters and every detail counts. A firehose of sensor data is only useful if none of it leaks on the floor — and building the plumbing to guarantee that is quiet, essential work in the service of a pet's health.

Key Takeaways

• Separate receiving from processing. A dedicated read-only thread feeding a buffer keeps slow work off the critical path so the kernel socket never overflows.
• Buffer at multiple timescales. A large kernel socket buffer, a big application ring, and flow control each absorb a different burst duration.
• Push back before you drop. Sending a pause at a high-water mark throttles the sender at the source; dropping at the destination is the last resort, not the plan.
• Use hysteresis on flow control. A gap between pause and resume thresholds prevents rapid oscillation and keeps throughput smooth.
• Count every drop. A measured, logged overflow is a fixable bug; a silent one is corrupted data with no trail.

Author's Note

This receiver runs on the compute hub at the center of Hoomanely's Physical Intelligence ecosystem, ingesting the CAN FD traffic from its sensor nodes without losing a frame. It's the unglamorous counterpart to all the clever sensing: capture is only valuable if the data survives the trip to where it's understood. Building an ingestion path that stays lossless under a firehose — and tells you honestly when it can't — is how we keep every signal, from every device, on its way to better care.