Reading a Pet's Temperature Through a Thermal Overlay

Reading a Pet's Temperature Through a Thermal Overlay

By Vaishak C

A thermal camera can see a fever before a dog acts sick, but only if you know which warm pixels belong to the dog. A low-resolution thermal sensor and a normal camera look at the same scene from slightly different places, through different lenses, at different resolutions. Point them at a pet and you get two pictures that don't line up: the camera knows exactly where the animal's body is, and the thermal frame knows how hot things are, but neither alone can say "the dog's flank is 38.6°C." Our monitoring station fuses them, finding the pet in the sharp camera image, warping the fuzzy thermal frame onto it, and reading temperature at the right spot. The twist is that how you line them up depends on how close the pet is. This post is about that distance-aware registration.

The problem: two cameras that don't agree

Fusing a camera and a thermal sensor sounds like a one-time calibration: measure the fixed geometric relationship between them once, bake in a transform, done. It isn't, because of parallax.

The two sensors sit a few millimeters apart on the board. Like your two eyes, they see a near object from noticeably different angles but a far object from almost the same angle. So the pixel shift needed to align thermal onto camera changes with the subject's distance, a correction that's perfect for a dog sniffing the bowl is wrong for a dog standing back from it.

Two more complications stack on top. The camera uses a wide-angle lens that bends straight lines (fisheye distortion) and must be undistorted first. And the thermal frame is tiny, a coarse grid of temperatures versus a multi-megapixel image, so every camera pixel maps to a fractional thermal coordinate. Get any of this wrong and you don't just misplace the overlay, you read the temperature of the floor next to the dog and call it a fever.

The approach: let proximity drive the registration

The station has a third sensor that makes this tractable: a proximity sensor that reports the subject's distance in millimeters. Instead of a single fixed transform, we treat distance as the knob that tunes the fusion at the moment of capture.

The pipeline runs in stages. First, undistort the camera image with its calibration so geometry is trustworthy. Second, run pose segmentation to locate the pet's body and compute a centroid, the point we actually want a temperature for. Third, map that point into thermal space using the measured camera-to-thermal geometry, corrected for the current distance. Finally, sample the thermal frame around that point and report a temperature.

The distance-dependence shows up in three concrete places, and each is a small, explicit model rather than a magic constant. Distance sets how far the warm spot is expected to sit from the visual centroid, it sets how large a patch we average, and it rescales the parallax shift itself.

Three distance-aware rules

Rule 1: distance sets how far to look for the heat. The hottest part of a body isn't always exactly under the visual centroid, and that offset grows when the pet is very close, since more of its body fills the frame. We model the expected offset, in thermal pixels, as a base term plus a closeness term plus a body-size term:

prox_clamped = np.clip(prox_mm, PROX_MM_MIN, PROX_MM_MAX)

span = max(PROX_MM_MAX - PROX_MM_MIN, 1e-6)
proximity_norm = (prox_clamped - PROX_MM_MIN) / span  # 0 near threshold, 1 far
closeness = 1.0 - np.clip(proximity_norm, 0.0, 1.0)    # 1 when very close, 0 when far

d_exp_th = (
    K0_OFFSET
    + K1_PROX_TERM * closeness
    + K2_AREA_TERM * np.sqrt(mask_area_rgb) * scale_rgb_to_th
)

Note the unit discipline: the body-area term converts from camera pixels into thermal pixels via scale_rgb_to_th, because all of this reasoning happens in the thermal frame's coordinate system.

Rule 2: distance sets how big a patch to average. Sampling a single thermal pixel is noisy, but averaging too wide a patch bleeds in the background. So the sampling radius scales with distance, wider when the pet is close and fills the frame, tighter when it's far and small, with extra damping at very short range so we don't over-sample:

# Normalize so PROX_MM_MIN -> 0 and PROX_MM_MAX -> 1
norm = (prox_clamped - PROX_MM_MIN) / max(PROX_MM_MAX - PROX_MM_MIN, 1e-6)
radius = RADIUS_NEAR_TH - norm * (RADIUS_NEAR_TH - RADIUS_FAR_TH)

# Extra damping when we are closer than the threshold (very small working distance)
if prox_clamped <= CLOSE_PROX_THRESHOLD:
    close_span = max(CLOSE_PROX_THRESHOLD - PROX_MM_MIN, 1e-6)
    close_norm = (CLOSE_PROX_THRESHOLD - prox_clamped) / close_span
    radius -= close_norm * CLOSE_PROX_SHRINK

return float(np.clip(radius, RADIUS_FAR_TH, RADIUS_NEAR_TH))

Rule 3: distance rescales the parallax shift. This is the heart of the registration. A residual alignment shift measured at one depth must be rescaled for the current depth, because parallax is inversely proportional to distance. The code does exactly that, scaling the stored shift by the ratio of the calibration depth to the live depth:

new_depth_m = float(self.current_proximity_mm) / 1000.0
if new_depth_m > 1e-6:
    scale = float(stored_depth) / new_depth_m
    scaled_shift = (
        float(stored_shift[0]) * scale,
        float(stored_shift[1]) * scale,
    )

A shift calibrated at half a meter is doubled when the pet is at a quarter meter, and halved at a full meter. That single ratio keeps the thermal overlay locked to the body across the whole working range, instead of drifting off the animal as it moves toward or away from the station.

Reading the temperature. With the geometry resolved, the actual measurement is a lookup. A precomputed map turns each camera pixel in the region of interest into thermal coordinates; we round to integer thermal pixels, clip to bounds, gather the valid temperatures, and average them:

tx = np.rint(region_map_x[valid]).astype(int)
ty = np.rint(region_map_y[valid]).astype(int)
tx = np.clip(tx, 0, w_th - 1)
ty = np.clip(ty, 0, h_th - 1)

temps = self.thermal_data[ty, tx]
temps = temps[np.isfinite(temps)]
if temps.size == 0:
    return None
return float(np.mean(temps))

The isfinite filtering matters: bad or out-of-range thermal readings get dropped rather than averaged in, so a single garbage pixel can't fake a fever or hide one.

The results

The payoff is a temperature that stays pinned to the animal, not the scene, as the pet approaches and retreats. Because the offset, the sampling radius, and the parallax shift all flex with the proximity reading, a dog at the bowl and a dog a half-meter back both yield a temperature sampled from their actual body, with a patch size appropriate to how much of the frame they occupy.

It's also honest about uncertainty. Out-of-range thermal pixels are filtered, the sampling radius is clamped to a sensible band, and the whole thing degrades to "no reading" rather than a confidently wrong one when the geometry can't be resolved. That conservatism is exactly what a health metric needs, a missing temperature is recoverable, a fabricated one erodes trust in every reading after it.

Why it matters at Hoomanely

Hoomanely is reinventing healthcare for pets, replacing reactive, imprecise care with continuous, clinical-grade monitoring that catches problems early. Our devices form a Physical Intelligence ecosystem: sensors fused at the edge, feeding the Biosense AI Engine that turns raw signals into personalized, preventive insights.

Body temperature is one of the most clinically meaningful vitals we can capture passively, and capturing it without a rectal probe or a restrained, stressed animal is exactly the kind of problem our physical-intelligence approach exists to solve. But a thermal number is only useful if it provably came from the pet, which is why fusing camera, thermal, and proximity matters more than any one sensor. Registration is the measurement.

Key takeaways

Camera-thermal alignment is distance-dependent, not fixed. Parallax between two closely-spaced sensors changes with subject distance, so a single static transform drifts off the target. Make proximity a first-class input, using the measured distance to drive the expected hotspot offset, the sampling radius, and the parallax-shift rescaling. Work in one coordinate system, converting body-size and offset terms into thermal-pixel units with an explicit scale factor. Average a distance-appropriate patch, and filter it, since dropping non-finite thermal pixels means noise can't fake or mask a reading. And fail to "no reading," never to a wrong one, since a clamped, validated pipeline protects the integrity of the whole temperature trend.

Read more