Pet facial recognition: how a camera knows which dog is at the bowl
Pet facial recognition identifies an individual dog, not a breed, by measuring the geometry of its face and comparing that measurement against a stored profile. In our bowl-mounted camera system, a frame passes through seven preprocessing steps, a 46-point landmark model aligns the face, an embedding model turns the aligned crop into a vector, and the vector is matched against a gallery of known pets using cosine similarity with a threshold of 0.50. On 149 manually reviewed frames from four days of real home footage, 100 frames produced a match and 94 of those 100 were the right dog.
The reason this matters in a kitchen rather than a lab is simple: in a home with two or more dogs, any measurement taken at a shared bowl is useless unless you know which dog was there. This post explains how recognition works, where it fails, and how a new dog gets added without retraining anything.
The short version
- Pet facial recognition is identity matching, not classification: the system compares a face embedding to stored profiles, so new pets can be added in seconds without retraining the model.
- The pipeline has seven preprocessing steps that reject blurry, dark, profile and partial frames before the expensive models run.
- On real footage, 67% of frames produced a match and 94% of those matches were correct; the misses were mostly framing, distance and unenrolled dogs, not model confusion.
- Enrolling a new dog from phone photos requires a short fine-tune step; without it, precision on unseen dogs was 14.3%, and with it 72.7%.
Why recognition, not classification
The obvious first design is a classifier: train a model on photos of the household’s dogs and have it output a label. It breaks the moment the household changes. A classifier needs a fixed set of identities at training time, and real homes add puppies, foster dogs and visiting relatives’ pets. Retraining a model every time that happens is not a product.
Face recognition sidesteps this. The model is trained once to produce embeddings, numeric vectors in which the same dog’s faces land close together and different dogs land far apart. Each known pet is stored as a single mean vector in a gallery. Adding a pet means embedding a few of its face crops and appending the average to the gallery file. Our post on building the multi-stage vision pipeline explains why we rejected the classifier approach and what the three stages (training, gallery building, inference) share.
How the pipeline works, step by step
Dog face recognition is not a solved problem the way human face recognition is. There is no dog equivalent of FaceNet with comparable quality, and public datasets do not look like the frames an overhead camera produces: fisheye distortion, dogs inches from the lens, low-contrast lighting. Most of the pipeline had to be built for those conditions.
- Quality gate. Each frame is rejected if its Laplacian variance (a sharpness proxy) is below threshold, or if mean brightness is too dark or blown out. This removes most night and fast-motion frames before any model runs.
- Dog detection. YOLOv8-nano, pretrained on COCO, finds the dog’s bounding box. The highest-confidence detection is kept and padded by 15% so later models see the full head.
- Eye gate. A fast eye segmentation model checks for a geometrically plausible pair of eyes: horizontally aligned, at least 15 pixels apart, both in the upper half of the crop. Frames without a valid pair are skipped before the slower landmark model runs.
- Landmark detection. A 46-point TFLite model runs at 384 by 384 resolution and returns normalized coordinates across four groups: outer head and ear periphery, cheek and face boundary, eyes, and nose and muzzle.
- Structure check. A rotation-invariant check confirms the landmarks form a coherent face before alignment.
- Alignment and cropping. The face is rotated and scaled so that eyes and nose land in fixed positions, which removes pose variation before embedding.
- Embedding and matching. The aligned crop goes through the embedding model, and the resulting vector is compared against every gallery entry by cosine similarity. A match is returned only if the best similarity is at least 0.50.
The embedding model is a ResNet50 backbone trained with ArcFace loss on our own labeled crops combined with the public DogFaceNet dataset, which adds 1,254 dog identities and improves general discriminability. The landmark model’s accuracy indoors depends on having been trained on imperfect frames; the post on reducing false detections by training on bad indoor data describes why clean public datasets produced eyes detected on fur and noses on bowls until failure cases were added to training.
What happened on real footage
We evaluated 149 frames captured on June 19, 20, 22 and 23, 2026, and reviewed every detection by hand.
Two things in that table deserve a closer look. The average cosine similarity on correct matches was 0.82, well above the 0.50 threshold, which means the model has margin on dogs it knows. And the largest failure bucket, dog not detected, is a framing problem: the dog is too far from the bowl, at a sharp angle, or only partly in frame. The general-purpose COCO detector was never trained on an overhead bowl view, and training our own detector on frames from our own cameras is the most direct fix.
The “no gallery match” bucket is partly correct behavior. One dog, Sophie, visited the bowl on several days but had never been enrolled, and the system returned nothing rather than guessing. A recognition system that says “I do not know this dog” is doing its job. The full breakdown, including the six wrong matches, is in dog identification in the real world.
Adding a new dog without retraining
The gallery design means enrollment is cheap, but the source of the enrollment photos matters more than expected. Galleries built from bowl camera frames work well against bowl camera queries. Galleries built from an owner’s phone photos face a domain gap: phone photos are taken at human height with different lighting and depth of field, and a model trained on overhead crops does not embed them the same way. Our post on rethinking the dog identity gallery for multi-pet homes measures that gap and the pipeline changes it forced.
The fix was not more photos. It was a short fine-tune at enrollment. Holding five dogs out of training entirely, the base model identified them from phone-photo galleries with 14.3% precision. After a 15-epoch fine-tune that added those five dogs (phone photos only, with the existing dogs kept in the mix), precision on the five rose to 72.7%, and the 25 previously seen dogs improved from 85.1% to 88.7%. Verification accuracy across all 30 dogs reached 91.8% on the harder cross-domain task. The method and the embedding separation numbers are in fine-tuning at enrollment time.
Where facial recognition for pets still falls short
Profile views fail by design. The eye gate rejects frames where only one eye is visible, because profile crops do not align well and produce unreliable embeddings. A separate profile recognition path would be needed to recover them.
Similar-looking dogs of the same breed are the hardest case for any embedding model, and per-pet threshold calibration is the direction we are exploring instead of lowering the global threshold, which would trade missed matches for wrong ones.
Detection rate varies by day. On the four evaluation days it ranged from 56% to 79%, which reflects how dogs approached the bowl on those days rather than model drift. Any published accuracy number for pet facial recognition should state the detection rate alongside it.
What this means for EverBowl™
EverBowl™ learns how a dog eats and drinks over weeks and months, and that record only has meaning if every visit is attributed to the right dog. In a multi-dog home an owner cannot stand by the bowl all day, and memory of who ate what blurs by evening. Facial recognition at the bowl is what lets EverBowl understand each dog separately in a shared kitchen, so a change in one dog’s pattern is not hidden by the other dog’s normal day. It does not diagnose anything; the value is the record, which gives the vet the date a change started and how it has moved since, attributed to the right dog. The honest limit is in the numbers above: a third of frames produce no identification, and the system returns nothing rather than a guess.
Conclusion
Pet facial recognition works when it is treated as identity matching with strict gates, not as a classifier. Reject bad frames early, align faces on landmarks, embed with a metric-learning model, and match against a gallery with a fixed threshold. Measure detection rate and match accuracy separately, and expect enrollment from phone photos to need a fine-tune step. The next improvement in our pipeline is a dog detector trained on our own overhead footage, which targets the largest remaining failure bucket.
FAQs
How does pet facial recognition work?
A camera frame is filtered for quality, the dog is detected, facial landmarks are located and the face is aligned to a standard pose. An embedding model converts the aligned face into a vector, and that vector is compared against stored vectors for known pets. If the closest match exceeds a similarity threshold, the pet is identified.
How accurate is dog facial recognition?
In our evaluation on 149 real home frames, 67% produced a match and 94% of those matches were correct. Accuracy on detected frames and detection rate are different numbers, and both should be reported.
Can it tell apart two dogs of the same breed?
Usually, because the embedding captures individual facial geometry rather than breed features, but same-breed look-alikes are the hardest case. Per-pet thresholds and more enrollment images from the bowl camera improve separation.
Does adding a new dog require retraining the model?
Adding a dog to the gallery takes seconds and needs no retraining. Enrolling from phone photos, however, works far better with a short fine-tune step: in our test, precision on unseen dogs rose from 14.3% to 72.7% after fine-tuning.
What happens when an unknown dog uses the bowl?
The best similarity falls below the 0.50 threshold and the system returns no identity. Returning nothing is the intended behavior; it avoids attributing one dog’s visit to another.
Is the recognition processed on the device or in the cloud?
The landmark model is a TFLite model built to run on the camera node, and the quality and eye gates run before any heavier model so most frames are rejected cheaply. Our post on edge AI cameras covers the on-device design in detail.
