When the model meets a new dog: fine-tuning at enrollment time
The last post ended on an open problem: we could not make phone-photo galleries work well enough to matter. Adding more enrollment images made things worse, not better. This post is about what we changed and what finally moved the numbers.
The setup
Before getting into results, it helps to understand how we structured this experiment, because the split matters.
We trained a base model on 25 dogs using both phone and bowl camera crops. ResNet50 backbone, ArcFace loss, combined with the Zenodo DogFaceNet dataset (1,254 dog identities). Seven thousand seven hundred images total. Five dogs, Panda, Peanut, Penguin, Shadow, and Sophie, were held out completely. They appear in none of the training data. Not a single bowl crop, not a single phone photo.
At inference time, the goal is to simulate what actually happens when a new customer joins. They upload a handful of phone photos of each pet. The system builds a gallery from those photos. The bowl camera has never seen these dogs before. The model has never seen them either.
That is a harder problem than what we were testing in the previous experiments, where the model at least had some signal from the domain it was trained on. Holding five dogs out entirely forces us to ask whether the architecture can generalize to genuinely new identities.
What happened without fine-tuning
The base model, trained on Zenodo plus the 25 seen dogs, gets 85.1% identification precision on dogs it has trained on. That is reassuring. When it encounters the five unseen dogs, phone-gallery and bowl-query, it gets 14.3% precision. Out of seven queries it attempts to identify, only one is correct.
This is not surprising. The model has no identity representation for these dogs at all. It is doing its best to find the nearest neighbor in a gallery of total strangers, and the cross-domain gap between phone photos and bowl camera images makes that search harder.
Fine-tuning with only phone photos
Here is the experiment that changed things. After training the base model, we ran a second training pass with the five unseen dogs added in. Phone photos only, which is exactly what an owner would provide. We extended the ArcFace classification head from 1,279 to 1,284 classes and trained for 15 epochs, keeping the seen dogs' data in the mix so the model does not forget what it already knows.
The results were not gradual. By epoch 3, cross-domain accuracy for the enrolled dogs had jumped to 70%. It stabilized around 73% by epoch 6, which is where we took the best checkpoint.
After fine-tuning: 72.7% precision on the five unseen dogs. The 25 seen dogs moved from 85.1% to 88.7%. Fine-tuning on the new dogs did not hurt the existing ones.
That 72.7% figure is not perfect, but it is a completely different category of result than 14.3%. The model can now handle dogs it has never seen during base training, as long as they go through a fine-tune step at enrollment.
Embedding quality check
We ran a verification pass to see how clean the embeddings actually are, separate from the top-1 identification numbers.
For the five enrolled dogs (phone-only, cross-domain bowl queries), genuine pairs averaged a cosine similarity of 0.447. Impostor pairs averaged 0.106. That is a separation of 0.342. At a threshold of 0.26, verification accuracy comes out to 93.7%, with an EER of 6.6%.
For the 25 seen dogs, genuine pair similarity averaged 0.643, separation was 0.554, accuracy 91.5%.
Overall across all 30 dogs: 91.8% verification accuracy. DogFaceNet reports 92% on same-domain open-set verification. Our 91.8% is on cross-domain pairs, phone gallery against bowl queries, which is the harder version of the task. The gap between our numbers and theirs is basically noise at this point.
What this changes
The phone-gallery idea was not working because we were treating it as a gallery problem. More images, better averaging, smarter filtering. None of that was the issue.
The issue was that the model had no learned representation for a new dog at all. A gallery embedding built from phone photos of a dog the model has never trained on is a weak embedding. The geometry is right (ArcFace alignment, same preprocessing), but the model has not been asked to distinguish this dog from others, so the embedding space around it is not shaped correctly.
Fine-tuning fixes that. When we train even briefly on the five new dogs, the model learns where in embedding space these identities should sit relative to each other and to everyone else. Then the phone-to-bowl cross-domain gap is something the model can handle, because it has seen the training-domain version of these dogs.
The practical consequence is that phone-photo gallery building probably needs to include a fine-tune step as part of enrollment. Not a full retrain: 15 epochs took about 20 minutes on our hardware. But some targeted gradient update on the new dog's photos before deployment.
Where this fits in the product
The version of onboarding we wanted was: owner uploads photos, system builds gallery, done. This is still the right direction. But "builds gallery" now means two things: construct the gallery embeddings, and run a short fine-tune so the model actually knows how to place those dogs correctly.
The five unseen dogs in this experiment did not exist anywhere in training until fine-tune. Their bowl camera images were held out entirely. If 72.7% precision holds up on a larger unseen set, that changes what the cold-start experience looks like. A new customer would wait 20 minutes instead of days. That is a meaningful difference.
We are now running this on more unseen dogs to see if the result holds, and testing whether fewer phone photos per dog affects the fine-tune quality. More on that once the numbers are in.
This is the fourth post in our dog identification series. The first covered pipeline design, the second covered real-world results on bowl camera footage, and the third covered the pivot to phone-photo galleries.
