Beyond the Dropdown: Engineering a Seamless Onboarding

Onboarding friction kills conversion. When building Hoomanely's pet parent onboarding flow, we discovered that one seemingly simple question, what breed is your pet, can cause unexpected dropoff. Pet parents would pause, scroll through endless dropdown menus, type partial breed names that didn't match our taxonomy, or abandon the flow entirely. We needed a solution that respected the user's time while maintaining data accuracy.

The answer wasn't just adding autocomplete or better search. We eliminated the question altogether by predicting the breed from the pet's photo, a photo users were already uploading as part of onboarding.

Why breed selection matters

Pet breed isn't vanity metadata. At Hoomanely, breed information drives personalised recommendations, a Labrador's dietary needs differ from a Chihuahua's, and breed-specific health risks inform preventive care suggestions. But collecting this data through traditional UI patterns created problems. Decision paralysis, dropdowns with 200-plus dog breeds overwhelm users, and the American Kennel Club recognises 200 dog breeds alone, before you even add mixed breeds and regional variations. Input errors, users misspell names or use colloquial terms. Mixed breed ambiguity, users don't know which breed to prioritise. Mobile friction, scrolling and searching adds cognitive load when users are already uploading photos and entering other details.

The photo upload step was already mandatory, we needed it for the pet's profile. If we could extract breed information from that image, we'd eliminate an entire friction point while improving data quality.

Evaluating technical approaches

We evaluated three distinct approaches for image-based breed detection. Specialised CNN models, purpose-built networks trained specifically on pet breed datasets, achieve high accuracy on known breeds, 90%-plus on standardised datasets, with fast inference and small model size, but a rigid taxonomy where adding new breeds requires retraining, poor handling of mixed breeds, and no contextual reasoning.

Vision APIs from Google, AWS, or Microsoft offer zero model maintenance and handle diverse image qualities, but come with higher latency including network round-trip, per-request costs at scale, and inconsistent breed taxonomies across providers with no explanation capability when predictions seem wrong.

Multimodal LLMs like GPT-4V, Claude with vision, or Llama Vision can process images and generate structured responses in natural language, giving flexible outputs not constrained to predefined breed lists, handling ambiguity gracefully, working across species with a single model, and providing reasoning. The cons are higher latency, 1-2 seconds for full inference, more expensive per request, non-deterministic outputs requiring parsing, and larger infrastructure requirements.

Why we chose Llama Vision

The decision wasn't about raw performance metrics, it was about product fit and operational flexibility. Pure classification models output a breed label. Llama Vision outputs reasoning, "this appears to be a Golden Retriever mix, possibly with Labrador, based on coat texture and ear shape." For edge cases and user trust, that matters. When users see "Golden Retriever / Lab Mix," they understand the system's logic, a dropdown pre-filled with the highest-confidence CNN class feels presumptuous and wrong.

Multimodal LLMs handle uncertainty naturally, when encountering an unusual breed or poor photo quality, the model can respond with "unable to determine specific breed, appears to be a medium-sized terrier mix," preserving the onboarding flow while signaling data quality issues. Specialised models either force a low-confidence guess or fail entirely. One model handles breed prediction and edge cases, we don't maintain separate pipelines or retraining schedules, prompt engineering lets us refine outputs without touching model weights.

Our onboarding flow isn't latency-critical, users upload a photo then continue entering their pet's name, age, and medical history, actions taking 30-60 seconds. A 2-second breed prediction happening asynchronously doesn't block the user, we display a tasteful loading state while the prediction runs in the background. Running Llama Vision on AWS proves cheaper than Cloud Vision APIs at scale, avoiding vendor lock-in and external API dependencies.

This choice wasn't without compromise. The latency ceiling means LLM inference won't match CNN speeds, so applications requiring sub-200ms responses need a different approach. Output parsing overhead means LLMs return natural language requiring regex or structured prompting, mitigated with schema-enforced JSON mode but adding complexity. Non-determinism means the same image might occasionally produce slightly different outputs. And infrastructure requirements mean running 11B parameter models needs GPU instances, a higher entry barrier than calling a Vision API for startups without ML infrastructure.

Key takeaways

• Choosing between specialised models and multimodal LLMs isn't about which is best, it's about which trade-offs align with your product constraints.
• Use CNNs when latency is critical, your taxonomy is fixed, and you need deterministic outputs.
• Use Cloud Vision APIs when you lack ML infrastructure and volume is low-to-medium.
• Use multimodal LLMs when you need flexible, evolving outputs, edge case handling matters more than raw speed, and natural language reasoning improves UX.
• For Hoomanely, eliminating onboarding friction while maintaining breed data quality required a solution that understood ambiguity and communicated reasoning.