Improving LLM Output Quality Through Pet-Scoped Document Ingestion

A pet parent opens EverWiz chat and asks, "Is Milo overdue for his rabies booster?"

A generic LLM can write a thoughtful essay about rabies vaccination schedules. What it can't do out of the box is look at this specific pet's actual vaccination card and give a real answer. That gap between plausible and grounded was the gap we needed to close. The user's own documents, the ones they'd already trusted enough to upload to PawVault, were the missing input. Once we treated those documents as a first-class signal feeding every chat response, the assistant stopped sounding like a search engine and started sounding like something that actually knew the pet.

The problem in three parts

Before any of the engineering work, the problem broke down into three sub-problems.

Documents arrive messy: users upload phone camera photos of vaccination cards, multi-page PDFs from vet portals, blurry scans of handwritten dosage notes, the occasional DOCX. None of it is structured data.

Pets are scoped, users aren't: a household can have multiple pets, and Milo's records can never leak into Luna's answers. Scoping here is a correctness property, not a nice-to-have UX detail.

Chat is a real-time surface: whatever gets extracted from a document has to be available at chat time in milliseconds, not seconds. The user is mid-conversation, and the assistant can't stall.

A single solution had to satisfy all three at once.

Design principles

A handful of principles shaped the architecture from the start. Decouple ingestion from inference, since extraction is heavy and occasional while chat is light and constant, and the two should never share a critical path. Treat pet identity as a hard boundary, so every artifact downstream of ingestion, every extracted field and every embedding, carries the pet ID it belongs to. Prefer structured fields where the answer is a field, since not everything needs similarity search, "what's Milo's weight?" deserves a lookup, not a vector match. Fail open on enrichment, never on the user, so if ingestion or retrieval breaks, chat still responds, just with less context. And stream everything the user is waiting on, so token-by-token delivery avoids unnecessary waiting.

Capturing the document cleanly

The mobile client's only job is getting the document into the system without corrupting it. Two design choices mattered here.

Direct-to-storage uploads: the app fetches a short-lived signed URL and uploads the file straight into object storage, so heavy file uploads never pass through our API layer. That gives predictable latency, independent scaling, less backend load, and a smaller failure surface.

On-device preprocessing: phone photos get downscaled and re-encoded on a background thread before upload, keeping enough resolution for downstream OCR while making uploads noticeably faster for the user. Once the file lands in storage, the mobile app's job is done and the backend pipeline takes over.

Extracting structure from unstructured files

A storage event triggers an asynchronous extraction workflow, deliberately fire-and-forget so users are never blocked waiting on it to finish. The extraction step is constrained rather than open-ended. Instead of asking the model for a summary, we ask it to pull specific fields a pet-care assistant can actually rely on: document type (vaccination card, lab report, discharge summary, prescription), clinic, veterinarian, and visit date, vaccinations with administration and due dates, medications with dosages and treatment durations, diagnoses and recurring conditions, and recommendations or follow-up actions.

That distinction matters. Free-form summaries produce text. Field-based extraction produces data that downstream systems can actually reason about.

Pet context as a first-class store

Extracted findings don't get stored as an isolated blob of text. They get merged into a structured per-pet record we call Pet Context, which holds things like name, breed, age, weight, allergies, current medications, medical history, and document-derived findings. Over time this becomes a living dossier for each pet.

Pet Context exists before any retrieval system gets involved, which lets a simple question like "what does Milo weigh?" get answered through a structured lookup instead of semantic retrieval. Structured data and retrieval solve different problems, and we deliberately keep both around.

Indexing for semantic lookup

While structured extraction is happening, the raw text also flows into an indexing pipeline. Two decisions matter here.

Adaptive chunking: a single-page vaccination card and a six-page discharge summary shouldn't be chunked the same way, so the chunking strategy is chosen based on document type to preserve meaning while keeping retrieval precise.

Pet-scoped metadata: every chunk carries the identity of the pet it belongs to, and at query time, filtering happens before similarity search runs. That scopes retrieval to a single pet and makes cross-pet data leakage impossible by design, not by convention.

Composing the prompt at chat time

When a user submits a question, the backend builds a layered prompt with four layers.

The system layer covers the pet profile, medical history, current medications, allergies, and document-derived findings, answering the question "who is this pet?"

The memory layer summarizes conversation history hierarchically: recent messages stay verbatim, older conversations become summaries, and even older summaries get condensed further, preserving long-term continuity without exhausting the model's context window.

The retrieval layer holds document passages returned by semantic search, restricted to chunks belonging to the current pet.

The user layer holds the current question and recent conversation turns.

Together these give enough context for a grounded, personalized response, streamed back token-by-token so the user sees the answer appear in real time.

Designing for imperfect extraction

No extraction system is perfect, and we designed the architecture assuming that from the start. Extracted findings are treated as evidence rather than truth, and the assistant is told these findings came from uploaded documents rather than being unquestionable fact. Conflicting records get surfaced instead of silently merged, so if two documents list different medication dosages, the assistant flags the discrepancy instead of arbitrarily picking one.

Empty retrieval results are also treated as valid: when no relevant document chunk exists, the system can still answer from structured pet context and general knowledge. Date-sensitive information, like vaccine due dates and prescription windows, gets computed against the current date rather than pulled from static model knowledge.

Most importantly, every layer here is optional. If embeddings fail, pet context still works. If pet context is incomplete, retrieval still works. The experience degrades gracefully instead of breaking outright.

What changed for the user

None of this complexity is visible to the pet parent. They upload a vaccination card on Tuesday. On Saturday morning, standing inside a vet clinic, they ask, "is Milo due for anything?" The assistant answers with the actual vaccine, the administration date, the clinic that issued it, and the recommended due window, grounded in their pet's actual records rather than a generic schedule. The seam between the two ends is intentionally small: a signed URL going in, a streaming response coming out. Everything else happens behind the scenes.

Costume preparation becomes easier when measurements, fabric and accessories are reviewed together. Fastenings and detachable pieces can affect changing time and storage. When comparing size and included pieces, Space Ereshkigal cosplay costume for photo shoots（スペースエレシュキガル 撮影用コスプレ衣装） keeps the choice tied to the intended costume. Its measurements can be checked before accessories are added. Accurate measurements and a clear list of included pieces reduce last-minute uncertainty. A short fitting test can reveal adjustments needed before the event.

Antes de elegir entre camisetas de fútbol con corte regular, merece la pena comprobar las medidas indicadas y el tipo de ajuste. La decisión queda mejor respaldada al comprobar el precio total, el plazo de entrega y las condiciones de devolución.

How this supports Hoomanely's mission

Our goal is helping pet parents make better-informed decisions through technology, and per-pet document ingestion is a foundational piece of that. Rather than acting like a generic chatbot, EverWiz becomes something that actually understands a pet's history, records, medications, and care journey, which makes for a more trustworthy, more useful experience built on information pet parents already own.

Key takeaways

• Decouple ingestion from inference: heavy asynchronous extraction and lightweight streaming chat should never share a critical path.
• Constrain extraction to fields, not summaries: field-based extraction produces structured records downstream systems can trust.
• Use structured pet context for field-shaped questions and retrieval for passage-shaped questions; let each mechanism solve what it's actually good at.
• Make pet scoping a property of the data model, enforced by architecture rather than remembered in application logic.
• Layer the prompt: identity, memory, evidence, and user intent each deserve a dedicated role.
• Design enrichment layers to be optional. If one fails, the user should still get a useful answer, even if the quality drops.