From Generic to Personal: Lessons from Building LLM-Powered Experiences

Most LLM personalisation demos look great. You pass in a profile, ask the model to "tailor this", and the first output reads as if someone who knows the user wrote it. Then you run it across a few dozen real profiles, read every result, and the picture changes. Some outputs have quietly fallen back to generic content. Others mention the user by name but could have been written for anyone.

We ran into both recently while generating personalised, pet-specific content. This post is about the gap between output that mentions a user and output that is actually about them. Most of what closed that gap was engineering decisions, not prompt edits.

THE PROBLEM: A FALLBACK LOOKS LIKE SUCCESS

Every production LLM feature needs a safety net. When generation fails, you serve something generic so nobody sees an empty screen.

The catch is that a fallback is a successful response. It returns a 200, renders fine and logs nothing alarming. In our early test runs, most profiles were getting the fallback, and none of the metrics we watched said so. We only found out by reading the outputs one profile at a time.

Four ordinary failures were hiding behind it:

• The model didn't have the facts. Asked to personalise on a specific attribute, it mostly ignored it. When it did use it, it sometimes made up claims. In health-adjacent content, a made up claim isn't a style problem. It's a safety problem.
• One malformed reply sank everything. Without strictly enforced structured output, a reviewing model sometimes answered in a looser format than we asked for. A strict parser read that as a full rejection and threw all the output away.
• A safety filter blocked the reviewer. Safety content has to talk about hazards. A filter in front of the model saw hazard words in an internal review call and blocked it, so the content that most needed review never got any.
• The deadline was shorter than the work. Generating, reviewing and repairing took longer than the timeout we'd set, so users who moved fast got the generic version.

Each of these is a small bug on its own. Together they turned a personalised feature into a generic one, and nobody could see it happening.

THE APPROACH: FACTS FROM CODE, WORDS FROM THE MODEL

The most important decision was which part of the job the model should own. Models are good at phrasing, tone and making a point land. They're unreliable with domain facts, and they sound just as confident when they're wrong.

So we moved the facts into data we curate and version. For each profile, code looks up the attributes that matter and turns them into a short, numbered list of grounded facts. The generator must build every item on one of those facts and say which one it used. The reviewer treats that list as the only facts it accepts. The curated data also carries one hard rule: it never describes anything unsafe as safe, so there's no dangerous claim in it for the model to borrow.

When the curated data doesn't cover a profile, the generator reasons only from general, observable attributes. It still can't make safety claims.

The second decision was to stop treating generation as one all-or-nothing call.

Three parts of this design carry most of the weight:

• Focused generators. A narrow prompt that covers one kind of fact writes sharper output than one long prompt that tries to cover everything. Running them in parallel keeps the extra latency small.
• Repair, don't retry. A rejected item no longer sinks the whole output. The next round regenerates only what's missing, for a fixed number of rounds. Generic content fills whatever is still empty after that, so a failure is partial instead of total.
• Code owns the format. Length limits, structure and ordering are checked in code, not just requested in the prompt. Anything that breaks a rule gets regenerated.

THE PROCESS: LESSONS FROM READING EVERY OUTPUT

Every change came from a test run on realistic profiles, followed by reading every output by hand. These lessons apply well beyond our use case.

Parse leniently, then ask again. Without enforced structured output, models drift from the format you asked for. We now accept reasonable variations, and re-ask only when a reply really is unusable. The strict parser never made the system safer. It just made it fail quietly.

Filter what users read, not your internal calls. Everything users see is still screened. The internal review step's verdicts never reach a user, so screening that call only blocked the review we needed most.

Temperature isn't variety. When outputs started to feel samey, the obvious move was to turn up the temperature. We made it a runtime setting and measured it instead. Very different temperatures gave almost the same amount of variety. Sampling wasn't the limit; the input was. The brief had a small set of examples that the model treated as a checklist, and the reviewer penalised anything outside it. Variety came from richer grounded inputs and from telling the model that examples are a reference, not a list to reword.

Steer by topic, block only copies. Content that comes back regularly has to avoid repeating itself. The obvious approach is to filter new items against history by topic. In a narrow domain, history covers most topics quickly. By the second session, topic filtering was rejecting almost everything new, and users fell back to generic content. What we actually wanted was this: a topic can come back, but the same item can't. So we split the job in two.

# illustrative
prompt.avoid = recent_items(profile)                  # steer: soft, in the prompt
kept = [d for d in drafts
        if not is_near_copy(d, recent_items(profile))]  # block: hard, in code

The same split worked for related profiles generated at the same time, such as several pets in one home. We generate those one after another, and each sees the earlier outputs as steering. Filtering those by topic was just as harmful.

Set deadlines by the work. Once generation included review and repair rounds, the old timeout was too short. We sized the deadline to the measured generation time, so moving fast no longer means getting the generic version.

RESULTS

In repeated test runs on realistic profiles:

• Personalised output went from a small minority of profiles to all of them.
• Items built on each profile's own attributes became the norm rather than the exception, and made up attribute claims stopped.
• Distinct items across consecutive sessions rose substantially, and generic filler in later sessions nearly disappeared.
• Checked runs showed no duplicates and no format violations.

These are deliberate test runs, not production metrics. They're small enough to read every output, and that's the point: none of these failures showed up in error rates.

TAKEAWAYS

• Your fallback hides your failure rate. Track how often users get the generic path, and read real outputs per user, not per prompt.
• Facts in code, phrasing in the model. If a wrong claim could hurt someone, it comes from data you curate. The model only gets to word it.
• Make failure granular. Rounds that regenerate only the missing pieces beat a pass/fail on the whole output.
• Keep steering separate from blocking. Use the prompt to nudge, and code to block only true copies. Blocking by topic starves the generator.
• Measure before you reach for a knob. Temperature looked like the fix for sameness. The real limit was the input.

Personalisation isn't a word you put in a prompt. It's a property you build into the system, one deterministic guarantee at a time.

ABOUT HOOMANELY

Hoomanely builds technology that helps pet parents keep their pets healthier, every day and not only at the vet. Short, personalised moments of learning are part of that mission: guidance shaped around your own pet teaches more than a generic leaflet ever will. Keeping that guidance accurate, with safety facts we curate rather than facts a model guesses, is how we earn the trust that preventive pet care depends on. The same discipline now shapes every place where we put generated content in front of a pet parent.