Adding AI Features to an Existing Product, Responsibly

Adding a language model to a product now takes a few lines of code. Doing it so that people can trust the result takes more thought. Our own hands-on work with LLMs so far is inside Cognix, our in-house trading research platform, where a scheduled Claude agent proposes research ideas. Most of our client automation has been scheduled jobs, rule engines and messaging. This guide sets out what that work has taught us about adding AI to software that already works.

Pick a task, not a technology

The best first AI features take something people already do slowly and make it faster: drafting a reply, summarising a long record, sorting incoming requests, pulling fields out of documents or suggesting tags. These tasks have three things in common. A person can check the output quickly, a wrong answer is cheap to catch, and the current manual process gives you real examples to test against.

Be wary of features where the model makes the final decision, especially about money, health, eligibility or anything a user cannot easily undo. Those can come later, if at all, once you have evidence of how the model behaves on your data.

Also ask whether you need a model at all. A clear rule, a search index or a scheduled job is often cheaper, faster and easier to explain. We regularly recommend one of those first.

Start outside the critical path

In Cognix, the language model does not run inside the application at all. Once a day, a scheduled, headless Claude run reads the context it is given and proposes one strategy idea as a ticket. That idea then goes through the same automated checks as any idea a person has: backtests that account for fees and slippage, an eight-stage validation process and a 14-day paper-trading probation. Only a person can promote a strategy to active, and most ideas are rejected along the way.

With that design, a bad suggestion costs some compute time and nothing else. It also builds a record of how useful the model's ideas really are before anyone relies on them.

In a typical business product, the same pattern looks like this: the model drafts and a person approves, or the model suggests and a rule checks. Users see AI output as a suggestion they can edit, not a result they have to accept.

Choose the simplest approach that works

There are three common ways to give a model what it needs, from least to most effort:

1. Prompting with context. Put the instructions and the relevant data for this one request into the prompt. This covers most first features, and it is where we start.
2. Retrieval. When the information you need is spread across many documents or records, search for the most relevant pieces first and send only those. It adds an index to maintain and its own failure modes, such as retrieving the wrong passage.
3. Fine-tuning. Training a model on your own examples. It is rarely the right first step: it is slower to iterate on and needs a large set of good examples.

Keep the integration thin: one module that builds the request, calls the provider and validates the response. That makes it straightforward to switch models or providers later as prices and capabilities change.

Build the test set before the prompt

An LLM feature without a test set is a demo. Before you settle the prompt, collect a few dozen real examples with the answer you would accept, including the awkward cases: empty input, very long input, other languages and attempts to make the model ignore its instructions. Run the whole set every time you change the prompt, the model or the code around it, and track the results like any other test.

Where the output has a structure, check it in code. Ask for JSON, validate it against a schema, and reject or retry anything that does not match. Free text that reaches users should still pass simple checks on length, required fields and content you never want to show.

Guardrails for data, cost and speed

• Send only what the task needs. Remove personal data where you can, and check the provider's data-retention and training terms before anything goes live.
• Treat model input as untrusted. Text from users, documents or web pages can contain instructions of its own. Do not give the model tools or permissions the task does not need.
• Set budgets. Cap tokens per request, requests per user and spend per month, and alert before a limit is reached, not after.
• Plan for latency. Model calls take seconds, not milliseconds. Stream the response, run slow work in the background, or prepare results on a schedule, as Cognix does.
• Log for review. Keep inputs, outputs and decisions where your team can audit them, and protect those logs like any other personal data.

Design for when it is wrong

The model will sometimes be wrong, so decide in advance what the user sees. Show AI output as a draft, make editing easy, say plainly that it was generated, and always keep the non-AI path available. If the provider is down or the budget is spent, the product should still work, just without the suggestion. A feature that fails silently or blocks the main flow does more harm than no feature at all.

Then measure whether it helps. Compare time spent, edit rates or acceptance rates before and after. If people rewrite most of what the model drafts, the feature is not ready, however good the demo looked.

Where to start

Pick one task, write down how you will judge the output, and build the smallest version that keeps a person in charge of the result. If you have a product and a task in mind, our AI and automation service explains how we scope this work, and the Cognix case study shows the idea agent and validation pipeline in more detail. We will also tell you if a simpler automation would do the job better, and you can start that conversation here.