Is your data ready for AI? A plain guide to RAG, fine-tuning and data quality

Your data is ready for AI if a new member of staff could do the job using only what you’d give the system. That means it’s written down, current, consistent and in a place a computer can reach. For most businesses that’s true of some data and not the rest. The work is finding which, before you spend money on a model.

We’d go further. The model is now the part you can buy and replace. What you collect, and what you do with it, is the thing a competitor can’t copy.

Why does your data matter more than the model?

Because the model is the part you don’t have to build. The language models behind ChatGPT, Copilot and the rest are already very capable. What they don’t have is your price list, your policies, your customer history or your way of doing things.

“The real value we’re providing… is the data that we collect and what we do with it.”

Every useful business AI system is a good model plus your data. If your data is out of date, contradictory or locked in a format nothing can read, the model will give confident answers built on it. That’s worse than no answer, because people believe it.

Models also change every few months. Your data stays. Money spent getting it in order carries over to whichever model you use next year.

Warm start or cold start?

We call this a warm start versus a cold start. The usual terms are using a pre-trained model versus training from scratch.

A cold start means training your own model from nothing. For a language model, that needs vast amounts of data, specialist staff and serious computing budget. Almost no SME should do it. Small models for forecasting or scoring, trained on your own records, are a different matter.

A warm start means taking a model that already exists and adding your data to it. It already reads, writes and reasons. You give it your knowledge by looking things up at answer time (RAG). If you also need it to behave differently, you can train it further (fine-tuning).

Nearly every project we see should be a warm start. RAG and fine-tuning are the two tools for it, and they do different jobs.

What is RAG?

RAG stands for retrieval-augmented generation. In plain terms, the system looks something up before it answers.

1. Someone asks a question, like “what’s our notice period for contractors?”
2. The system searches your documents and pulls out the few passages most likely to hold the answer.
3. It hands those passages to the model with the question.
4. The model writes an answer from those passages, and can show which document it used.

To make that search work, documents are usually split into chunks and turned into embeddings. An embedding is a list of numbers that captures what a passage means. It lets the system find passages about “notice periods” even when they say “termination”. Many systems also use ordinary keyword search, which is better for exact codes and names.

RAG is the right starting point for most businesses. It’s quick to update: change the document and the next answer changes. It can show its sources, so people can check. And your data isn’t trained into the model. It’s sent with each question, so the vendor still processes it, and your contract with them still matters.

What is fine-tuning?

Fine-tuning means training an existing model further on your own examples. You show it hundreds or thousands of “here’s the input, here’s the right output” pairs. It adjusts, and from then on it behaves more like your examples.

Fine-tuning is good at changing how a model behaves:

• writing in a very specific format or house style, every time
• classifying items into your own categories with high consistency
• doing a narrow task with a smaller, cheaper, faster model

It’s poor at teaching a model facts that change. If your prices change next month, a fine-tuned model still remembers the old ones until you retrain it.

It also learns whatever is in the examples, mistakes and personal data included. If a training example holds a customer’s details, the model may repeat them later. Keep personal data in documents you control, not inside a model you can’t easily edit.

RAG or fine-tuning? Usually that’s the wrong question

They get compared because both put your business into a model. But they fix different problems, and plenty of systems use both.

So the useful question is which gap you have. If answers are wrong or out of date, that’s a knowledge gap, and RAG fixes it. If answers are right but in the wrong format, tone or category, that’s a behaviour gap, and fine-tuning can fix it.

The two combine well. A model fine-tuned to write in your house format can still use RAG to answer from this week’s documents.

In practice, most first gaps are knowledge gaps. So we usually start with RAG, clear instructions and a few worked examples, then measure. We add fine-tuning only when a specific behaviour gap is still there.

Some systems need neither. Pulling fields out of invoices, or sorting emails into five buckets, can often be done with good instructions alone. Don’t buy complexity you haven’t proved you need.

Have you outgrown your spreadsheets?

This is the most common data problem we meet, and it’s rarely called a data problem. A growing business runs on hacked-together spreadsheets, WhatsApp groups and one person who knows where everything is. Nothing is in one place.

At The Home Help Company, the team spent too long on schedules, payroll and client messages by hand. The fix wasn’t AI. We moved the team from WhatsApp to Slack, digitised scheduling, and automated billing and PAYE. That saved the CEO 30 hours a month.

We’ve also spoken to a youth sports charity that had helped a thousand or more children. Its admin had outgrown the team. Records were scattered, which created safeguarding risk. It also meant the charity couldn’t show funders what their money had achieved.

Neither business needed a model first. They needed their data captured in one place, the same way, every time. Once that’s true, AI has something to work with.

What should you measure before you collect anything?

Decide the outcome first, then collect the data that tells you about it. It sounds obvious, but most businesses collect what’s easy instead.

Clicks are easy to count. Purchases, renewals and resolved complaints are what you actually care about. A system trained or tuned on clicks will get very good at clicks. If the goal is purchases, record purchases, and link them back to what led to them.

The same goes for judging the AI itself. Before you build, agree how you’ll score its answers. Measure how a person does today on the same examples, so you have a number to beat.

“If you don’t know the percentage, you were already in trouble.”

The target isn’t 100%. It’s better than the current process, on a measure you trust.

The data readiness checklist

Run this against the data your first AI project would use. Each “no” is work to do before, or as part of, the project.

Can a computer reach it?

• Is the data in a system with an export or an API, not just on paper or in someone’s inbox?
• Are documents in text form, or scanned images that need reading first?
• Do you know every place the data lives, including shared drives and WhatsApp groups?

Is it current?

• Is there one version of each document, or three with different dates?
• Is it clear which version is the live one?
• Does someone own keeping it up to date?

Is it consistent?

• Do the same things have the same names across systems?
• Are dates, amounts and categories recorded the same way?
• Would two staff fill in the same form the same way?

Is it complete?

• Is the knowledge written down, or does it live in someone’s head?
• Are there examples of good outcomes, not just inputs?
• Is the outcome you care about recorded, not just the activity around it?

Is it safe to use?

• Do you know what personal data is in it, and what lawful basis covers using it this way?
• Can you limit who sees what, so the AI doesn’t show one team another team’s records?
• Do you know where the AI provider will process and store it?

The last group overlaps with data protection law. Our guide to AI, GDPR and governance covers that properly.

How much data do you need?

Less than people think for RAG, more than people think for fine-tuning.

For a RAG assistant, the question isn’t volume. It’s coverage. If the answers to your team’s common questions are in the documents, a few dozen well-kept files can be enough. If half the answers live in email threads, a thousand documents won’t help.

For fine-tuning, you need many examples of the right output, checked by someone who knows. Even for narrow tasks, we’d plan in hundreds of examples, not dozens.

For forecasting, you need history that covers seasons, promotions and bad months, recorded the same way throughout. A year captured three different ways is closer to three short datasets than one long one.

What does a data readiness check look like in practice?

A good check is short and focused on one job.

1. Pick the job. One process, with a named owner and a number to improve.
2. Pull a sample. Fifty real cases, with the inputs and the outcome a person reached.
3. Trace the data. For each case, note where every piece of information came from and how fresh it was.
4. Try it. Run the sample through a quick prototype and compare its answers with the person’s.
5. List the gaps. Every wrong answer gets a reason, like a missing document, a stale version or an unclear rule.

The output is a short list of fixes, ranked by how many wrong answers each would remove. Sometimes you can build straight away. Sometimes the honest answer is three months of tidying first.

How do you fix data that isn’t ready?

Start small. You only need to clean the data the first project touches.

• Pick one source of truth for each kind of document, and archive the rest.
• Write down the missing knowledge. An hour with the person who knows is often worth more than any tool.
• Agree names and formats for the handful of fields the project depends on.
• Collect good examples as people do the job, so you can test the system against them.
• Build collection in early. For Ultra Goldfield Logistics, we built data capture into their driver app and customer portal from day one. Future machine learning on routes will have history to learn from.

Where does Databricks fit?

We’re a Databricks partner, so take this with that in mind. Databricks is a platform for storing and working with data in one governed place. Tables, files and documents sit together, and reporting, pipelines and AI run on the same copy.

It earns its place when:

• your data is spread across many systems and the useful answers need them joined up
• you have large volumes, or data arriving continuously
• you need real-time reporting and AI on the same data
• you need clear records of who accessed what, and where data came from

For Ymobile, the UK’s first eSIM-only mobile network, that was the situation. We built their data pipeline infrastructure in Terraform, which means the infrastructure is defined in code and can be rebuilt by re-running it. It feeds Databricks, giving the team real-time analytics on how customers use the network. Compliance was designed in from the start.

It doesn’t earn its place for a small team with one main database and a shared drive. Tidy the shared drive and start with RAG. You can move to a platform later, and the clean-up carries over.

When isn’t this worth doing?

• The job doesn’t depend on your data. Drafting generic emails needs a licence, not a data project.
• The data will never be kept up. If nobody will own it, an AI system on top will decay within months.
• You’d be cleaning data for a project you haven’t chosen. Pick the job first. Our AI readiness scorecard helps.

If you’re not sure what state your data is in, that’s a good first conversation. Talk to us, and we’ll look at what you have and tell you plainly what an AI project on top of it would need.