A licensed assistant like Microsoft 365 Copilot is the right default when the answer already lives in your email, documents, and chats, and the person asking has permission to see it. But plenty of the highest-value questions do not fit that shape: answering from a thousand-page product manual, a library of contracts, or years of support tickets, with a citation to the exact source. For those, grounding your own AI in your own data usually beats another licence.
This is the heart of our AI services work, and we will tell you plainly when a licence you already own is the better answer.
What “grounding” means
Large language models are confident by default and wrong often enough to matter. Grounding, usually via retrieval-augmented generation (RAG), fixes this by retrieving the relevant passages from your own content first, then asking the model to answer using only those passages and to cite them. The result is an assistant that quotes your handbook rather than inventing one, and that can show its working.
The moving parts are fewer than the acronym suggests:
- A search index over your documents, built from chunks of text turned into vectors, so the system can find passages by meaning rather than keywords alone.
- A retrieval step that pulls the most relevant chunks for each question.
- A model, such as Azure OpenAI or Anthropic Claude, that answers from those chunks, with citations back to the source.
Why build one when Copilot exists
A grounded assistant earns its keep in specific situations:
- Your knowledge lives outside Microsoft 365, in a product database, a document management system, or a public website.
- You need tight control over exactly what the model may use, with an audit trail of sources.
- You want to expose it to customers, not just staff, where a per-user licence makes no sense.
- The task is narrow and repeated thousands of times, where accuracy and cost per answer matter more than breadth.
The parts people underestimate
The model is the easy part. The work that decides success is unglamorous:
- Data preparation: clean, current, well-structured source content. A grounded assistant over stale or contradictory documents will confidently cite the wrong version, the same oversharing risk we flagged in the Copilot readiness post.
- Permissions: if the underlying content has access rules, the assistant must honour them, not flatten them.
- Evaluation: a written set of real questions and correct answers, run on every change, so you know whether an “improvement” actually improved anything.
Cost and residency, honestly
Running costs are usually modest and pay-as-you-go: you pay per unit of text processed, plus a small hosting bill for the search index. For Australian organisations, the questions that matter are where the model runs and where the data sits. Azure OpenAI can be deployed in Australian regions, and enterprise model providers do not train their foundation models on your API data. Confirm the current terms against your own compliance needs, and document the data flows before you go live.
What to do this quarter
- Pick one narrow, high-value question your team answers repeatedly from a known body of documents, and make it the pilot.
- Assemble and clean that source content first, then stand up a small grounded prototype with citations.
- Write ten real questions with correct answers, and use them to judge the prototype honestly before spending further.
Grounded AI is not a bigger Copilot, it is a different tool for a different job: narrow, cited, and built on data you control. If you have a question your team answers the same way a hundred times a week, book a consultation.