LLM ENGINEERING
RAG implementation services for systems that have to be right. Chunking, hybrid search, re-ranking, citation and permission-aware retrieval, tuned against a scored retrieval set - so answers come from your documents, cite where they came from, and never show someone what they should not see.
What retrieval-augmented generation actually is
Retrieval-augmented generation is a pattern where the system searches your own content for relevant passages and gives them to a language model as the basis for its answer, rather than relying on what the model absorbed in training. Done properly, the answer cites the passage it came from.
The pattern is simple and the engineering is not. Almost every disappointing RAG system fails at retrieval rather than generation: the right passage was never fetched, so the model answered from something adjacent and sounded confident doing it. Chunk boundaries split a table from its heading. Semantic search missed the document that used your internal acronym. Nothing re-ranked the twenty candidates, so the best one sat eleventh and never made it into the context.
So that is where we work. We build a scored retrieval set first - real questions with the passages that should have been returned - and tune against it. Retrieval quality is measurable, and measuring it is what separates a system that improves from one that gets adjusted.
You probably need this if you recognize these
- Your assistant answers confidently from a document that is three versions out of date.
- It cannot answer questions your documentation clearly does answer.
- Answers are right for common questions and wrong for the specific ones that matter.
- Nobody can tell where an answer came from.
- Different users should see different answers, and they do not.
- It works on clean documents and falls apart on the PDFs with tables.
The retrieval stack, component by component
A scored retrieval set
Real questions paired with the passages that should be returned. Built first, because it is what makes every later decision measurable.
Ingestion and parsing
Handling the formats you actually have: PDFs with tables, scanned documents, wiki exports, tickets, spreadsheets and the shared drive nobody has audited.
Chunking strategy
Boundaries that respect document structure, so a clause keeps its heading and a table keeps its column names. Tuned per document class rather than globally.
Hybrid search
Semantic and keyword retrieval together, because your internal terminology and product codes defeat pure embeddings.
Re-ranking
A second pass over candidates so the best passage reaches the context window rather than the merely nearby one.
Permission-aware retrieval
Access checked at query time against your identity system, so retrieval can never become a data-leak path.
Citations
Every answer traceable to its source passage and version, which is what makes the system auditable and trusted.
Freshness
Re-indexing on change, with staleness visible rather than assumed.
How a question becomes a grounded answer
RAG implementation and AI knowledge systems
Interpret
The question is expanded and, where useful, decomposed into sub-queries.
Self-healed - retried with fallback tool. Human not required.
Retrieve
Hybrid search runs across the index, filtered by the asker's permissions before anything is returned.
Re-rank
Candidates are scored again against the question, and the weakest are discarded.
Assemble
The context is built to a budget, preserving structure and provenance for each passage.
Answer
The model answers from the supplied passages and cites them, or states that it cannot.
Record
Question, retrieved passages, versions and answer are logged together for later diagnosis.
Systems that live or die on retrieval
Knowledge-Base Agent
The clearest case: its answer quality is retrieval quality, almost entirely.
Meeting Prep Agent
Retrieval across accounts, history and notes, with permissions that differ per user.
Compliance Watch Agent
Retrieval where a missed passage is the whole failure, and citation is mandatory.
Where your knowledge actually lives
The hard part is rarely the vector store. It is the permissions model and the formats - which is why [data engineering for AI](/services/data-engineering-for-ai/) often runs alongside this work.
- Document management
- Wikis and internal knowledge bases
- Ticketing and support history
- Code repositories
- Shared drives and object storage
- CRM notes
- Email archives
- Identity providers, for permissions
Platform names are shown as examples of the categories agents connect to. They are not partnerships or endorsements.
How retrieval is kept safe
Autonomy boundary
Retrieval returns only what the asker is entitled to see, checked at query time against your identity system rather than baked into the index.
Approval gates
Adding a new source to the index is a deliberate, reviewed step. Sources are not discovered and ingested automatically.
What stays human
Deciding which sources are authoritative, and resolving contradictions between them. The system surfaces the conflict rather than picking a winner.
Logging
Every query records who asked, what was retrieved, which versions and what was answered, which is what makes a leak investigable and an answer defensible.
How the work runs
Typical ranges from our engagement model (doc 04 §5), not a quote.
| Stage | Typical | What happens |
|---|---|---|
| Pilot | 3-10 days audit, then 2-4 weeks | Source inventory, a scored retrieval set built from real questions, and a measured baseline of whatever exists today. |
| Build | 3-8 weeks | Ingestion, chunking, hybrid search, re-ranking, permission filtering, citations and freshness. |
| Release | 1-2 weeks | Rolled out by audience, with retrieval scores and citation rates watched at each step. |
| Managed | ongoing, optional | Re-tuned as sources change, retrieval set extended with the questions that failed. |
What we measure
- Retrieval score
- Measured against your scored set before any generation quality is discussed (Target)
- 100%
- Of answers carrying a citation to a source and version (Target)
- Query-time
- Permissions enforced per request, never assumed from the index (Target)
- Your set
- The questions and expected passages you agree, which every change is scored against (Yours)
“ Until then these are design targets and measurement commitments, not results.
What this looks like in practice
Support triage hive
Reference scenario · SaaS. Three channels, one queue, answered from documentation with citations.
Reference scenario - a composite build illustrating our method. Figures are modeled and the model is shown.
Frequently asked questions
Should we use RAG or fine-tune a model?
Different problems. RAG supplies knowledge the model does not have and needs to cite; fine-tuning shapes behavior, format and style, or makes a smaller model competent at a narrow task. Knowledge that changes weekly belongs in retrieval, because retraining is a poor update mechanism. Most production systems use both, and fine-tuning and evaluation covers the other half.
Our documentation is a mess. Do we need to fix it first?
No, and waiting for that is how these projects stall for a year. We build against what you have, and the system reports which questions it could not answer - which is usually the most useful documentation backlog anyone has produced. Contradictory sources are surfaced rather than silently resolved, so you find out where the mess actually is.
How do you stop it showing people things they should not see?
Permissions are enforced at query time against your identity provider, before any passage reaches the model. Retrieval is one of the easiest ways to build an accidental data-leak path, because a single index can flatten access controls that took years to get right. We treat that as a security design problem, and every query is logged with who asked and what was returned.
Which vector database do you use?
Whichever fits the workload, and the choice matters far less than most vendor comparisons suggest. Chunking strategy, hybrid search and re-ranking move retrieval quality far more than the store does. We build behind an interface so the store can be changed without touching the rest of the system.
How do we know it is actually getting better?
Because retrieval is scored against a fixed set of real questions with known correct passages. Every change is measured against it, so improvement is a number rather than an impression. Without that set, tuning a RAG system is guesswork with confident-sounding output.
How current are the answers?
As current as the indexing schedule, which we set per source: near-real-time for tickets and prices, scheduled for documentation, on-change for anything with a publish event. Staleness is shown rather than hidden, because an answer from a superseded version is worse than no answer.
Related services
Fine-Tuning & Evaluation
The other half of the RAG-or-fine-tune decision.
Conversational AI & Voice Agents
Where grounded retrieval is most visible to a customer.
Data Engineering for AI
The pipelines and permissions underneath the index.