Retrieval-augmented generation (RAG) is a way of making an AI chatbot answer from your own information instead of from what its language model happened to learn in training. Before the model writes a reply, the system searches your documents for the passages that match the customer's question and hands those passages to the model with the question. The model then answers from that context.
For a WhatsApp chatbot this is the difference between a bot that guesses and one that quotes your actual opening hours, prices and return policy.
Why WhatsApp chatbots get questions wrong
Traditional chatbots use rules or basic natural language understanding (NLU): they match a message to an intent someone set up in advance and send the scripted answer. They handle simple, repeated questions with clear intent. When a customer phrases something differently, asks two things at once, or asks about something nobody scripted, they fail, and the conversation goes to a person or stalls.
A large language model (LLM) on its own understands the phrasing, but its knowledge is fixed at the time it was trained. It does not know your stock, your prices or the policy you changed last week. Asked anyway, it can produce an answer that sounds right and is not. That is called a hallucination: plausible text that the model's training data or the provided context does not support.
RAG addresses both problems. The language model handles the phrasing, and retrieval supplies the facts.
How RAG works
The customer sends a question. For example, "Do you deliver to Johor on Sundays?"
The system searches your knowledge base. The question is turned into a numerical representation (an embedding) and compared with the passages in your documents to find the closest matches.
The best passages are added to the prompt. The model receives the question together with, say, the delivery policy and the Sunday schedule.
The model writes the reply from that context. Because the facts are in front of it, it does not have to rely on what it remembers from training.
Updating what the bot knows means updating the documents. Nothing has to be retrained.
The parts of a RAG system
The knowledge base
Everything the bot may answer from: FAQs, product information, policies, manuals, PDFs, web pages, and structured data such as a product catalog. Documents are split into smaller chunks so the search can return just the relevant part. Chunks that are too large dilute relevance, and chunks that are too small lose context.
The retriever
The retriever finds the chunks that match the question. Two methods are common, and many systems combine them:
Dense retrieval compares embeddings in a vector database, which finds passages with the same meaning even when the words differ ("closing time" and "opening hours").
Sparse retrieval matches keywords with methods such as TF-IDF or BM25, which is reliable for exact terms like product codes and names.
Combining the two is called hybrid search. The retriever decides what the model sees, so it decides how accurate the answer can be.
The generator
The generator is the language model. It reads the question and the retrieved passages and writes a reply in natural language. Because the passages are known, a RAG system can also show which source an answer came from, which makes answers easier to check.
Traditional chatbots, plain LLMs and RAG compared
| Feature | Rule-based chatbot | LLM on its own | RAG chatbot |
|---|---|---|---|
| Knowledge source | Scripted answers and intents | Training data, fixed at a cut-off date | Training data plus your documents, searched at answer time |
| Understanding varied phrasing | Poor outside the scripted intents | Good | Good |
| Accuracy on your business facts | Accurate only for what was scripted | Prone to hallucination | Grounded in the retrieved passages |
| Keeping up to date | Edit every script | Retrain the model | Update the documents |
| Showing sources | Not applicable | No | Possible |
What RAG improves
Fewer made-up answers. The model answers from retrieved facts instead of inventing them.
Current information. A change to a document reaches the next answer.
Answers specific to your business. The bot knows your products and policies without a custom-trained model.
Checkable answers. Answers can point back to the document they came from.
More languages. A capable model can answer in the customer's language from documents written in another.
Typical uses are customer service from product manuals and FAQs, internal help desks over company policies, and regulated fields such as healthcare, finance and legal work, where an answer needs to trace back to an approved source.
Setting up RAG for a WhatsApp chatbot
Collect the content. Gather FAQs, product information, policies, price lists and support documents. Remove outdated versions.
Index it. Split documents into chunks, create embeddings and store them in a vector database, with a keyword index alongside if you use hybrid search.
Build the retrieval step. Match each incoming WhatsApp message to the most relevant chunks, and include enough of the recent conversation that follow-up questions such as "and on Saturdays?" still make sense.
Connect the language model. Pass the retrieved chunks and the question to the model, with instructions to answer only from the context and to say when it does not know.
Review and iterate. Read real conversations, find wrong or missing answers, and fix the documents or the retrieval rather than the prompt alone.
Keeping a RAG chatbot accurate
Keep one current source per fact. Two documents with different prices will both be retrieved, and the model may pick the wrong one.
Write documents to be found. Clear headings, one topic per section and consistent formatting make retrieval more precise.
Tune the chunk size against real questions rather than leaving the default.
Watch response time. Customers on WhatsApp expect a reply in seconds, and each retrieval and model call adds delay.
Let the bot hand over. When the documents do not cover a question, a handover to a person is better than a guess.
How Volty uses your content
Volty answers from your business information, FAQs, files and websites. It sends replies itself or drafts them for your team to approve (Draft or Live per inbox), so you can read its answers before customers do while you check what your content covers.
Frequently asked questions
Is RAG the same as fine-tuning a language model?
No. Fine-tuning changes the model's weights by training it further on your data. RAG leaves the model as it is and gives it relevant passages from your documents with each question. The two can be combined: retrieval supplies current facts, fine-tuning adjusts tone and domain style. For a business whose prices and policies change, updating documents is usually cheaper and faster than retraining.
Can RAG be used with any large language model?
Yes, as long as the model can take extra context in its prompt. RAG is a method, not a model: the retrieved passages are added to the input and any capable model can answer from them.
What are the main limitations of RAG?
The answer can only be as good as what is retrieved. Outdated, missing or contradictory documents produce wrong answers, and a retriever that picks the wrong passages misleads the model. The retrieval step also adds some delay and running cost, and grounding in documents does not remove bias that is already in those documents.
How does RAG handle conflicting information in the knowledge base?
Not well on its own. The retriever ranks passages by relevance, so two documents with different prices can both reach the model. Re-ranking helps, but the reliable fix is to remove the outdated document and keep one current source for each fact.
What kinds of content can a RAG chatbot use?
Unstructured text such as FAQs, PDFs, policies and web pages, as well as structured data such as product catalogs and databases. What matters is that the content can be indexed and searched, and that it is kept up to date.
How should sensitive data be handled in a RAG chatbot on WhatsApp?
Keep personal data out of the knowledge base unless the bot needs it, redact what it does not need, and limit who can add or change documents. Customer conversations are still subject to data protection law such as the GDPR or Singapore's PDPA, so the same rules apply to anything the bot stores or retrieves.










