Retrieval-augmented generation, usually shortened to RAG, is a way of making an AI model answer from your information instead of from memory. Before the model writes a reply, a search step finds the most relevant passages in your documents, such as your shipping policy, your size guide or your product specs, and hands them to the model with the question.

You will meet RAG mostly as a buyer. Help desk tools, website chat widgets and "chat with your documents" apps use it to answer customer questions. Knowing how it works helps you judge those tools, prepare your content so the answers are right, and spot the vendors who promise more than the technique can deliver.

What is RAG?

A large language model on its own writes from what it learned during training. It has never seen your return policy, so if a customer asks "can I return a sale item?", it guesses. RAG fixes this by adding two steps before the answer: retrieval, which finds the relevant parts of your own content, and augmentation, which inserts those parts into the prompt. The generation step then writes a reply based on them.

A simple way to picture it: without RAG, the model takes a closed-book exam. With RAG, it takes an open-book exam, and someone has already bookmarked the right pages.

What RAG is not:

  • Not training. Your documents do not change the model itself. They are looked up at question time, which means you can update a policy and the next answer reflects it immediately.
  • Not fine-tuning. Fine-tuning retrains a model on examples to change its style or skills. RAG changes what it knows for one answer. For factual support questions, RAG is usually cheaper and easier to keep current.
  • Not a guarantee of accuracy. If the search step pulls the wrong passage, or your documents contradict each other, the answer can still be wrong.

Related vocabulary: an embedding is a list of numbers that represents the meaning of a piece of text; a vector database stores those embeddings and finds similar ones quickly; a chunk is a short passage cut from a longer document; grounding means tying an answer to a source. Frameworks such as LangChain provide ready-made building blocks for developers who assemble RAG systems.

Why it matters

RAG is what separates a useful AI support tool from a risky one. A chatbot that answers from general knowledge will invent a 30-day return window when yours is 14 days. A RAG-based tool quotes your 14 days, and good ones show the source.

Take a skincare brand that receives 400 customer messages a month. About half are repeat questions with answers already written somewhere: delivery times, ingredients, how to use a serum, how to apply a discount code. Answering each one takes about 6 minutes, so those 200 messages cost 20 hours a month.

The brand connects a help desk tool that uses RAG over its FAQ, product pages and policies. The tool resolves 150 of those 200 messages without a human, and many such tools charge around $1 per resolved conversation. That is $150 a month to save 15 hours, or $10 per hour. The other 50 messages still go to a person, often with a suggested draft. Whether that trade is worth it depends on how you value your time and how often the tool gets things wrong.

The quality of your documents now drives the quality of your support. A vague shipping page produces vague answers.

How it works

A typical RAG system runs in two phases.

The preparation phase happens once, and again whenever your content changes:

  • Collect the sources. FAQ pages, policies, product descriptions, PDFs, past support replies.
  • Split them into chunks of a few hundred words, so each piece covers one topic.
  • Turn each chunk into an embedding with an embedding model.
  • Store the embeddings in a vector database or a search index, with a link back to the original text.

The answer phase happens every time someone asks a question:

  • Embed the question with the same model.
  • Retrieve the closest chunks, usually the top 3 to 10. Many systems combine this meaning-based search with classic keyword search to catch exact terms like a SKU or a product name.
  • Build the prompt: instructions, the retrieved chunks and the question, often with a rule such as "answer only from the sources below, and say you don't know otherwise".
  • Generate the answer and, in good tools, show which sources were used.

The simplest form of RAG needs no developer at all. Uploading your policies to a ChatGPT project or a note tool that answers from your files is a small RAG system run for you.

Benchmarks and examples

Rough reference points:

  • Resolution rates claimed by AI support tools usually fall between 30% and 70% of incoming conversations. Stores with clear, complete FAQ content land at the high end, stores with thin documentation at the low end.
  • Pricing is either per seat, per resolved conversation (often $0.50 to $1.50) or included in a higher help desk plan.
  • Preparation time for a small store is a few hours: cleaning up the FAQ, removing outdated policies, and writing answers to the top 20 questions if they do not exist yet.
  • Freshness depends on how often the tool re-reads your content. Daily is common. Check this before you change a policy.

Typical situations: a creator selling a $149 course uses a chat widget grounded in the course outline and refund terms, so prospects get answers at 2 a.m. A small furniture store feeds its assembly PDFs to a tool that answers "which screws go where" questions. A seller with 1,500 products uses RAG to answer spec questions from the product catalog, which works far better than a generic chatbot on questions like "is the large tote waterproof?".

Common mistakes

  • Feeding outdated or conflicting documents. If an old policy says 30 days and the new one says 14, the tool may quote either.
  • Assuming the tool knows live data. Most RAG setups search documents, not your live stock or order status, unless the vendor connects them specifically.
  • Skipping the "I don't know" rule. Without it, the model fills gaps with plausible inventions.
  • Never reading the transcripts. The first month of conversations shows exactly which answers are wrong and which documents are missing.
  • Letting the bot make commitments. Refunds, exceptions and compensation should stay with a person.

Best practices

  • Write one clear source per topic. A single shipping page with zones, prices and delays beats three partial pages.
  • Use plain question-and-answer format. FAQ entries phrased the way customers ask retrieve better than long paragraphs.
  • Include exact names and codes. Product names, variant names and SKUs in your content help keyword matching.
  • Demand source citations. Pick tools that show which document an answer came from, so you and the customer can check.
  • Set a clear handoff. Define topics that always go to a human, such as damaged items, refunds and complaints.
  • Review a sample every week. Read 20 conversations, fix the documents behind the wrong answers, and repeat.

In Roctify

Roctify does not include an AI assistant or a RAG chatbot. What Roctify gives you is the raw material a RAG tool needs: clear product descriptions, variants and SKUs in one shared catalog. Add your shipping, returns and common questions to a document you keep up to date, and any external AI support tool you choose has good sources to draw from. Pro plan exports give you a clean product list if a tool asks you to upload your catalog.

Customer emails can be handled in Roctify's email inbox (Creator plan and up), where you can paste replies drafted with an AI tool after checking them. If you need a custom connection between your store data and an outside system, that kind of integration is discussed on the Enterprise plan.

FAQ

Do I need a developer to use RAG?

Not for most small stores. Many help desk and chat tools include RAG and only ask you to point them at your FAQ pages or upload documents. A developer becomes useful when you want to connect live data, such as stock or order status, or build a custom assistant.

Is RAG better than fine-tuning a model on my products?

For factual questions about policies and products, usually yes. RAG is cheaper, updates as soon as your documents change, and can cite its sources. Fine-tuning is better suited to teaching a model a style or a format, not to keeping facts current.

Can a RAG chatbot still give wrong answers?

Yes. It can retrieve the wrong passage, misread a correct one, or answer when no source covers the question. Clear documents, an explicit "say you don't know" rule and regular transcript reviews keep the error rate low, but never at zero.