Skip to content
Submit a ticket Customer portal
  • There are no suggestions because the search field is empty.

What affects GPT response speed? RAG & optimisation tips

This article explains how Retrieval-Augmented Generation (RAG) uses your dataset to provide accurate answers. Learn how to optimise your sources for better performance.

The speed at which GPT generates responses can vary depending on several factors, including the model, the technical setup, and the complexity of the query. 

What is RAG, and why does it matter?

At Ebbot, we use a technique called Retrieval-Augmented Generation (RAG) to enhance the accuracy of our GPT responses. 
While this term might sound technical, it’s a straightforward concept that plays a crucial role in delivering relevant and high-quality answers. Let’s break it down.

Understanding RAG in simple terms

RAG is like giving GPT a "cheat sheet" to ensure it provides accurate responses. 
Here’s how it works:

  1. Your question: When you ask a question or request, the embedder processes it.

  2. Finding helpful documents: Instead of relying solely on its pre-trained knowledge, GPT searches through a collection of documents provided by you (the dataset) to find the most relevant information.

  3. Generating an answer: With the help of these documents, GPT creates a response that is tailored to your specific needs and context.

Think of RAG as a librarian assisting an author. The librarian (RAG) hands over the most relevant books (documents) to the author (GPT), who then uses them to write a detailed, accurate answer.

Why relevant documents are essential:

The quality of GPT’s response depends heavily on the documents included in the dataset. If the dataset contains outdated, irrelevant, or unclear information, the answers will reflect that. On the other hand, a well-curated dataset ensures:

  • Better accuracy: GPT has access to the most accurate and up-to-date facts.

  • Faster responses: By narrowing the pool of information, GPT spends less time sifting through unnecessary details.

  • More context-specific answers: The response aligns closely with your organization’s needs, tone, and goals.

Tips for organizing your dataset

It’s important to provide our GPT models with the right materials to get the most out of RAG. 
Here’s what to consider when building your dataset:

  1. Focus on relevance: Include documents directly addressing the questions your customers are likely to ask.

  2. Keep it updated: Regularly review your dataset to remove outdated or inaccurate information.

  3. Organize clearly: Group documents by category or topic to make it easier for the system to find help the system.

By curating a high-quality dataset, you ensure that GPT delivers fast, meaningful, and precise answers.