Retrieval Augmented Generation (RAG) Basics

To use Sagepost effectively, it helps to understand how Sagepost improves the student Q&A experience.

At its core is a Retrieval Augmented Generation (RAG) Large Language Model (LLM).

What is RAG?

Normally, an LLM generates answers based on the massive dataset it was trained on. This is like a student trying to answer a question on a test using only their general knowledge from everything they've ever read. While this works for many topics, it can lead to hallucinations (making up facts) or providing outdated information because its knowledge is static.

RAG solves this by giving the LLM a new superpower: the ability to look up information from a specific knowledge base before answering. Sagepost uses course materials and instructor responses to guide the ansewrs. This makes the answers more accurate, specific, and up-to-date.

How Does it Work?

The RAG process has two main steps:

Retrieval: When you ask a question, the system doesn't just send it straight to the LLM. Instead, it first searches the course knowledge base which contains relevant material and previously verified answers. This database, known as a vector database, stores information in a way that makes it easy to find relevant content. The system retrieves the most relevant pieces of information (e.g., specific lecture slides) related to your question.

Generation: The system then combines your original question with the retrieved information. This combined prompt is then sent to the LLM. The LLM uses this new context to generate a precise answer. It's like telling the LLM, "Here's the user's question, and here are the specific passages from the class material that can help you answer it." This significantly reduces the chance of errors and ensures the answer is grounded in the provided source material.

Why is RAG Important for Sagepost?

For a platform like Sagepost designed to help students, RAG is critical because it ensures the LLM's answers are:

Accurate and Relevant: Answers are based on your specific course materials and instrucor expertise, not on general internet knowledge. This means if you ask a question about a specific concept, the LLM will first try to find the answer in your professor's course material.

Trustworthy: By citing the specific source material used to generate the answer, RAG helps students verify the information and learn where to find it themselves for future reference.

Up-to-Date: The knowledge base can be updated in real-time with new lecture slides or readings, so the LLM's information is always current. This avoids the problem of an LLM providing old information from its possibly outdated training data.