dokk

RAG tutorial

Example page

The project, data and links on this page are made up, to show what dokk’s components can do. It does not document dokk itself.

In this tutorial you build a small app that answers questions about your own Markdown files: it finds the relevant passages and asks a model to answer from them only. This is retrieval-augmented generation, RAG for short.

What you need #

  1. Python 3.11 or newer.
  2. An API key for a model provider.
  3. A folder of Markdown files to ask questions about.

Set up the project #

pip install sentence-transformers

It brings NumPy along. Then put your settings in .env:

VariableDefaultDescription
OPENAI_API_KEY required — For the answering model.
DOCS_DIR ./notes The Markdown files to search.
TOP_K 4 How many passages go into the prompt.

When you are done, the project looks like this:

  • .env
  • app.pythe whole app
  • notes/
    • …

Embed your documents #

An embedding model turns each passage into a vector; similar passages get similar vectors. This tutorial uses a small one that runs on a laptop:

app.py
from pathlib import Path

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
passages = [p for f in Path("notes").glob("*.md") for p in f.read_text().split("\n\n")]
vectors = model.encode(passages, normalize_embeddings=True)


def search(question: str, k: int = 4) -> list[str]:
    scores = vectors @ model.encode(question, normalize_embeddings=True)
    return [passages[i] for i in np.argsort(-scores)[:k]]

Ask the model #

The prompt gives the model the passages and tells it to stay within them. Try it: fill in the fields and open it in a chat.

Answer from the passages user

Answer the question using only the passages below. If they do not contain the answer, say so.

Question: {{ question }}

Passages: {{ passages }}

Tip

Ask the model to quote the passage it used. Wrong answers are then easy to spot.

Check it #

When is the team offsite?

The offsite is on 14 November in Utrecht (“Offsite: Thu 14 Nov, Utrecht”, from notes/team.md).

What is the wifi password there?

The passages do not say.

Going further

Store the vectors in a file so you embed only once, split long files into smaller passages, and compare a few embedding models on your own questions.