Embeddings and vector search: build a useful retrieval pipeline

What embeddings buy you

An embedding turns text into a vector that can be compared with other vectors. The OpenAI embeddings guide describes embeddings as useful for measuring relatedness and powering search, clustering, recommendations and classification. In a retrieval system, you embed each document chunk, store vectors in a database and embed the user's query at search time. The nearest chunks become evidence for the next step. This is powerful because a query can match related meaning even when wording differs. It is dangerous when people assume related meaning is the same as correct evidence.

Pipeline shape

1. Collect: choose source documents and keep original URLs or file paths.
2. Clean: remove navigation, boilerplate and duplicate text.
3. Chunk: split by headings or semantic sections, not arbitrary tiny fragments.
4. Embed: create vectors with one model and record the model name.
5. Store: write vectors plus payload such as title, date, product, permissions and source URL.
6. Search: retrieve candidates with vector similarity and filters.
7. Evaluate: check whether known questions retrieve the expected chunks.

Qdrant's collections documentation describes collections as named sets of points containing vectors and payload. That payload is not decorative. It lets you filter by product, tenant, date or access level before ranking.

Concrete example

Imagine a help-center search for a photo-editing app. A naive chunk says: Click Export. That chunk is almost useless because it lost the heading and product context. A better chunk includes the section title and nearby steps: Export transparent PNG. Open File, choose Export, set background to transparent, then save as PNG. Payload records product=desktop, version=current, source=/help/export-transparent-png. A query for save image without background now has a chance to retrieve the right chunk even without the exact word transparent.

Search is more than nearest neighbors

Vector similarity is only one part of retrieval. Qdrant's search documentation discusses nearest-vector search and filtering concepts. In real systems, filters often matter as much as similarity. If a user asks about the Android app, a high-scoring desktop article may be wrong. If a policy changed last week, an older chunk may be dangerous. If the user is in one tenant, another tenant's content must not be searched at all. Treat metadata design as a security and quality feature.

Evaluation set

Create a small spreadsheet before adding a chatbot. Columns: question, acceptable source chunks, unacceptable chunks, required filters and notes. Include easy questions, jargon questions, misspellings, product-specific questions and questions that should return no result. Run retrieval only. Measure whether the right chunks appear in the top results. If retrieval cannot find the evidence, generation will create fluent uncertainty.

Problems to check

• Chunk amnesia: chunks lose headings, dates or definitions needed to interpret them.
• Dimension mismatch: vectors from different embedding models are mixed in one index. Equal dimensions do not make different models' vector spaces interchangeable.
• Permission leak: private documents are embedded into a shared collection without access filters.
• Semantic false friend: similar wording retrieves a related but wrong policy.
• No deletion path: removed content remains in the vector database.

Next steps

Start with one document family and fifty questions. Do not build the chat UI yet. Build the ingestion script, inspect chunks manually, store payload, run retrieval tests and tune chunking. When retrieval is reliable enough, add answer generation with source display and an abstention rule. The boring retrieval work is what makes the impressive interface trustworthy.

Refresh policy

Add an update policy before launch. Decide whether documents re-embed on every edit, on a schedule or after manual approval. Record deleted source IDs and remove their vectors. When embedding models change, rebuild a separate index and compare retrieval results before swapping it into production. A vector database is part of your content system, not a one-time export.

Keyword backstop

Keep a lexical search path for identifiers, error codes, invoice numbers and policy names. Embeddings are useful for meaning, but exact strings often carry business meaning. A hybrid search that combines vector results with keyword or filter logic usually gives reviewers better evidence than either approach alone.

Keep a trace for each search

A search trace should record the query, index version, required access filters, retrieved source IDs and the chunks eventually shown to the model. Use a synthetic user with access to only one product area and ask for a document from another area. The inaccessible chunk should never reach the generation step. Also test a deleted document and a recently edited document. These checks cover permission and freshness failures that similarity scores alone cannot reveal. Avoid storing private query text by default; preserve the identifiers and measurements you need to diagnose a problem under your data policy.

Sources and further reading

Documentation checked October 4, 2026. Product features and availability can change.

• OpenAI embeddings guide
• Qdrant collections documentation
• Qdrant search documentation