What is a vector database, and does your team need one?

What a vector database is

Pinecone's explainer defines a vector database as one that "indexes and stores vector embeddings for fast retrieval and similarity search". Embeddings are produced by an AI model and carry semantic information: two sentences that mean the same thing get nearby vectors even if they share no words.

Search works in three moves. Each document chunk is turned into an embedding and stored next to its original text. A question is turned into an embedding with the same model. The database then finds the stored vectors most similar to the question's, using a similarity measure such as cosine similarity, and usually an approximate nearest-neighbour index that trades a little accuracy for a lot of speed.

It doesn't have to be a separate product. pgvector, an open-source extension, adds vector similarity search to Postgres so you can "store your vectors with the rest of your data".

Why a small team should care

• It is behind "chat with your docs". Knowing how it works explains why some answers miss.
• Garbage in, nearest garbage out. The database returns the closest chunks, including stale ones. Two versions of a policy mean two near matches.
• Chunking decides what can be found. Microsoft's search guide explains that documents are split into chunks to fit model limits, and the split changes results.

A worked example

Illustration: a made-up business. Parcel Pal, a seven-person shipping software company, builds a help assistant over 200 help articles stored in a vector database. Asked "can I ship batteries to Canada?", it returns a 2023 article about lithium batteries to the US and a carrier page about Canada, and blends them into a confident wrong answer.

The fix is in the content. The team deletes 30 outdated articles, splits long ones so each section covers one country or one item type with the country named in the text, and adds a date to every article. With 25 test questions written before the change, correct answers rise from 15 to 22. The database never changed.

What to prepare before choosing one

1. Pick the job the search must do and the questions it must answer.
2. Inventory the documents with an owner and a date each, and remove duplicates.
3. Split documents into self-contained sections that name their topic, product and date.
4. Write 20 test questions with the expected source for each.
5. Try the simplest option first (a project with uploaded files, or pgvector in a database you already run) and measure with the test questions.

Common confusions

• "It makes the AI smarter." It makes the AI find things. What it finds is only as good as what you stored.
• "Keyword search is obsolete." Exact terms such as product codes often still need keyword matching; many systems combine both.
• "Questions about the whole collection work too." Similarity search finds passages, not patterns across all documents; that is what GraphRAG tries to fix.

Questions

What is a vector database in simple terms?

A database that stores text (or images) as lists of numbers called embeddings, which capture meaning. It finds the stored items whose meaning is closest to a question, so 'how do I get my money back' finds the refund policy even if the word 'refund' isn't in the question.

Do I need a vector database to use RAG?

Not as a separate product at the start. Claude and ChatGPT projects search uploaded files for you, and existing databases such as Postgres can store vectors with the pgvector extension. A dedicated vector database matters when you build your own search over many documents.

What is an embedding?

A list of numbers an AI model produces for a piece of text, such that texts with similar meaning get similar numbers. Pinecone describes embeddings as a vector representation that carries semantic information.