Vector data management (Vectors API)

This document covers how to manage the vector data (embeddings) the RAG system uses — upsert, delete, search, metadata management — everything you need to operate the vector store.

1. What a vector store is

A vector store persists text as vectors (numeric arrays) so you can do similarity search. In this platform it's used for:

  • Semantic search over text chunks (documents)
  • The retrieval target for RAG (Retrieval-Augmented Generation)
  • Similarity search over FAQs, knowledge, internal docs

Vectors are stored as arrays of 768 or 1,536 dimensions (varies by model), generated by the LLM / embedding model.

2. Vectors API overview

Method Endpoint Description
POST/api/vectors/upsertInsert or update vector data
POST/api/vectors/searchVector similarity search
DELETE/api/vectors/{id}Delete a specific vector
GET/api/vectors/{id}Fetch metadata for a specific vector

3. Upsert

Insert or update a vector. Same ID overwrites, new ID is created.

■ Request
POST /api/vectors/upsert
Content-Type: application/json

{
  "id": "chunk-001",
  "values": [0.12, -0.44, 0.91, ...],
  "metadata": {
    "text": "AIとは、人間の知的行動を模倣する技術の総称である。",
    "source": "wiki-ai",
    "path": "/docs/ai",
    "updatedAt": "2025-01-14T10:00:00Z"
  }
}
■ Response
{
  "status": "ok",
  "id": "chunk-001",
  "operation": "updated"
}

4. Similarity search

Returns documents closest to the query by cosine similarity.

■ Request
POST /api/vectors/search
Content-Type: application/json

{
  "vector": [0.11, -0.52, 0.33, ...],
  "topK": 5,
  "includeMetadata": true
}
■ Response
{
  "results": [
    {
      "id": "chunk-001",
      "score": 0.92,
      "metadata": {
        "text": "AIとは〜",
        "source": "wiki-ai",
        "path": "/docs/ai"
      }
    }
  ]
}

5. Fetch metadata

Retrieve the details of a stored vector.

GET /api/vectors/chunk-001

{
  "id": "chunk-001",
  "dimensions": 1536,
  "metadata": {
    "text": "...",
    "source": "wiki-ai",
    "path": "/docs/ai"
  }
}

6. Delete

Delete a specific vector.

DELETE /api/vectors/chunk-001

{
  "status": "deleted",
  "id": "chunk-001"
}

7. Best practices

  • 300–600 chars per chunk works best
  • Attach text / path / source / updatedAt as metadata to boost search quality
  • When re-processing the same document, pin the ID and upsert.
  • Run semantic evaluations regularly to watch for bias drift.

Next: RAG overall architecture →