Skip to content

cyborginc/cyborgdb-migrate

Repository files navigation

cyborgdb-migrate

PyPI - Version PyPI - License PyPI - Python Version

A TUI wizard and CLI for migrating vector data from popular vector databases into CyborgDB — the encrypted vector database.

Supports Pinecone, Qdrant, Weaviate, ChromaDB, and Milvus as sources, with encrypted-at-rest indexes, checkpoint & resume, and post-migration verification built in.

Key Features

  • Interactive TUI — step-by-step wizard powered for migration
  • Headless CLI — non-interactive mode for scripts and CI/CD pipelines
  • Checkpoint & resume — automatically saves progress; resume interrupted migrations
  • Spot-check verification — post-migration vector and metadata integrity checks
  • Double-buffered I/O — overlaps extraction and upsert for maximum throughput

Getting Started

To get started in minutes, check out the CyborgDB Quickstart Guide.

Installation

pip install cyborgdb-migrate

Install with support for your source database:

# Individual sources
pip install "cyborgdb-migrate[pinecone]"
pip install "cyborgdb-migrate[qdrant]"
pip install "cyborgdb-migrate[weaviate]"
pip install "cyborgdb-migrate[chromadb]"
pip install "cyborgdb-migrate[milvus]"

# All sources at once
pip install "cyborgdb-migrate[all]"

Usage

Interactive (TUI)

cyborgdb-migrate

The wizard walks you through selecting a source, entering credentials, connecting to CyborgDB, and running the migration with live progress.

Headless (CLI)

Create a TOML config file (see example-config.toml):

[source]
type = "pinecone"
api_key = "${PINECONE_API_KEY}"
index = "my-index"

[destination]
host = "http://localhost:8000"
api_key = "${CYBORGDB_API_KEY}"
create_index = true
index_name = "my-cyborgdb-index"
index_type = "ivfflat"

[options]
batch_size = 200
checkpoint_every = 10
cyborgdb-migrate --config migration.toml

Resume an interrupted migration:

cyborgdb-migrate --config migration.toml --resume

Supported Sources

Source Extras Notes
Pinecone pinecone Supports namespaces
Qdrant qdrant Scroll-based pagination
Weaviate weaviate Supports named vectors
ChromaDB chromadb Local and remote modes
Milvus milvus Supports partitions, content field heuristic

Configuration Reference

For full configuration details, CLI options, and exit codes, see the Configuration Guide.

Documentation

For more information on CyborgDB, see the Cyborg Docs.

License

The CyborgDB Migration Tool is licensed under the MIT License.

About

TUI migration tool from various vector DBs to CyborgDB

Resources

License

Contributing

Stars

Watchers

Forks

Contributors