Skip to content

MB-62182: fast merge of vector indexes#47

Merged
abhinavdangeti merged 13 commits intoblevefrom
merge_from
Mar 25, 2026
Merged

MB-62182: fast merge of vector indexes#47
abhinavdangeti merged 13 commits intoblevefrom
merge_from

Conversation

@Thejas-bhat
Copy link
Copy Markdown
Member

@Thejas-bhat Thejas-bhat commented Mar 5, 2025

  • The existing merge_from API can be utilized to merge 2 IVF indexes provided they have identical centroid layout.
  • To avoid re-training in a segment architecture, a template index can be first created out of a random sample from the data when its size is known and fixed.
  • This template can be copied between segments and the inverted lists can be merged without affecting the learned context of the data distribution. This would also involve the merging of the direct map as well which stores the exact location of a key (vector) within the inverted lists.

@Thejas-bhat Thejas-bhat changed the title Draft: making a IndexIVF's merge_from API a little more flexible MB-62182: fast merge of vector indexes Jan 29, 2026
@Thejas-bhat Thejas-bhat moved this from Todo to In Progress in Fast Merge Jan 30, 2026
@Thejas-bhat Thejas-bhat marked this pull request as ready for review February 19, 2026 00:15
Comment thread c_api/IndexIVF_c_ex.cpp Outdated
Comment thread c_api/IndexIVF_c_ex.cpp Outdated
Comment thread faiss/IndexIVF.cpp Outdated
Comment thread c_api/IndexIVF_c_ex.cpp Outdated
@abhinavdangeti abhinavdangeti dismissed capemox’s stale review March 25, 2026 18:21

Comments addressed, re-review.

@abhinavdangeti abhinavdangeti merged commit 0f86a01 into bleve Mar 25, 2026
@abhinavdangeti abhinavdangeti deleted the merge_from branch March 25, 2026 21:47
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Fast Merge Mar 25, 2026
abhinavdangeti pushed a commit to blevesearch/go-faiss that referenced this pull request Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants