train_ivf_with_gpu.ipynb

Your dataset requires 1 TiB ram! Try to reduce the dimension as well as the nlist, or you will never get this index trained.

My dataset isnt getting trained on cpu rather that gpu inspite of using
index_ivf = faiss.extract_index_ivf(index2) clustering_index = faiss.index_cpu_to_all_gpus(faiss.IndexFlatL2(64)) index_ivf.clustering_index = clustering_index
Can any one help me

Can we use the GPU version of the Binary Flat index as the clustering index for the binary indexes? Like below:
faiss.index_cpu_to_all_gpus(faiss.IndexBinaryFlat(d))

My dataset isnt getting trained on cpu rather that gpu inspite of using index_ivf = faiss.extract_index_ivf(index2) clustering_index = faiss.index_cpu_to_all_gpus(faiss.IndexFlatL2(64)) index_ivf.clustering_index = clustering_index Can any one help me

Do you have faiss-gpu installed? What do you get for faiss.get_num_gpu()?

Hi @mdouze , is it correct that something like:
"PCAR1024,IVF262144_HNSW32,PQ512x8" (with inner product)
cannot be trained on the GPU?
https://gist.github.com/mdouze/46d6bbbaabca0b9778fca37ed2bcccf6?permalink_comment_id=3111847#gistcomment-3111847
You mention that PQ quantisation is not implemented?

hi,dear
my dataset dim=768 and num=1billion
and what type of ivf PQ is good for the index in CPU ?
thx
@mdouze

the up code
index2 could be write directly ?
such as faiss.write_index(index2,"index2.idx") ?