gistfile1.txt

matthijs@devfair0144:~/faiss_versions/faiss_cuda9/faiss/gpu$ make
/public/apps/cuda/9.0//bin/nvcc  -I /public/apps/cuda/9.0//targets/x86_64-linux/include/ -Xcompiler -fPIC -Xcudafe --diag_suppress=unrecognized_attri
bute -gencode arch=compute_60,code="compute_60" -gencode arch=compute_61,code="compute_61" --std c++11 -lineinfo -ccbin g++ -DFAISS_USE_FLOAT16 -o te
st/demo_ivfpq_indexing_gpu test/demo_ivfpq_indexing_gpu.cpp libgpufaiss.a ../libfaiss.a -Xcompiler -fopenmp -lcublas \
-Xlinker --no-as-needed -Xlinker -rpath,/public/apps/intel/mkl/2018.0.128/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/public/apps
/intel/mkl/2018.0.128/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64   -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -ldl -lpthread
matthijs@devfair0144:~/faiss_versions/faiss_cuda9/faiss/gpu$ test/demo_ivfpq_indexing_gpu
[1.887 s] Generating 100000 vectors in 128D for training
[2.111 s] Training the index
Training IVF quantizer on 100000 vectors in 128D
Clustering 100000 points in 128D to 1788 clusters, redo 1 times, 10 iterations
  Preprocessing in 0.75 s
  Iteration 9 (0.45 s, search 0.20 s): objective=930934 imbalance=1.255 nsplit=0
computing residuals
training 4 x 256 product quantizer on 16384 vectors in 128D
Training PQ slice 0/4
Clustering 16384 points in 32D to 256 clusters, redo 1 times, 25 iterations
  Preprocessing in 0.00 s
  Iteration 24 (2.39 s, search 2.14 s): objective=27271.5 imbalance=1.018 nsplit=0
Training PQ slice 1/4
Clustering 16384 points in 32D to 256 clusters, redo 1 times, 25 iterations
  Preprocessing in 0.00 s
  Iteration 24 (2.19 s, search 1.97 s): objective=27193.4 imbalance=1.016 nsplit=0
Training PQ slice 2/4
Clustering 16384 points in 32D to 256 clusters, redo 1 times, 25 iterations
  Preprocessing in 0.00 s
  Iteration 24 (3.25 s, search 2.93 s): objective=27230.8 imbalance=1.021 nsplit=0
Training PQ slice 3/4
Clustering 16384 points in 32D to 256 clusters, redo 1 times, 25 iterations
  Preprocessing in 0.00 s
  Iteration 24 (2.73 s, search 2.45 s): objective=27174 imbalance=1.023 nsplit=0
[14.089 s] storing the pre-trained index to /tmp/index_trained.faissindex
[14.092 s] Building a dataset of 200000 vectors to index
[14.902 s] Adding the vectors to the index
[15.315 s] done
[15.317 s] Searching the 40 nearest neighbors of 9 vectors in the index
[15.317 s] Query results (vector ids, then distances):
query  0:    1234   11667  163213   25740  111346  140205   81080  190057   42227  157764  100335   82245    7335   77912   11175  123134   92454   23485   12665  100394  157942   73313  164858   84646   98977   15104   83039   29935  196068  140839   20712  103332   56767    3379    2469   13439  171273  139737   27110  126536
     dis: 6.67284 9.36822 9.73539 9.74364 10.1151 10.2513 10.3035 10.3917 10.4391  10.459 10.4919  10.492  10.581  10.639 10.6476 10.6741 10.7218 10.7331 10.7442 10.7574 10.7846 10.8171  10.857 10.8661 10.8808 10.9818 11.0389 11.0507 11.0735 11.0791 11.1004 11.1179 11.1357 11.1387 11.1416 11.1629 11.1758 11.1931 11.1972 11.2068
query  1:    1235  155021  191832  144480  128782  104890   58160  118897   45733  139097    1148   36408   49648   81301   88894   56663  154238  173619  113870   76478  119405   41019   97785   98461  170692  170422    5080  117320   44241   41427   94510   13782  100048    3081   22166   66545   40030  106581    7135  112511
     dis:  7.8503 10.4889 10.9427 11.0414 11.2547 11.3249 11.3687 11.3724 11.4034 11.4253 11.4814 11.5192 11.5761 11.6073 11.6301 11.6418  11.704 11.7627 11.8125 11.8257 11.8308 11.8508 11.8567 11.9022 11.9366  11.993 12.0717 12.0914 12.1223 12.1555 12.1705 12.1722 12.2338 12.2394 12.2431 12.2438 12.3512 12.3761 12.4568 12.5087
query  2:    1236   79071   57167   89180  164280   80604   71507   35279   26315   26024    6220   99318   19173  112175  161386   11199  185497   21028   95432  115136  199527  164483   97406    9345  165637  189758  193673    1507   89680  197285   27794  185206  199392  117052    2544   12847  190655   48332  103610   99137
     dis: 8.41928 10.8765 11.1952 11.3061 11.4082 11.4307 11.4532 11.5826 11.5838 11.6815 11.7085  11.715  11.797  11.808 11.8369 11.8748 11.8994 11.9007 12.0304 12.0874 12.1119 12.2301 12.2301 12.2457 12.2785 12.2853 12.2917 12.3101 12.3202 12.3615 12.3682 12.4038 12.4069 12.4282 12.4338 12.4971 12.5139 12.5833 12.6006 12.6246
query  3:    1237   85352  179827   16149  129967   23764  134024   10312   59166   43870  192413  183182   82065   79873  107144   48293   76199   65420  107489   67082  133167  180081   68333   33141  180239  160466    9674   64812   48199   45404   60451   23991  111196   81415  147145   85618   54899   54623  171163   15811
     dis: 7.68749 9.98241 10.3924 10.5489 10.7406 10.9116 10.9296 11.0807 11.1075 11.1572 11.1689 11.1956 11.2681  11.354 11.5283 11.5438 11.5829 11.6019 11.6072 11.6399 11.6786 11.7004 11.7163 11.7353 11.7389 11.7524  11.767 11.8145 11.8227 11.9115 11.9133 11.9204 11.9556 11.9965  12.071 12.0991 12.1259 12.1669 12.1801 12.1894
query  4:    1238  197415  110268   18115   48471  154600  109792   35907  102142  136499  116224   90808   40425   71782   69284   67874   91514  133874  156044  112960     861   38908  191867   32847   23084  126804   29679   91055   11308  128941  105800   62304   56131  181043  116735  199447   71309   24635   16611    6864
     dis: 7.73137 10.0799  10.268 10.4167 10.7833 10.8219 10.8267 10.8584 10.8953 10.9044 10.9189 10.9645 11.0112 11.0246 11.0249 11.0386 11.0981 11.1178 11.1185 11.1552 11.1571 11.1639 11.3341 11.3391 11.3533 11.3832 11.4058 11.4152 11.4239 11.4422 11.4423 11.4517 11.5054 11.5586  11.562 11.5666 11.5701 11.5709 11.5734 11.5942
query  5:    1239  113040  172814   57562   86287   99066  143840   72414   41043  116394   91802  135602   58637   96530  112672  190620   20818  152392  172431   70772  154736  184190  130277  161676  183556  158284   77650  109187  182454  119889  128184  152190  163088  101051  109722   60791  139535  177412  108637  163066
     dis: 7.86749 9.35082 10.1998 10.3026 10.3908 10.5122 10.6011 10.7522 10.7817 10.7998 10.8607  10.928 10.9596 10.9598  11.005  11.019 11.1708 11.3589 11.4255 11.4384 11.4787 11.5659 11.6238 11.6433 11.6554 11.6905 11.7444 11.7631 11.8202 11.8384 11.8477 11.8603 11.8853  11.931 11.9545 12.0103 12.0129   12.03 12.0375 12.0743
query  6:    1240  153562   56211   91931  193953   26649   13979  138276  110725  162475    7508   79370   94485   15434   63248  145256  135102   92577  127662  111045  184852  151170  142591  173887  169971  122081  173480  120371  129206   37980  193340   10759  145687   61978  145698   15232   15066  143116  199619   55073
     dis: 7.75459 9.87971 10.2025 10.3361  10.424 10.5129 10.6073 10.7071 10.8162 10.8206 10.8635 10.8923 10.9074 10.9144 10.9966 11.0713 11.1043 11.1626 11.1675 11.3083 11.3353 11.3829 11.4237 11.4344 11.5283 11.5307 11.5349 11.5584 11.5661 11.5767 11.6312 11.6408  11.643 11.6542 11.6827 11.7854 11.8077  11.814  11.834 11.8778
query  7:    1241   40016  138265   14696  139353  162441  112428  110390    8799  139861    5572  124052  111074   84732  194151  106562  165450  141566  164927   38511  170850   94659   42266  177555    4851   78223   71206     439  114692   41653  182600  134279   34276  195227    3277  125067  119340  192188   99651   30037
     dis: 8.38696 11.7385 11.9085 12.0128  12.044 12.0708 12.2936 12.3234 12.5058 12.5168 12.5368  12.553 12.5726 12.5986 12.6685 12.7367 12.8121 12.9095 12.9465 12.9511 12.9693 12.9724 12.9782 12.9919 13.1622 13.1742 13.1826  13.186 13.1945 13.2172 13.2223 13.2493 13.2844 13.3515 13.3724 13.3741 13.3789 13.3956  13.403 13.4185
query  8:    1242  150490   66094    7357  115727    3851   10030  170360   77870   62743  121407  172453  133284  191405   14500  178426  189063    1706  127755    3275   13773  197897   84396  170168   46855  156532  152719   34055  150681   66419    8376   33524  177867  182952   64481  115307   60887  107211  192878  114303
     dis: 7.86947  10.836 11.2878 11.3013 11.3301  11.339 11.4253 11.4814 11.5142 11.5386 11.6377 11.7312 11.7543 11.7575 11.7685 11.9347  12.027 12.0383 12.1057 12.1144  12.227 12.2533 12.2541 12.3434 12.3703 12.3763 12.3769 12.3853 12.4107 12.4426 12.4712 12.5709 12.5845 12.5903 12.6299 12.6335 12.6359 12.6785  12.679 12.7003
note that the nearest neighbor is not at distance 0 due to quantization errors