gistfile1.txt

>>> import faiss
>>> import numpy as np
>>>
>>>
>>> n = 9000 # number of data points
>>> d = 32    # dimension of each data point
>>> x = np.random.rand(n, d).astype('float32')
>>> n, d = x.shape
>>> m = 8 # number of subspaces decomposed
>>> c = 4 # 0,1,2,..., 2^c (number of centroids)
>>>
>>> pq = faiss.ProductQuantizer(d, m, c)
>>>
>>> opq = faiss.OPQMatrix(d, m)
>>> opq.pq = pq
>>> opq.verbose = True # verbose output
>>> opq.niter = 2      # reduce nb of iterations to get a shorter gist
>>> opq.train(x)
OPQMatrix::train: training an OPQ rotation matrix for M=8 from 9000 vectors in 32D -> 32D
  OPQMatrix::train: making random 32*32 rotation
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 40 iterations
  Preprocessing in 0.00 s
  Iteration 39 (0.14 s, search 0.10 s): objective=994.357 imbalance=1.007 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 40 iterations
  Preprocessing in 0.00 s
  Iteration 39 (0.08 s, search 0.05 s): objective=979.67 imbalance=1.019 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 40 iterations
  Preprocessing in 0.00 s
  Iteration 39 (0.09 s, search 0.06 s): objective=995.232 imbalance=1.018 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 40 iterations
  Preprocessing in 0.00 s
  Iteration 39 (0.10 s, search 0.07 s): objective=989.035 imbalance=1.012 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 40 iterations
  Preprocessing in 0.00 s
  Iteration 39 (0.12 s, search 0.09 s): objective=998.066 imbalance=1.016 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 40 iterations
  Preprocessing in 0.00 s
  Iteration 39 (0.08 s, search 0.04 s): objective=1009.64 imbalance=1.014 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 40 iterations
  Preprocessing in 0.00 s
  Iteration 39 (0.12 s, search 0.07 s): objective=1002.83 imbalance=1.017 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 40 iterations
  Preprocessing in 0.00 s
  Iteration 39 (0.61 s, search 0.42 s): objective=992.544 imbalance=1.018 nsplit=0
    Iteration 0 (40 PQ iterations):1.367 s, obj=0.88444
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 4 iterations
  Using 16 centroids provided as input (not frozen)
  Preprocessing in 0.00 s
  Iteration 3 (0.16 s, search 0.12 s): objective=988.728 imbalance=1.007 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 4 iterations
  Using 16 centroids provided as input (not frozen)
  Preprocessing in 0.00 s
  Iteration 3 (0.08 s, search 0.06 s): objective=973.777 imbalance=1.016 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 4 iterations
  Using 16 centroids provided as input (not frozen)
  Preprocessing in 0.00 s
  Iteration 3 (0.05 s, search 0.03 s): objective=989.404 imbalance=1.016 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 4 iterations
  Using 16 centroids provided as input (not frozen)
  Preprocessing in 0.00 s
  Iteration 3 (0.17 s, search 0.14 s): objective=982.748 imbalance=1.012 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 4 iterations
  Using 16 centroids provided as input (not frozen)
  Preprocessing in 0.00 s
  Iteration 3 (0.11 s, search 0.07 s): objective=992.873 imbalance=1.014 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 4 iterations
  Using 16 centroids provided as input (not frozen)
  Preprocessing in 0.00 s
  Iteration 3 (0.09 s, search 0.08 s): objective=1006.39 imbalance=1.011 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 4 iterations
  Using 16 centroids provided as input (not frozen)
  Preprocessing in 0.00 s
  Iteration 3 (0.04 s, search 0.04 s): objective=998.428 imbalance=1.017 nsplit=0
Clustering 9000 points in 4D to 16 clusters, redo 1 times, 4 iterations
  Using 16 centroids provided as input (not frozen)
  Preprocessing in 0.00 s
  Iteration 3 (0.05 s, search 0.05 s): objective=987.832 imbalance=1.016 nsplit=0
    Iteration 1 (4 PQ iterations):2.196 s, obj=0.879914