hweller1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {$vectorSearch:{ | |
| "index": "default", | |
| "queryVector": embedding, | |
| "path": "doc_embedding", | |
| "filter" : { "$and" : [{"beds": {"$gte" : 1}} , "score": {"$gte" : 91}}]}, | |
| "k": 100, | |
| "numCandidates": 1000 | |
| } | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "$vectorSearch": { | |
| "index":"default", | |
| "queryVector": vector_query, | |
| "path": "embedding", | |
| "limit": 5, | |
| "numCandidates": 50 | |
| } | |
| }, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "type": "vectorSearch, | |
| "fields": [{ | |
| "path": "embedding", | |
| "dimensions": 768, # the dimension of `mpnet-base` model | |
| "similarity": "euclidean", | |
| "type": "vector" | |
| }] | |
| } |
hweller1
/ sample_index3.py
Created
December 1, 2023 20:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "type": "vectorSearch, | |
| "fields": [{ | |
| "path": "embedding", | |
| "dimensions": 1536, | |
| "similarity": "cosine", | |
| "type": "vector" | |
| }] | |
| } |
hweller1
/ sample_index2.py
Created
December 1, 2023 20:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "type": "vectorSearch, | |
| "fields": [{ | |
| "path": "plot_embedding_hf", | |
| "dimensions": 384, | |
| "similarity": "dotProduct", | |
| "type": "vector" | |
| }] | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "type": "vectorSearch, | |
| "fields": [{ | |
| "path": "plot_embedding", | |
| "dimensions": 1536, | |
| "similarity": "cosine", | |
| "type": "vector" | |
| }] | |
| } |
hweller1
/ filter_experiments.py
Created
November 13, 2023 19:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $search": { | |
| "index":'1M_sphere_index', | |
| "knnBeta": { | |
| "path": "vector", | |
| "vector": embedding.tolist(), | |
| "k": k * multiplier, | |
| # "filter":{ | |
| # "equals":{ | |
| # "path":"low_card", | |
| # "value":1 |
hweller1
/ compute_overlap.py
Created
November 9, 2023 17:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def compute_overlap(exact_result_set: List, approx_result_set: List) -> float: | |
| # each result set is a list of urls, order not considered | |
| return len(exact_result_set.intersection(set(approx_result_set)) / len(set(exact_result_set)) |
hweller1
/ rrf_via_unionWith.py
Last active
February 23, 2024 09:40
Perform reciprocal rank fusion using $push to expose rank, and $unionWith and $group to join result sets of vector search and full text search
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pymongo | |
| import time | |
| from sentence_transformers import SentenceTransformer | |
| from companies import names # list of company names in a separate python file | |
| ### DESCRIPTION | |
| """ | |
| Search against the Sphere dataset using vector search results fused with full text search results via reciprocal rank fusion. |
hweller1
/ rsf_via_union_with.py
Last active
August 28, 2024 23:36
relative score fusion using unionWith and group, using the scores yielded from the searchMeta
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pymongo | |
| import time | |
| from sentence_transformers import SentenceTransformer | |
| from companies import names # List of company names from another python file | |
| ### DESCRIPTION | |
| """ |