We typicaly do this through Hetzner, but EC2 or Compute Instances on GCP would also be ok. However, Hetzner is preferred unless you have a strong preference against it.
The preferred region is US-West.
For operating system, pick Ubuntu.
skeptrune skeptrunedev
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "params": { | |
| "vectors": { | |
| "1024_vectors": { | |
| "size": 1024, | |
| "distance": "Cosine", | |
| "hnsw_config": { | |
| "on_disk": false | |
| }, | |
| "quantization_config": { |
skeptrunedev
/ delete-all-git-branches-except-main.sh
Created
July 2, 2024 22:41
Delete All Git Branches Except Main Unix Shell Script
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| git branch | grep -v main | xargs git branch -D |
skeptrunedev
/ create-500-trieve-datasets-for-testing.js
Created
July 3, 2024 18:33
Create 500 Trieve Datasets for Testing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| const createDatasetPayload = JSON.parse("{\"dataset_name\":\"test14\",\"organization_id\":\"95b7c53e-2c24-49a1-97fa-c87188c7324b\",\"server_configuration\":{\"LLM_BASE_URL\":\"\",\"LLM_DEFAULT_MODEL\":\"\",\"EMBEDDING_BASE_URL\":\"https://embedding.trieve.ai\",\"EMBEDDING_MODEL_NAME\":\"jina-base-en\",\"MESSAGE_TO_QUERY_PROMPT\":\"\",\"RAG_PROMPT\":\"\",\"EMBEDDING_SIZE\":768,\"N_RETRIEVALS_TO_INCLUDE\":8,\"DUPLICATE_DISTANCE_THRESHOLD\":1.1,\"DOCUMENT_UPLOAD_FEATURE\":true,\"DOCUMENT_DOWNLOAD_FEATURE\":true,\"COLLISIONS_ENABLED\":false,\"FULLTEXT_ENABLED\":true,\"QDRANT_COLLECTION_NAME\":null,\"EMBEDDING_QUERY_PREFIX\":\"Search for: \",\"USE_MESSAGE_TO_QUERY_PROMPT\":false,\"FREQUENCY_PENALTY\":null,\"TEMPERATURE\":null,\"PRESENCE_PENALTY\":null,\"STOP_TOKENS\":null,\"INDEXED_ONLY\":false,\"LOCKED\":false},\"client_configuration\":\"{}\"}"); | |
| for (let i = 0; i<500; i++) { | |
| createDatasetPayload.dataset_name = `test_${i}`; | |
| fetch("http://localhost:8090/api/dataset", { | |
| "headers": { | |
| "accept": |
skeptrunedev
/ trieve-search-playground
Created
July 25, 2024 18:26
Trieve Search Playground Screenshot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| I need to upload image in comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // TODO: latency optimize this so it can be uncommented | |
| if phrases.is_empty() { | |
| let potential_query_split_whitespace = potential_query.split_whitespace().collect_vec(); | |
| if potential_query_split_whitespace.len() > 5 { | |
| continue; | |
| } | |
| let query_without_stop_words = potential_query | |
| .split_whitespace() | |
| .filter(|word| !stop_words.contains(&word.to_lowercase())) | |
| .collect::<Vec<&str>>(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| foo.bar.com { | |
| root * /home/git_projects/trieve-hn-discovery/frontend/dist | |
| try_files {path} {path}/ /index.html | |
| file_server | |
| } |
skeptrunedev
/ cargo.toml
Created
August 6, 2024 22:57
Rust actix-web implementation for actix-extensible-rate-limit crate (Trieve)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| redis = { version = "0.25", features = [ | |
| "tokio-rustls-comp", | |
| "aio", | |
| "connection-manager", | |
| ] } | |
| actix-extensible-rate-limit = { version = "0.3.1", features = ["redis"] } |
skeptrunedev
/ history-of-hn-search.md
Last active
August 12, 2024 21:42
Blog Post Documenting the History of HN Search from 2014 to 2024
| publishDate | author | title | excerpt | image | category | tags | |
|---|---|---|---|---|---|---|---|
2024-08-12 08:45:00 UTC |
skeptrune (Nick K) |
History of HackerNews Search: From 2007 to 2024 |
The history of HackerNews (HN) search spans three generations. Starting in 2007 with Disqus founder Jason Yan followed by a series of other sites, Octopart/ThriftDB-powered HNSearch in 2011, and finally Algolia-powered search from 2014 to today. |
History |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curl -fsSL https://get.docker.com -o get-docker.sh | |
| sudo sh get-docker.sh |