Marty Schoch mschoch
mschoch
/ gist:5181f4a3e2f273c30bd4
Created
November 15, 2014 23:10
Custom bleve analyzer for words containing hyphens
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| func buildIndexMapping() (*bleve.IndexMapping, error) { | |
| indexMapping := bleve.NewIndexMapping() | |
| err = indexMapping.AddCustomTokenizer("words_with_hyphens_tokenizer", | |
| map[string]interface{}{ | |
| "regexp": "[0-9A-Za-z_\-]+", | |
| "type": "regexp", | |
| }) | |
| if err != nil { | |
| return nil, err |
mschoch
/ gist:b62bf9ca81e1e783ab2a
Created
November 18, 2014 20:17
custom mapping to index subwords from compound words
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| func buildIndexMapping() (*bleve.IndexMapping, error) { | |
| indexMapping := bleve.NewIndexMapping() | |
| var err error | |
| err = indexMapping.AddCustomTokenMap("sv_compound_auto_wordmap", | |
| map[string]interface{}{ | |
| "type": `custom`, | |
| "tokens": []interface{}{ | |
| `olje`, | |
| `luft`, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #include <stdio.h> | |
| #include <string.h> | |
| #include <assert.h> | |
| #include "libforestdb/forestdb.h" | |
| int main(int argc, char**argv) { | |
| fdb_file_handle *file; | |
| fdb_kvs_handle *kvs, *snapshot; | |
| fdb_status status; | |
| fdb_config config; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #include <stdio.h> | |
| #include <string.h> | |
| #include <assert.h> | |
| #include "libforestdb/forestdb.h" | |
| int main(int argc, char**argv) { | |
| fdb_file_handle *file; | |
| fdb_kvs_handle *kvs, *snapshot; | |
| fdb_status status; | |
| fdb_config config; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #include <stdio.h> | |
| #include <string.h> | |
| #include <assert.h> | |
| #include "libforestdb/forestdb.h" | |
| int main(int argc, char**argv) { | |
| fdb_file_handle *file; | |
| fdb_kvs_handle *kvs, *snapshot; | |
| fdb_status status; | |
| fdb_config config; |
mschoch
/ test_iterator_snapshot_with_commit.c
Created
December 27, 2014 02:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #include <stdio.h> | |
| #include <string.h> | |
| #include <assert.h> | |
| #include "libforestdb/forestdb.h" | |
| int main(int argc, char**argv) { | |
| fdb_file_handle *file; | |
| fdb_kvs_handle *kvs, *snapshot; | |
| fdb_status status; | |
| fdb_config config; |
mschoch
/ test_iterator_snapshot_without_commit.c
Created
December 27, 2014 02:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #include <stdio.h> | |
| #include <string.h> | |
| #include <assert.h> | |
| #include "libforestdb/forestdb.h" | |
| int main(int argc, char**argv) { | |
| fdb_file_handle *file; | |
| fdb_kvs_handle *kvs, *snapshot; | |
| fdb_status status; | |
| fdb_config config; |
mschoch
/ gist:ee4ce66de20b42f7efb9
Created
January 12, 2015 19:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mschoch@gamer:~/go/src/github.com/blevesearch/bleve/index/upside_down$ go test -v -test.run=XXX -test.bench=BenchmarkNullIndexing1Workers100Batch -tags 'leveldb forestdb' -test.cpu=1 -test.cpuprofile=nullindex100.profile | |
| PASS | |
| BenchmarkNullIndexing1Workers100Batch 10 118690503 ns/op | |
| ok github.com/blevesearch/bleve/index/upside_down 1.314s | |
| mschoch@gamer:~/go/src/github.com/blevesearch/bleve/index/upside_down$ go tool pprof upside_down.test nullindex100.profile | |
| Entering interactive mode (type "help" for commands) | |
| (pprof) top25 | |
| 1150ms of 1310ms total (87.79%) | |
| Showing top 25 nodes out of 87 (cum >= 20ms) | |
| flat flat% sum% cum cum% |
mschoch
/ gist:c5196096013a32525856
Created
January 12, 2015 19:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mschoch@gamer:~/go/src/github.com/blevesearch/bleve/index/upside_down$ go tool pprof upside_down.test nullindex100.profile | |
| Entering interactive mode (type "help" for commands) | |
| (pprof) top25 | |
| 1280ms of 1350ms total (94.81%) | |
| Showing top 25 nodes out of 62 (cum >= 10ms) | |
| flat flat% sum% cum cum% | |
| 200ms 14.81% 14.81% 340ms 25.19% unicode.Is | |
| 150ms 11.11% 25.93% 150ms 11.11% scanblock | |
| 140ms 10.37% 36.30% 140ms 10.37% unicode.is16 | |
| 130ms 9.63% 45.93% 130ms 9.63% github.com/blevesearch/segment.in |
mschoch
/ gist:d30fa46c4edac6837cd0
Created
January 12, 2015 20:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mschoch@gamer:~/go/src/github.com/blevesearch/bleve/index/upside_down$ go test -v -test.run=XXX -test.bench=BenchmarkNullIndexing1Workers100Batch -tags 'leveldb forestdb' -test.cpu=1 -test.cpuprofile=nullindex100.profile | |
| PASS | |
| BenchmarkNullIndexing1Workers100Batch 10 101808166 ns/op | |
| ok github.com/blevesearch/bleve/index/upside_down 1.129s | |
| mschoch@gamer:~/go/src/github.com/blevesearch/bleve/index/upside_down$ go tool pprof upside_down.test nullindex100.profile | |
| Entering interactive mode (type "help" for commands) | |
| (pprof) top25 | |
| 990ms of 1120ms total (88.39%) | |
| Showing top 25 nodes out of 67 (cum >= 20ms) | |
| flat flat% sum% cum cum% |