simple benchmarking SQLite3 with Go and go-sqlite3 lib
licence of materials on this gist are MIT license
Copylight 2023 Ryo Kanbayashi
Ryo Kanbayashi ryogrid
ryogrid
/ parallel_txn_insert_after_select_cpu_prof.txt
Last active
October 13, 2023 10:43
10万エントリを並列にINSERTして、その後、その中からランダムに並列にSELECTした時のcpu profile
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (pprof) top200 | |
| Showing nodes accounting for 130.23s, 82.06% of 158.71s total | |
| Dropped 688 nodes (cum <= 0.79s) | |
| flat flat% sum% cum cum% | |
| 16.15s 10.18% 10.18% 16.17s 10.19% runtime.stdcall2 | |
| 13s 8.19% 18.37% 13s 8.19% runtime._ExternalCode | |
| 5.61s 3.53% 21.90% 20.12s 12.68% runtime.mallocgc | |
| 4.54s 2.86% 24.76% 20.85s 13.14% runtime.scanobject | |
| 4.08s 2.57% 27.33% 4.09s 2.58% runtime.stdcall1 | |
| 3.79s 2.39% 29.72% 8.55s 5.39% runtime.findObject |
This file has been truncated, but you can view the full file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "buster": "", | |
| "timestamp": 1696058788895, | |
| "clientState": { | |
| "mutations": [], | |
| "queries": [ | |
| { | |
| "state": { | |
| "data": { | |
| "content": { |
ryogrid
/ result_of_unittest_230922_1225.txt
Created
September 24, 2023 03:25
result of unittest of Optimizer (Selinger) 230922_1225
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| === RUN TestSimplePlanOptimization | |
| System logging is active. | |
| Pattern Name: Simple(SequentialScan) | |
| [ select Sc1.c1 from Sc1 where Sc1.c1 = 2; ] | |
| ================================================== | |
| ProjectionPlanNode [sc1.c1, ] | |
| ProjectionPlanNode [sc1.c1, ] | |
| SelectionPlanNode [ colIndex:0 2 = ] | |
| SeqScanPlanNode [sc1.c1, sc1.c2, sc1.c3, ] |
ryogrid
/ passed_optimization_patterns.txt
Created
September 23, 2023 09:26
passed optimization patterns 20230923_1825
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| === RUN TestSimplePlanOptimization | |
| System logging is active. | |
| Pattern Name: Simple(SequentialScan) | |
| [ select Sc1.c1 from Sc1 where Sc1.c1 = 2; ] | |
| ================================================== | |
| ProjectionPlanNode [sc1.c1, ] | |
| ProjectionPlanNode [sc1.c1, ] | |
| SelectionPlanNode [ colIndex:0 2 = ] | |
| SeqScanPlanNode [sc1.c1, sc1.c2, sc1.c3, ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| NOTE: linux environment is WSL1 on Windows10 | |
| # based benchmark code: n-body (added little modifications by ryo_grid for excecuting same logic) | |
| https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-gcc-3.html | |
| https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-go-3.html | |
| ryo@DESKTOP-IOASPN6:/mnt/c/Data/work/go-bench$ uname -a | |
| Linux DESKTOP-IOASPN6 4.4.0-19041-Microsoft #1237-Microsoft Sat Sep 11 14:32:00 PST 2021 x86_64 x86_64 x86_64 GNU/Linux |
ryogrid
/ go-comiler-comparison.txt
Last active
August 16, 2023 08:56
go compiler coparison: gc go1.20.7 linux/amd64 vs gccgo (Ubuntu 10.5.0-1ubuntu1~20.04) 10.5.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| NOTE: linux environment is WSL1 on Windows10 | |
| ryo@DESKTOP-IOASPN6:/mnt/c/Data/work/go-bench$ uname -a | |
| Linux DESKTOP-IOASPN6 4.4.0-19041-Microsoft #1237-Microsoft Sat Sep 11 14:32:00 PST 2021 x86_64 x86_64 x86_64 GNU/Linux | |
| ryo@DESKTOP-IOASPN6:/mnt/c/Data/work/go-bench$ go version | |
| go version go1.20.7 linux/amd64 | |
| ryo@DESKTOP-IOASPN6:/mnt/c/Data/work/go-bench$ gccgo --version | |
| gccgo (Ubuntu 10.5.0-1ubuntu1~20.04) 10.5.0 |
ryogrid
/ NostrNIP36ImageRedirector_EN.md
Last active
August 30, 2023 23:56
For posting images includng sensitive content with considering of non-NIP-36 compliant clients
The NIP-36 specification requires that post be tagged with a content-warning tag (so that users can clearly identify the post as such when they post it). However, even if this is done, non-NIP-36 compliant clients still display tagged posts as usual.
As a result, although overall compliance with the specification is progressing, a non-negligible percentage of clients do not support NIP-36, leaving users with a psychological barrier that makes it difficult to post Sensitive Content.
Therefore, the purpose of the specification is to avoid users of non-NIP-36 compliant clients from unintentionally viewing content with content-warning tags, and to allow Nostr users to think that it is safe to post Sensitive Content if they specify content-warning tags.
The following service was developed to avoid unintentional viewing of content with content-warning tags and to make Nostr users feel it is safe to post sensitive content with specifing content-warning tag.
ryogrid
/ todo_opt.txt
Last active
September 24, 2023 00:59
TODO comments of optimizer impl on SamehadaDB codebase
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ./execution/plans/aggregation.go | |
| 103: // TODO: (SDB) [OPT] not implemented yet (AggregationPlanNode::GetDebugStr) | |
| ./execution/plans/delete.go | |
| 36: // TODO: (SDB) [OPT] not implemented yet (DeletePlanNode::GetDebugStr) | |
| ./execution/plans/insert.go | |
| 56: // TODO: (SDB) [OPT] not implemented yet (InsertPlanNode::GetDebugStr) | |
| ./execution/plans/limit.go | |
| 46: // TODO: (SDB) [OPT] not implemented yet (LimitPlanNode::GetDebugStr) | |
| ./execution/plans/orderby.go | |
| 80: // TODO: (SDB) [OPT] not implemented yet (OrderbyPlanNode::GetDebugStr) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ■SamehadaDBのTODO | |
| ・【済】TransactionManager?のBeginとCommitを現在のexecutorのテストケースではどう暫定実装しているのか確認 | |
| ・【済】executor系のテストやloggingのテストでTransactionManagerのBeginとCommitが呼び出されるようにする | |
| ・【済】TransactionManagerクラスとTransactionクラスの実装 | |
| ・【済】"// TODO(student): add logging here" と cpp実装でコメントされている箇所のif文をSamehadaDBのgoのコードベースにも加える | |
| ・【済】logging/recoveryの実装 | |
| -【済】C++実装版の非同期I/Oの仕組みをチェックしておく（別のBusTubフォークを参照する形かな） | |
| -【済】flush thread が logging に必須であるか確認し、必要なら対応する | |
| -【済】recovery_test のテストケースのセットアップをSamehadaInstanceを使う形にしたが、他のテストケースと比較して、 | |
| 必要な処理が行われているかを確認・対応 |