PGLX BoltDB statistics

18GB.pprof.alloc_objects.alloc_space.svg

18GB.pprof.sample.cpu.svg

bolt check @ 18GB (compacted, run for 12 hours)

bolt check pglx_post_compact.db
page 4561898: unreachable unfreed
page 4561899: unreachable unfreed
page 4561900: unreachable unfreed
page 4561901: unreachable unfreed
page 4561902: unreachable unfreed
page 4561903: unreachable unfreed
page 4561904: unreachable unfreed
page 4561905: unreachable unfreed
page 4561906: unreachable unfreed
page 4561907: unreachable unfreed
page 4561908: unreachable unfreed
page 4561909: unreachable unfreed
page 4561910: unreachable unfreed
page 4561911: unreachable unfreed
page 4561912: unreachable unfreed
page 4561913: unreachable unfreed
page 4561914: unreachable unfreed
page 4561915: unreachable unfreed
18 errors found
invalid value

bolt check @ 32GB

bash-4.3# bolt check pglx_precompact.db
page 4186884: unreachable unfreed
page 4274073: unreachable unfreed
page 4292652: unreachable unfreed
page 4292653: unreachable unfreed
page 4328963: unreachable unfreed
page 4425699: unreachable unfreed
page 4499620: unreachable unfreed
page 4603387: unreachable unfreed
page 4694459: unreachable unfreed
page 4760938: unreachable unfreed
page 4779888: unreachable unfreed
page 4786845: unreachable unfreed
page 4786846: unreachable unfreed
page 4786847: unreachable unfreed
page 4938715: unreachable unfreed
page 4997292: unreachable unfreed
page 5058614: unreachable unfreed
page 5164105: unreachable unfreed
page 5311991: unreachable unfreed
page 5313856: unreachable unfreed
page 5617609: unreachable unfreed
page 5866284: unreachable unfreed
page 6071266: unreachable unfreed
page 6227304: unreachable unfreed
page 6365484: unreachable unfreed
page 6462022: unreachable unfreed
page 6823106: unreachable unfreed
page 6906384: unreachable unfreed
page 7001332: unreachable unfreed
page 7113191: unreachable unfreed
page 7205368: unreachable unfreed
page 7283184: unreachable unfreed
page 7419292: unreachable unfreed
page 7566703: unreachable unfreed
page 7721746: unreachable unfreed
page 7844839: unreachable unfreed
page 7937068: unreachable unfreed
page 7983158: unreachable unfreed
page 8002600: unreachable unfreed
page 8117992: unreachable unfreed
page 8178406: unreachable unfreed
page 8229475: unreachable unfreed
page 8269082: unreachable unfreed
page 8286824: unreachable unfreed
page 8286825: unreachable unfreed
page 8287962: unreachable unfreed
page 8290214: unreachable unfreed
page 8290215: unreachable unfreed
page 8290216: unreachable unfreed
page 8290217: unreachable unfreed
page 8290218: unreachable unfreed
page 8290219: unreachable unfreed
page 8290220: unreachable unfreed
page 8290221: unreachable unfreed
page 8290222: unreachable unfreed
page 8290223: unreachable unfreed
page 8290224: unreachable unfreed
page 8290225: unreachable unfreed
page 8290226: unreachable unfreed
page 8290227: unreachable unfreed
page 8290228: unreachable unfreed
page 8290229: unreachable unfreed
page 8290230: unreachable unfreed
page 8290231: unreachable unfreed
page 8290232: unreachable unfreed
page 8290233: unreachable unfreed
page 8290234: unreachable unfreed
page 8290235: unreachable unfreed
page 8290236: unreachable unfreed
page 8290237: unreachable unfreed
page 8290238: unreachable unfreed
page 8290239: unreachable unfreed
page 8290240: unreachable unfreed
73 errors found
invalid value

bolt stats @ 18GB (compacted, run for 12 hours)

bash-4.3# bolt stats pglx_post_compact.db
Aggregate statistics for 3 buckets

Page count statistics
	Number of logical branch pages: 44441
	Number of physical branch overflow pages: 0
	Number of logical leaf pages: 2665780
	Number of physical leaf overflow pages: 137697
Tree statistics
	Number of keys/value pairs: 11617011
	Number of levels in B+tree: 7
Page size utilization
	Bytes allocated for physical branch pages: 182030336
	Bytes actually used for branch data: 87027920 (47%)
	Bytes allocated for physical leaf pages: 11483041792
	Bytes actually used for leaf data: 6440765993 (56%)
Bucket statistics
	Total number of buckets: 4985
	Total number on inlined buckets: 100 (2%)
	Bytes used for inlined buckets: 27216 (0%)

bolt stats @ 32GB

bash-4.3# bolt stats pglx_precompact.db
Aggregate statistics for 3 buckets

Page count statistics
	Number of logical branch pages: 66008
	Number of physical branch overflow pages: 0
	Number of logical leaf pages: 4018932
	Number of physical leaf overflow pages: 236002
Tree statistics
	Number of keys/value pairs: 17497257
	Number of levels in B+tree: 7
Page size utilization
	Bytes allocated for physical branch pages: 270368768
	Bytes actually used for branch data: 131202744 (48%)
	Bytes allocated for physical leaf pages: 17428209664
	Bytes actually used for leaf data: 10005395891 (57%)
Bucket statistics
	Total number of buckets: 4970
	Total number on inlined buckets: 118 (2%)
	Bytes used for inlined buckets: 30590 (0%)

DB.Stats() @ 18GB (10 second diff included)

{
  "Diff": {
    "TxStats": {
      "WriteTime": 5712911974,
      "Write": 4351,
      "SpillTime": 86445975,
      "Spill": 3767,
      "PageCount": 4059,
      "PageAlloc": 3995090944,
      "CursorCount": 2947,
      "NodeCount": 3608,
      "NodeDeref": 0,
      "Rebalance": 0,
      "RebalanceTime": 0,
      "Split": 159
    },
    "OpenTxN": 0,
    "TxN": -5,
    "FreelistInuse": 13597880,
    "FreeAlloc": 6962106368,
    "PendingPageN": 3343,
    "FreePageN": 1696390
  },
  "Cumulative": {
    "TxStats": {
      "WriteTime": 391640083251,
      "Write": 291125,
      "SpillTime": 6163144543,
      "Spill": 251465,
      "PageCount": 271295,
      "PageAlloc": 272141475840,
      "CursorCount": 196276,
      "NodeCount": 240737,
      "NodeDeref": 0,
      "Rebalance": 0,
      "RebalanceTime": 0,
      "Split": 10725
    },
    "OpenTxN": 0,
    "TxN": 341,
    "FreelistInuse": 13597880,
    "FreeAlloc": 6962106368,
    "PendingPageN": 3343,
    "FreePageN": 1696390
  }
}