Skip to content

Instantly share code, notes, and snippets.

@smira
Last active August 29, 2015 14:03
Show Gist options
  • Save smira/38808d336f3778aacceb to your computer and use it in GitHub Desktop.
Save smira/38808d336f3778aacceb to your computer and use it in GitHub Desktop.
goleveldb possible lost keys
db1.* - initial database after creation
db2.* - after compaction
db3.* - dropped CURRENT file, after recover, some keys lost
db4.* - after recover without compaction, all keys present
=============== Jun 28, 2014 (MSK) ===============
01:24:11.827657 db@open opening
01:24:11.827857 journal@recovery F·1
01:24:11.828189 journal@recovery recovering @475
01:24:11.828843 mem@flush created L0@477 N·1 S·2KiB "R..f,v417783":"R..f,v417783"
01:24:11.829491 db@janitor F·35 G·0
01:24:11.829508 db@open done T·1.76881ms
01:24:12.276429 table@compaction L0·2 -> L1·4 S·5MiB Q·417924
01:24:12.339337 table@build created L1@480 N·3 S·2MiB "E..4,v408626":"E..6,v405600"
01:24:12.365772 table@build created L1@481 N·3340 S·738KiB "E..d,v417725":"x..5,v405849"
01:24:12.406303 table@build created L1@482 N·4350 S·953KiB "x..b,v411745":"x..0,v417754"
01:24:12.460338 table@build created L1@483 N·3944 S·2MiB "x..0,v414193":"x..d,v392168"
01:24:12.460738 table@build created L1@484 N·6 S·3KiB "x..7,v392172":"x..a,v392192"
01:24:12.461861 table@compaction commited F-1 S-7KiB D·13 T·185.340609ms
01:24:12.664499 table@compaction expanding L1+L2 (F·1 S·738KiB)+(F·10 S·20MiB) -> (F·2 S·3MiB)+(F·10 S·20MiB)
01:24:12.664512 table@compaction L1·2 -> L2·10 S·23MiB Q·418232
01:24:12.767701 table@build created L2@485 N·6 S·4MiB "E..2,v264924":"E..6,v405600"
01:24:12.898655 table@build created L2@486 N·25085 S·2MiB "E..d,v417725":"P..f,v188106"
01:24:12.963766 db@close closing
01:24:12.966457 table@build exiting
01:24:12.966481 table@build rollback @485
01:24:12.966849 table@build rollback @486
01:24:12.968752 db@close done T·4.981259ms
=============== Jun 28, 2014 (MSK) ===============
01:24:11.795156 db@open opening
01:24:11.795372 journal@recovery F·1
01:24:11.795801 journal@recovery recovering @468
01:24:11.797159 mem@flush created L0@474 N·18 S·8KiB "E..6,v417781":"x..f,v417777"
01:24:11.798175 db@janitor F·34 G·0
01:24:11.798206 db@open done T·2.963479ms
01:24:11.818296 db@close closing
01:24:11.818409 db@close done T·109.982us
=============== Jun 28, 2014 (MSK) ===============
01:25:09.808626 db@open opening
01:25:09.808825 journal@recovery F·1
01:25:09.809196 journal@recovery recovering @478
01:25:09.814623 mem@flush created L0@485 N·618 S·61KiB "E..f,v418402":"x..2,v418142"
01:25:09.816453 db@janitor F·35 G·0
01:25:09.816475 db@open done T·7.76795ms
01:25:10.052161 table@compaction L0·1 -> L1·3 S·4MiB Q·418403
01:25:10.149934 table@build created L1@488 N·4 S·2MiB "E..4,v408626":"E..6,v405600"
01:25:10.202691 table@build created L1@489 N·3648 S·752KiB "E..d,v417725":"x..5,v405849"
01:25:10.263426 table@build created L1@490 N·4581 S·981KiB "x..b,v411745":"x..0,v417754"
01:25:10.264252 table@build created L1@491 N·77 S·14KiB "x..5,v417838":"x..2,v418142"
01:25:10.265048 table@compaction commited F~ S-711B D·1 T·212.858203ms
01:25:10.283597 table@compaction L1·1 -> L2·11 S·21MiB Q·418403
01:25:10.432953 table@build created L2@492 N·10894 S·2MiB "x..0,v306033":"x..9,v412872"
01:25:10.609387 table@build created L2@493 N·10128 S·2MiB "x..a,v320071":"x..d,v266330"
01:25:10.745163 table@build created L2@494 N·10648 S·2MiB "x..2,v266859":"x..8,v288279"
01:25:10.843924 table@build created L2@495 N·10464 S·2MiB "x..6,v288283":"x..d,v300312"
01:25:11.004268 table@build created L2@496 N·8987 S·2MiB "x..2,v300315":"x..a,v369354"
01:25:11.151475 table@build created L2@497 N·8697 S·2MiB "x..b,v369358":"x..6,v406511"
01:25:11.280695 table@build created L2@498 N·10186 S·2MiB "x..4,v401498":"x..6,v74200"
01:25:11.384982 table@build created L2@499 N·10201 S·2MiB "x..7,v74204":"x..a,v157069"
01:25:11.514332 table@build created L2@500 N·10260 S·2MiB "x..c,v256970":"x..9,v193656"
01:25:11.668173 table@build created L2@501 N·10255 S·2MiB "x..4,v211225":"x..5,v1440"
01:25:11.717966 table@build created L2@502 N·4402 S·897KiB "x..3,v1444":"x..f,v47844"
01:25:11.718254 table@compaction commited F-1 S-665KiB D·3534 T·1.43458876s
01:25:11.732640 table@compaction expanding L1+L2 (F·1 S·752KiB)+(F·10 S·20MiB) -> (F·2 S·3MiB)+(F·10 S·20MiB)
01:25:11.732659 table@compaction L1·2 -> L2·10 S·23MiB Q·418403
01:25:11.900842 table@build created L2@503 N·7 S·4MiB "E..2,v264924":"E..6,v405600"
01:25:12.133132 table@build created L2@504 N·25085 S·2MiB "E..d,v417725":"P..f,v188106"
01:25:12.365867 table@build created L2@505 N·28104 S·2MiB "P..a,v188118":"P..0,v122563"
01:25:12.580455 table@build created L2@506 N·27235 S·2MiB "P..6,v122567":"R..d,v240372"
01:25:12.739527 table@build created L2@507 N·19319 S·2MiB "R..d,v241772":"x..2,v256919"
01:25:12.957102 table@build created L2@508 N·18942 S·2MiB "x..4,v157022":"x..0,v232187"
01:25:13.148611 table@build created L2@509 N·18839 S·2MiB "x..3,v232194":"x..f,v101697"
01:25:13.361650 table@build created L2@510 N·18217 S·2MiB "x..7,v252773":"x..1,v386557"
01:25:13.487829 table@build created L2@511 N·13386 S·2MiB "x..2,v386561":"x..1,v322684"
01:25:13.635555 table@build created L2@512 N·10682 S·2MiB "x..c,v322685":"x..1,v304541"
01:25:13.640315 table@build created L2@513 N·552 S·126KiB "x..c,v304542":"x..c,v306026"
01:25:13.654781 table@compaction commited F-1 S-385KiB D·1709 T·1.907838934s
01:25:14.034870 table@compaction expanding L1+L2 (F·1 S·2MiB)+(F·3 S·6MiB) -> (F·2 S·2MiB)+(F·3 S·6MiB)
01:25:14.034894 table@compaction L1·2 -> L2·3 S·8MiB Q·418403
01:25:14.105804 table@build created L2@514 N·5596 S·2MiB "x..b,v120740":"x..4,v414686"
01:25:14.197586 table@build created L2@515 N·3886 S·2MiB "x..7,v348052":"x..9,v362580"
01:25:14.301004 table@build created L2@516 N·3907 S·2MiB "x..3,v362584":"x..0,v377732"
01:25:14.395273 table@build created L2@517 N·3962 S·2MiB "x..e,v377736":"x..5,v392096"
01:25:14.395870 table@build created L2@518 N·18 S·9KiB "x..d,v392100":"x..d,v392168"
01:25:14.406658 table@compaction commited F~ S-1KiB D·0 T·371.739174ms
01:25:14.957117 range compaction L-1 <nil>:<nil>
01:25:14.957143 table@compaction L1·2 -> L2·0 S·1MiB Q·418403
01:25:14.996249 table@build created L2@519 N·3617 S·1MiB "x..7,v392172":"x..f,v405596"
01:25:14.996935 table@compaction commited F-1 S-118B D·0 T·39.771121ms
01:25:14.996950 db@close closing
01:25:14.997369 db@close done T·418.113us
=============== Jun 28, 2014 (MSK) ===============
01:24:11.827657 db@open opening
01:24:11.827857 journal@recovery F·1
01:24:11.828189 journal@recovery recovering @475
01:24:11.828843 mem@flush created L0@477 N·1 S·2KiB "R..f,v417783":"R..f,v417783"
01:24:11.829491 db@janitor F·35 G·0
01:24:11.829508 db@open done T·1.76881ms
01:24:12.276429 table@compaction L0·2 -> L1·4 S·5MiB Q·417924
01:24:12.339337 table@build created L1@480 N·3 S·2MiB "E..4,v408626":"E..6,v405600"
01:24:12.365772 table@build created L1@481 N·3340 S·738KiB "E..d,v417725":"x..5,v405849"
01:24:12.406303 table@build created L1@482 N·4350 S·953KiB "x..b,v411745":"x..0,v417754"
01:24:12.460338 table@build created L1@483 N·3944 S·2MiB "x..0,v414193":"x..d,v392168"
01:24:12.460738 table@build created L1@484 N·6 S·3KiB "x..7,v392172":"x..a,v392192"
01:24:12.461861 table@compaction commited F-1 S-7KiB D·13 T·185.340609ms
01:24:12.664499 table@compaction expanding L1+L2 (F·1 S·738KiB)+(F·10 S·20MiB) -> (F·2 S·3MiB)+(F·10 S·20MiB)
01:24:12.664512 table@compaction L1·2 -> L2·10 S·23MiB Q·418232
01:24:12.767701 table@build created L2@485 N·6 S·4MiB "E..2,v264924":"E..6,v405600"
01:24:12.898655 table@build created L2@486 N·25085 S·2MiB "E..d,v417725":"P..f,v188106"
01:24:12.963766 db@close closing
01:24:12.966457 table@build exiting
01:24:12.966481 table@build rollback @485
01:24:12.966849 table@build rollback @486
01:24:12.968752 db@close done T·4.981259ms
=============== Jun 28, 2014 (MSK) ===============
01:26:20.116204 db@open opening
01:26:20.116405 journal@recovery F·1
01:26:20.116750 journal@recovery recovering @523
01:26:20.118063 db@janitor F·32 G·0
01:26:20.118079 db@open done T·1.790622ms
01:26:20.126664 table@move L0@503 -> L1
01:26:20.126934 table@move L0@504 -> L1
01:26:20.127176 table@move L0@505 -> L1
01:26:20.127375 table@move L0@506 -> L1
01:26:20.127559 table@move L0@507 -> L1
01:26:20.127748 table@move L0@508 -> L1
01:26:20.127920 table@move L0@509 -> L1
01:26:20.128080 table@move L0@510 -> L1
01:26:20.139844 db@close closing
01:26:20.140006 db@close done T·159.188us
=============== Jun 28, 2014 (MSK) ===============
01:26:16.847178 db@recovery F·32
01:26:16.847498 db@recovery found table @404 S·2MiB "x..d,v248284":"x..5,v81244"
01:26:16.848010 db@recovery found table @405 S·1001KiB "x..f,v138808":"x..7,v120736"
01:26:16.848254 db@recovery found table @492 S·2MiB "x..0,v306033":"x..9,v412872"
01:26:16.848432 db@recovery found table @493 S·2MiB "x..a,v320071":"x..d,v266330"
01:26:16.848701 db@recovery found table @494 S·2MiB "x..2,v266859":"x..8,v288279"
01:26:16.848897 db@recovery found table @495 S·2MiB "x..6,v288283":"x..d,v300312"
01:26:16.849135 db@recovery found table @496 S·2MiB "x..2,v300315":"x..a,v369354"
01:26:16.849357 db@recovery found table @497 S·2MiB "x..b,v369358":"x..6,v406511"
01:26:16.849927 db@recovery found table @498 S·2MiB "x..4,v401498":"x..6,v74200"
01:26:16.850187 db@recovery found table @499 S·2MiB "x..7,v74204":"x..a,v157069"
01:26:16.850373 db@recovery found table @500 S·2MiB "x..c,v256970":"x..9,v193656"
01:26:16.850571 db@recovery found table @501 S·2MiB "x..4,v211225":"x..5,v1440"
01:26:16.850727 db@recovery found table @502 S·897KiB "x..3,v1444":"x..f,v47844"
01:26:16.867006 db@recovery found table @503 S·4MiB "E..2,v264924":"E..6,v405600"
01:26:16.868816 db@recovery found table @504 S·2MiB "E..d,v417725":"P..f,v188106"
01:26:16.869076 db@recovery found table @505 S·2MiB "P..a,v188118":"P..0,v122563"
01:26:16.869637 db@recovery found table @506 S·2MiB "P..6,v122567":"R..d,v240372"
01:26:16.870209 db@recovery found table @507 S·2MiB "R..d,v241772":"x..2,v256919"
01:26:16.870436 db@recovery found table @508 S·2MiB "x..4,v157022":"x..0,v232187"
01:26:16.870684 db@recovery found table @509 S·2MiB "x..3,v232194":"x..f,v101697"
01:26:16.870900 db@recovery found table @510 S·2MiB "x..7,v252773":"x..1,v386557"
01:26:16.871095 db@recovery found table @511 S·2MiB "x..2,v386561":"x..1,v322684"
01:26:16.871290 db@recovery found table @512 S·2MiB "x..c,v322685":"x..1,v304541"
01:26:16.871376 db@recovery found table @513 S·126KiB "x..c,v304542":"x..c,v306026"
01:26:16.871514 db@recovery found table @514 S·2MiB "x..b,v120740":"x..4,v414686"
01:26:16.871652 db@recovery found table @515 S·2MiB "x..7,v348052":"x..9,v362580"
01:26:16.871798 db@recovery found table @516 S·2MiB "x..3,v362584":"x..0,v377732"
01:26:16.871921 db@recovery found table @517 S·2MiB "x..e,v377736":"x..5,v392096"
01:26:16.871981 db@recovery found table @518 S·9KiB "x..d,v392100":"x..d,v392168"
01:26:16.872116 db@recovery found table @519 S·1MiB "x..7,v392172":"x..f,v405596"
01:26:16.880963 db@open opening
01:26:16.881141 journal@recovery F·1
01:26:16.881193 journal@recovery recovering @520
01:26:16.882368 db@janitor F·33 G·1
01:26:16.882383 db@janitor removing manifest-521
01:26:16.882483 db@open done T·1.503562ms
01:26:16.882499 db@close closing
01:26:16.882546 table@move L0@503 -> L1
01:26:16.882556 table@move exiting
01:26:16.882835 db@close done T·334.041us
=============== Jun 28, 2014 (MSK) ===============
21:18:05.275338 db@recovery F·34
21:18:05.276078 db@recovery found table @368 S·2MiB "x..4,v315270":"x..c,v306026"
21:18:05.276372 db@recovery found table @369 S·2MiB "x..0,v306033":"x..6,v260037"
21:18:05.276573 db@recovery found table @400 S·2MiB "x..7,v39732":"x..b,v20416"
21:18:05.276755 db@recovery found table @401 S·2MiB "x..8,v20424":"x..8,v148065"
21:18:05.276960 db@recovery found table @402 S·2MiB "x..6,v148069":"x..3,v206428"
21:18:05.277134 db@recovery found table @403 S·2MiB "x..8,v206435":"x..f,v47844"
21:18:05.277304 db@recovery found table @404 S·2MiB "x..d,v248284":"x..5,v81244"
21:18:05.277440 db@recovery found table @405 S·1001KiB "x..f,v138808":"x..7,v120736"
21:18:05.277595 db@recovery found table @415 S·2MiB "x..b,v120740":"x..6,v348844"
21:18:05.278093 db@recovery found table @416 S·2MiB "x..8,v348848":"x..d,v364248"
21:18:05.278268 db@recovery found table @417 S·2MiB "x..d,v364252":"x..b,v379924"
21:18:05.279247 db@recovery found table @437 S·2MiB "E..2,v264924":"P..3,v6343"
21:18:05.279484 db@recovery found table @438 S·2MiB "P..b,v6371":"P..d,v204900"
21:18:05.279723 db@recovery found table @439 S·2MiB "P..d,v204909":"P..3,v102371"
21:18:05.280492 db@recovery found table @440 S·2MiB "P..8,v102391":"R..f,v244685"
21:18:05.281113 db@recovery found table @441 S·2MiB "R..a,v408554":"x..f,v190294"
21:18:05.281307 db@recovery found table @442 S·2MiB "x..9,v178320":"x..5,v14025"
21:18:05.281523 db@recovery found table @443 S·2MiB "x..5,v14029":"x..b,v127541"
21:18:05.281726 db@recovery found table @444 S·2MiB "x..a,v127545":"x..b,v403685"
21:18:05.281965 db@recovery found table @445 S·1MiB "x..5,v403689":"x..9,v319368"
21:18:05.282144 db@recovery found table @446 S·2MiB "x..8,v320094":"x..7,v266635"
21:18:05.282338 db@recovery found table @447 S·2MiB "x..3,v266918":"x..9,v283265"
21:18:05.282536 db@recovery found table @448 S·2MiB "x..f,v283266":"x..6,v299899"
21:18:05.282755 db@recovery found table @449 S·2MiB "x..2,v300422":"x..3,v371158"
21:18:05.283012 db@recovery found table @450 S·2MiB "x..5,v371162":"x..5,v133520"
21:18:05.283104 db@recovery found table @453 S·578KiB "x..5,v4708":"x..e,v39728"
21:18:05.283233 db@recovery found table @473 S·1MiB "x..f,v392196":"x..f,v405596"
21:18:05.296989 db@recovery found table @480 S·2MiB "E..4,v408626":"E..6,v405600"
21:18:05.298233 db@recovery found table @481 S·738KiB "E..d,v417725":"x..5,v405849"
21:18:05.298360 db@recovery found table @482 S·953KiB "x..b,v411745":"x..0,v417754"
21:18:05.298516 db@recovery found table @483 S·2MiB "x..0,v414193":"x..d,v392168"
21:18:05.298561 db@recovery found table @484 S·3KiB "x..7,v392172":"x..a,v392192"
21:18:05.299497 db@open opening
21:18:05.299669 journal@recovery F·1
21:18:05.300194 journal@recovery recovering @478
21:18:05.303403 mem@flush created L0@486 N·618 S·60KiB "E..f,v418402":"x..2,v418142"
21:18:05.304094 db@janitor F·36 G·1
21:18:05.304104 db@janitor removing manifest-479
21:18:05.304226 db@open done T·4.716802ms
21:18:05.304235 db@close closing
21:18:05.304386 db@close done T·150.556us
=============== Jun 28, 2014 (MSK) ===============
01:24:11.827657 db@open opening
01:24:11.827857 journal@recovery F·1
01:24:11.828189 journal@recovery recovering @475
01:24:11.828843 mem@flush created L0@477 N·1 S·2KiB "R..f,v417783":"R..f,v417783"
01:24:11.829491 db@janitor F·35 G·0
01:24:11.829508 db@open done T·1.76881ms
01:24:12.276429 table@compaction L0·2 -> L1·4 S·5MiB Q·417924
01:24:12.339337 table@build created L1@480 N·3 S·2MiB "E..4,v408626":"E..6,v405600"
01:24:12.365772 table@build created L1@481 N·3340 S·738KiB "E..d,v417725":"x..5,v405849"
01:24:12.406303 table@build created L1@482 N·4350 S·953KiB "x..b,v411745":"x..0,v417754"
01:24:12.460338 table@build created L1@483 N·3944 S·2MiB "x..0,v414193":"x..d,v392168"
01:24:12.460738 table@build created L1@484 N·6 S·3KiB "x..7,v392172":"x..a,v392192"
01:24:12.461861 table@compaction commited F-1 S-7KiB D·13 T·185.340609ms
01:24:12.664499 table@compaction expanding L1+L2 (F·1 S·738KiB)+(F·10 S·20MiB) -> (F·2 S·3MiB)+(F·10 S·20MiB)
01:24:12.664512 table@compaction L1·2 -> L2·10 S·23MiB Q·418232
01:24:12.767701 table@build created L2@485 N·6 S·4MiB "E..2,v264924":"E..6,v405600"
01:24:12.898655 table@build created L2@486 N·25085 S·2MiB "E..d,v417725":"P..f,v188106"
01:24:12.963766 db@close closing
01:24:12.966457 table@build exiting
01:24:12.966481 table@build rollback @485
01:24:12.966849 table@build rollback @486
01:24:12.968752 db@close done T·4.981259ms
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment