Created
November 14, 2015 14:16
-
-
Save mlowicki/0e89b7b9adb530556fca to your computer and use it in GitHub Desktop.
C* CPU spikes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| INFO [MemtableFlushWriter:4646] 2015-11-14 14:13:28,670 Memtable.java:347 - Writing Memtable-compactions_in_progress@1034188977(0.008KiB serialized bytes, 1 ops, 0%/0% of on/off-heap limit) | |
| INFO [MemtableFlushWriter:4646] 2015-11-14 14:13:28,671 Memtable.java:382 - Completed flushing /var/lib/cassandra/data2/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-tmp-ka-96759-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1447057819027, position=20682425) | |
| INFO [CompactionExecutor:55229] 2015-11-14 14:13:29,290 CompactionTask.java:274 - Compacted 8 sstables to [/var/lib/cassandra/data2/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1174751,/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1174790,/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1174824,/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1174833,/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1174839,/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1174846,/var/lib/cassandra/data2/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1174855,/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1174862,]. 1,352,024,071 bytes to 1,294,822,500 (~95% of original) in 1,254,726ms = 0.984150MB/s. 64,142 total partitions merged to 60,995. Partition merge counts were {1:57848, 2:3147, } | |
| INFO [CompactionExecutor:55252] 2015-11-14 14:13:29,995 ColumnFamilyStore.java:900 - Enqueuing flush of compactions_in_progress: 756 (0%) on-heap, 0 (0%) off-heap | |
| INFO [MemtableFlushWriter:4647] 2015-11-14 14:13:29,996 Memtable.java:347 - Writing Memtable-compactions_in_progress@1334727614(0.089KiB serialized bytes, 6 ops, 0%/0% of on/off-heap limit) | |
| INFO [MemtableFlushWriter:4647] 2015-11-14 14:13:29,997 Memtable.java:382 - Completed flushing /var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-tmp-ka-96760-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=144705781 | |
| 9027, position=23342737)INFO [CompactionExecutor:55253] 2015-11-14 14:13:30,003 CompactionTask.java:141 - Compacting [SSTableReader(path='/var/lib/cassandra/data2/system/compa | |
| ctions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-96758-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-96757-Data.db'), SSTableReader(path='/var/lib/cassandra/dat | |
| a/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-96760-Data.db'), SSTableReader(path='/var/lib/cassandra/data2/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-96759-Data.db')] | |
| INFO [CompactionExecutor:55252] 2015-11-14 14:13:30,009 CompactionTask.java:141 - Compacting [SSTableReader(path='/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-1173595-Data.db')] | |
| INFO [CompactionExecutor:55253] 2015-11-14 14:13:30,064 CompactionTask.java:274 - Compacted 4 sstables to [/var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-96761,]. 3,528 bytes to 3,182 (~90% of original) in 60ms = 0.050577MB/s. | |
| 14 total partitions merged to 11. Partition merge counts were {1:12, 2:1, }INFO [ScheduledTasks:1] 2015-11-14 14:13:30,800 MessagingService.java:922 - READ messages were dropped in last 5000 ms: 911 for internal timeout and 0 | |
| for cross node timeoutINFO [ScheduledTasks:1] 2015-11-14 14:13:30,800 StatusLogger.java:51 - Pool Name Active Pending Completed Blocked All Time | |
| BlockedINFO [ScheduledTasks:1] 2015-11-14 14:13:30,801 StatusLogger.java:66 - CounterMutationStage 0 0 96679 0 | |
| 0INFO [ScheduledTasks:1] 2015-11-14 14:13:30,802 StatusLogger.java:66 - ReadStage 32 1548 952175985 0 | |
| 0INFO [ScheduledTasks:1] 2015-11-14 14:13:30,802 StatusLogger.java:66 - RequestResponseStage 0 0 1082796798 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,803 StatusLogger.java:66 - ReadRepairStage 0 0 6274304 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,803 StatusLogger.java:66 - MutationStage 0 0 253238190 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,803 StatusLogger.java:66 - GossipStage 0 0 2155694 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,804 StatusLogger.java:66 - AntiEntropyStage 0 0 11539 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,804 StatusLogger.java:66 - CacheCleanupExecutor 0 0 0 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,808 StatusLogger.java:66 - MigrationStage 0 0 21 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,808 StatusLogger.java:66 - ValidationExecutor 1 1 3736 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,809 StatusLogger.java:66 - Sampler 0 0 0 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,811 StatusLogger.java:66 - MiscStage 0 0 0 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,812 StatusLogger.java:66 - CommitLogArchiver 0 0 0 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,813 StatusLogger.java:66 - MemtableFlushWriter 0 0 34020 0 | |
| 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,813 StatusLogger.java:66 - PendingRangeCalculator 0 0 109 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,814 StatusLogger.java:66 - MemtableReclaimMemory 0 0 34020 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,814 StatusLogger.java:66 - MemtablePostFlush 0 0 42188 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,815 StatusLogger.java:66 - CompactionExecutor 11 11 325146 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,815 StatusLogger.java:66 - InternalResponseStage 0 0 1 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,816 StatusLogger.java:66 - HintedHandoff 0 0 642 0 0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,818 StatusLogger.java:75 - CompactionManager 12 28 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,820 StatusLogger.java:87 - MessagingService n/a 1/0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,822 StatusLogger.java:97 - Cache Type Size Capacity KeysToSave | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,822 StatusLogger.java:99 - KeyCache 82065338 104857600 all | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,824 StatusLogger.java:105 - RowCache 0 0 all | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,824 StatusLogger.java:112 - ColumnFamily Memtable ops,data | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,825 StatusLogger.java:115 - system.schema_triggers 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,826 StatusLogger.java:115 - system.local 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,827 StatusLogger.java:115 - system.peers 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,827 StatusLogger.java:115 - system.batchlog 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,827 StatusLogger.java:115 - system.size_estimates 138600,16563063 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,827 StatusLogger.java:115 - system.schema_usertypes 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,835 StatusLogger.java:115 - system.compactions_in_progress 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,836 StatusLogger.java:115 - system.schema_keyspaces 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,836 StatusLogger.java:115 - system.compaction_history 1349,191220 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,836 StatusLogger.java:115 - system.paxos 42,11517 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,837 StatusLogger.java:115 - system.schema_columns 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,837 StatusLogger.java:115 - system.schema_columnfamilies 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,837 StatusLogger.java:115 - system.IndexInfo 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,838 StatusLogger.java:115 - system.range_xfers 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,838 StatusLogger.java:115 - system.peer_events 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,838 StatusLogger.java:115 - system.hints 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,839 StatusLogger.java:115 - system.sstable_activity 101450,2970874 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,839 StatusLogger.java:115 - OpsCenter.events_timeline 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,840 StatusLogger.java:115 - OpsCenter.rollups7200 1043,384952 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,840 StatusLogger.java:115 - OpsCenter.settings 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,840 StatusLogger.java:115 - OpsCenter.rollups60 2248,756129 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,841 StatusLogger.java:115 - OpsCenter.rollups300 3129,1080961 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,841 StatusLogger.java:115 - OpsCenter.bestpractice_results 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,841 StatusLogger.java:115 - OpsCenter.events 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,842 StatusLogger.java:115 - OpsCenter.backup_reports 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,842 StatusLogger.java:115 - OpsCenter.pdps 2412,763552 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,842 StatusLogger.java:115 - OpsCenter.rollups86400 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,843 StatusLogger.java:115 - sync.entity_by_id2 1069421,331217343 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,844 StatusLogger.java:115 - sync.user_device_progress 353,114369 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,844 StatusLogger.java:115 - sync.user_stats 2822,756454 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,845 StatusLogger.java:115 - sync.user_store 284,38453 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,846 StatusLogger.java:115 - sync.user_quota 10976,2619504 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,847 StatusLogger.java:115 - sync.entity2 236983,131858240 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,848 StatusLogger.java:115 - sync.user_device 9726,5265340 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,848 StatusLogger.java:115 - sync.device_token 3986,1121469 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,850 StatusLogger.java:115 - system_traces.sessions 0,0 | |
| INFO [ScheduledTasks:1] 2015-11-14 14:13:30,850 StatusLogger.java:115 - system_traces.events 0,0 | |
| INFO [CompactionExecutor:55252] 2015-11-14 14:13:36,092 ColumnFamilyStore.java:900 - Enqueuing flush of compactions_in_progress: 165 (0%) on-heap, 0 (0%) off-heap | |
| INFO [MemtableFlushWriter:4646] 2015-11-14 14:13:36,096 Memtable.java:347 - Writing Memtable-compactions_in_progress@1758978520(0.008KiB serialized byt | |
| es, 1 ops, 0%/0% of on/off-heap limit) | |
| INFO [MemtableFlushWriter:4646] 2015-11-14 14:13:36,098 Memtable.java:382 - Completed flushing /var/lib/cassandra/data/system/compactions_in_progress-5 | |
| 5080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-tmp-ka-96762-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=144705781 | |
| 9028, position=1438438) | |
| INFO [CompactionExecutor:55252] 2015-11-14 14:13:36,331 CompactionTask.java:274 - Compacted 1 sstables to [/var/lib/cassandra/data/sync/entity2-e24b504 | |
| 0199b11e5a30f75bb514ae072/sync-entity2-ka-1174866,]. 6,547,501 bytes to 6,547,501 (~100% of original) in 6,321ms = 0.987847MB/s. 223 total partitions | |
| merged to 223. Partition merge counts were {1:223, } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment