Skip to content

Instantly share code, notes, and snippets.

@agibsonccc

agibsonccc/gist:072a9a4eb117abb30f92af22a2ce849c

Created October 24, 2016 04:53
Show Gist options
  • Save agibsonccc/072a9a4eb117abb30f92af22a2ce849c to your computer and use it in GitHub Desktop.
Save agibsonccc/072a9a4eb117abb30f92af22a2ce849c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
WARNING: The LD_LIBRARY_PATH variable is not set. Defaulting to a blank string.
Starting demo_spark-master_1
Starting demo_zookeeper_1
Starting demo_elasticsearch_1
Starting demo_cassandra_1
Starting demo_hdfsyarn_1
Starting demo_spark-slave_1
Starting demo_logstash_1
Starting demo_elastichq_1
Starting demo_streamsets_1
Starting demo_hdfsyarndata1_1
Starting demo_kibana_1
Creating demo_kafka_1
Attaching to demo_spark-master_1, demo_zookeeper_1, demo_elasticsearch_1, demo_hdfsyarn_1, demo_cassandra_1, demo_spark-slave_1, demo_elastichq_1, demo_hdfsyarndata1_1, demo_logstash_1, demo_streamsets_1, demo_kibana_1, demo_kafka_1
zookeeper_1 | JMX enabled by default
zookeeper_1 | Using config: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
zookeeper_1 | 2016-10-24 04:50:05,393 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
hdfsyarn_1 | /
zookeeper_1 | 2016-10-24 04:50:06,173 [myid:] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
hdfsyarn_1 | Redirecting to /bin/systemctl start sshd.service
hdfsyarn_1 | WARNING: 'UsePAM no' is not supported in Red Hat Enterprise Linux and may cause several problems.
zookeeper_1 | 2016-10-24 04:50:06,173 [myid:] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
zookeeper_1 | 2016-10-24 04:50:06,173 [myid:] - INFO [main:DatadirCleanupManager@101] - Purge task is not scheduled.
hdfsyarn_1 | Could not load host key: /etc/ssh/ssh_host_ecdsa_key
hdfsyarn_1 | Could not load host key: /etc/ssh/ssh_host_ed25519_key
hdfsyarndata1_1 | /
zookeeper_1 | 2016-10-24 04:50:06,173 [myid:] - WARN [main:QuorumPeerMain@113] - Either no config or no quorum defined in config, running in standalone mode
hdfsyarn_1 | Failed to get D-Bus connection: Operation not permitted
hdfsyarndata1_1 | WARNING: 'UsePAM no' is not supported in Red Hat Enterprise Linux and may cause several problems.
zookeeper_1 | 2016-10-24 04:50:06,625 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
zookeeper_1 | 2016-10-24 04:50:06,626 [myid:] - INFO [main:ZooKeeperServerMain@95] - Starting server
streamsets_1 | Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
hdfsyarndata1_1 | Could not load host key: /etc/ssh/ssh_host_ecdsa_key
streamsets_1 | Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
hdfsyarndata1_1 | Could not load host key: /etc/ssh/ssh_host_ed25519_key
zookeeper_1 | 2016-10-24 04:50:07,208 [myid:] - INFO [main:Environment@100] - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
streamsets_1 | Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
kibana_1 | Stalling for Elasticsearch
zookeeper_1 | 2016-10-24 04:50:07,209 [myid:] - INFO [main:Environment@100] - Server environment:host.name=70e456521a7a
streamsets_1 | Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=1024m; support was removed in 8.0
hdfsyarndata1_1 | Redirecting to /bin/systemctl start sshd.service
zookeeper_1 | 2016-10-24 04:50:07,209 [myid:] - INFO [main:Environment@100] - Server environment:java.version=1.7.0_55
hdfsyarndata1_1 | Failed to get D-Bus connection: Operation not permitted
zookeeper_1 | 2016-10-24 04:50:07,209 [myid:] - INFO [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
zookeeper_1 | 2016-10-24 04:50:07,209 [myid:] - INFO [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
zookeeper_1 | 2016-10-24 04:50:07,209 [myid:] - INFO [main:Environment@100] - Server environment:java.class.path=/opt/zookeeper-3.4.6/bin/../build/classes:/opt/zookeeper-3.4.6/bin/../build/lib/*.jar:/opt/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/opt/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.6/bin/../conf:
zookeeper_1 | 2016-10-24 04:50:07,209 [myid:] - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
zookeeper_1 | 2016-10-24 04:50:07,209 [myid:] - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp
zookeeper_1 | 2016-10-24 04:50:07,210 [myid:] - INFO [main:Environment@100] - Server environment:java.compiler=<NA>
zookeeper_1 | 2016-10-24 04:50:07,211 [myid:] - INFO [main:Environment@100] - Server environment:os.name=Linux
zookeeper_1 | 2016-10-24 04:50:07,211 [myid:] - INFO [main:Environment@100] - Server environment:os.arch=amd64
zookeeper_1 | 2016-10-24 04:50:07,211 [myid:] - INFO [main:Environment@100] - Server environment:os.version=4.7.8-100.fc23.x86_64
zookeeper_1 | 2016-10-24 04:50:07,212 [myid:] - INFO [main:Environment@100] - Server environment:user.name=root
zookeeper_1 | 2016-10-24 04:50:07,212 [myid:] - INFO [main:Environment@100] - Server environment:user.home=/root
zookeeper_1 | 2016-10-24 04:50:07,212 [myid:] - INFO [main:Environment@100] - Server environment:user.dir=/
zookeeper_1 | 2016-10-24 04:50:07,269 [myid:] - INFO [main:ZooKeeperServer@755] - tickTime set to 2000
zookeeper_1 | 2016-10-24 04:50:07,269 [myid:] - INFO [main:ZooKeeperServer@764] - minSessionTimeout set to -1
zookeeper_1 | 2016-10-24 04:50:07,269 [myid:] - INFO [main:ZooKeeperServer@773] - maxSessionTimeout set to -1
zookeeper_1 | 2016-10-24 04:50:08,175 [myid:] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181
spark-master_1 | 16/10/24 04:50:26 INFO master.Master: Registered signal handlers for [TERM, HUP, INT]
elasticsearch_1 | [2016-10-24 04:50:34,073][INFO ][node ] [Blizzard] version[1.7.3], pid[1], build[05d4530/2015-10-15T09:14:17Z]
elasticsearch_1 | [2016-10-24 04:50:34,184][INFO ][node ] [Blizzard] initializing ...
spark-slave_1 | 16/10/24 04:50:34 INFO worker.Worker: Registered signal handlers for [TERM, HUP, INT]
elasticsearch_1 | [2016-10-24 04:50:40,193][INFO ][plugins ] [Blizzard] loaded [], sites []
elasticsearch_1 | [2016-10-24 04:50:42,095][INFO ][env ] [Blizzard] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/fedora-home)]], net usable_space [1.8tb], net total_space [2.1tb], types [ext4]
spark-master_1 | 16/10/24 04:50:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
spark-master_1 | 16/10/24 04:50:47 INFO spark.SecurityManager: Changing view acls to: root
spark-master_1 | 16/10/24 04:50:47 INFO spark.SecurityManager: Changing modify acls to: root
spark-master_1 | 16/10/24 04:50:47 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
spark-slave_1 | 16/10/24 04:50:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
spark-slave_1 | 16/10/24 04:50:52 INFO spark.SecurityManager: Changing view acls to: root
spark-slave_1 | 16/10/24 04:50:52 INFO spark.SecurityManager: Changing modify acls to: root
spark-slave_1 | 16/10/24 04:50:52 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
hdfsyarn_1 | Starting namenodes on [0.0.0.0]
cassandra_1 | INFO 04:50:57 Configuration location: file:/etc/cassandra/cassandra.yaml
spark-master_1 | 16/10/24 04:51:06 INFO util.Utils: Successfully started service 'sparkMaster' on port 7077.
spark-master_1 | 16/10/24 04:51:07 INFO master.Master: Starting Spark master at spark://172.21.0.2:7077
cassandra_1 | INFO 04:51:07 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=172.21.0.6; broadcast_rpc_address=172.21.0.6; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/var/lib/cassandra/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@7219ec67; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=dc; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=172.21.0.6; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=256; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/var/lib/cassandra/saved_caches; seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=172.21.0.6}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@45018215; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
cassandra_1 | INFO 04:51:07 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
spark-master_1 | 16/10/24 04:51:07 INFO master.Master: Running Spark version 1.6.2
spark-slave_1 | 16/10/24 04:51:09 INFO util.Utils: Successfully started service 'sparkWorker' on port 40225.
hdfsyarndata1_1 | Starting namenodes on [0.0.0.0]
hdfsyarn_1 | 0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts.
hdfsyarn_1 | 0.0.0.0: starting namenode, logging to /usr/local/hadoop/logs/hadoop-root-namenode-e2c548d76327.out
cassandra_1 | INFO 04:51:11 Global memtable on-heap threshold is enabled at 1988MB
cassandra_1 | INFO 04:51:11 Global memtable off-heap threshold is enabled at 1988MB
spark-master_1 | 16/10/24 04:51:13 INFO server.Server: jetty-8.y.z-SNAPSHOT
spark-master_1 | 16/10/24 04:51:13 INFO server.AbstractConnector: Started [email protected]:8280
spark-master_1 | 16/10/24 04:51:13 INFO util.Utils: Successfully started service 'MasterUI' on port 8280.
spark-master_1 | 16/10/24 04:51:13 INFO ui.MasterWebUI: Started MasterWebUI at http://172.21.0.2:8280
spark-master_1 | 16/10/24 04:51:14 INFO server.Server: jetty-8.y.z-SNAPSHOT
spark-master_1 | 16/10/24 04:51:14 INFO server.AbstractConnector: Started SelectChannelConnector@0b90f19d68f9:6066
spark-master_1 | 16/10/24 04:51:14 INFO util.Utils: Successfully started service on port 6066.
spark-master_1 | 16/10/24 04:51:14 INFO rest.StandaloneRestServer: Started REST server for submitting applications on port 6066
cassandra_1 | INFO 04:51:14 Hostname: a5640b09102a
cassandra_1 | INFO 04:51:14 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_91
cassandra_1 | INFO 04:51:14 Heap size: 7.766GiB/7.766GiB
cassandra_1 | INFO 04:51:14 Code Cache Non-heap memory: init = 2555904(2496K) used = 4130176(4033K) committed = 4194304(4096K) max = 251658240(245760K)
cassandra_1 | INFO 04:51:14 Metaspace Non-heap memory: init = 0(0K) used = 15545024(15180K) committed = 15990784(15616K) max = -1(-1K)
cassandra_1 | INFO 04:51:14 Compressed Class Space Non-heap memory: init = 0(0K) used = 1890384(1846K) committed = 2097152(2048K) max = 1073741824(1048576K)
cassandra_1 | INFO 04:51:14 Par Eden Space Heap memory: init = 671088640(655360K) used = 120807072(117975K) committed = 671088640(655360K) max = 671088640(655360K)
cassandra_1 | INFO 04:51:14 Par Survivor Space Heap memory: init = 83886080(81920K) used = 0(0K) committed = 83886080(81920K) max = 83886080(81920K)
cassandra_1 | INFO 04:51:14 CMS Old Gen Heap memory: init = 7583301632(7405568K) used = 0(0K) committed = 7583301632(7405568K) max = 7583301632(7405568K)
cassandra_1 | INFO 04:51:14 Classpath: /etc/cassandra:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.36.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.7.jar:/usr/share/cassandra/apache-cassandra-thrift-3.7.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
cassandra_1 | INFO 04:51:14 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:+CMSClassUnloadingEnabled, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Xms8031M, -Xmx8031M, -Xmn800M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
spark-slave_1 | 16/10/24 04:51:14 INFO worker.Worker: Starting Spark worker 172.21.0.7:40225 with 8 cores, 30.4 GB RAM
spark-slave_1 | 16/10/24 04:51:14 INFO worker.Worker: Running Spark version 1.6.2
spark-slave_1 | 16/10/24 04:51:14 INFO worker.Worker: Spark home: /opt/spark
hdfsyarn_1 | localhost: Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
hdfsyarn_1 | localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-root-datanode-e2c548d76327.out
cassandra_1 | WARN 04:51:15 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
cassandra_1 | WARN 04:51:15 jemalloc shared library could not be preloaded to speed up memory allocations
cassandra_1 | WARN 04:51:15 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
cassandra_1 | WARN 04:51:15 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
cassandra_1 | INFO 04:51:15 Initializing SIGAR library
kafka_1 | [2016-10-24 04:51:15,566] INFO Verifying properties (kafka.utils.VerifiableProperties)
cassandra_1 | WARN 04:51:15 Cassandra server running in degraded mode. Is swap disabled? : false, Address space adequate? : true, nofile limit adequate? : true, nproc limit adequate? : true
cassandra_1 | WARN 04:51:15 Directory /var/lib/cassandra/data doesn't exist
cassandra_1 | WARN 04:51:15 Directory /var/lib/cassandra/commitlog doesn't exist
cassandra_1 | WARN 04:51:15 Directory /var/lib/cassandra/saved_caches doesn't exist
cassandra_1 | WARN 04:51:15 Directory /var/lib/cassandra/hints doesn't exist
kafka_1 | [2016-10-24 04:51:15,869] INFO Property advertised.host.name is overridden to 172.21.0.1 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,872] INFO Property advertised.port is overridden to 9092 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,872] INFO Property broker.id is overridden to 1 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,873] WARN Property create.topics is not valid (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,873] INFO Property log.cleaner.enable is overridden to false (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,874] INFO Property log.dirs is overridden to /kafka/kafka-logs-1 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,874] INFO Property log.retention.check.interval.ms is overridden to 300000 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,874] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,874] INFO Property log.segment.bytes is overridden to 1073741824 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,875] INFO Property num.io.threads is overridden to 8 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,875] INFO Property num.network.threads is overridden to 3 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,875] INFO Property num.partitions is overridden to 1 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,875] INFO Property num.recovery.threads.per.data.dir is overridden to 1 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,876] INFO Property port is overridden to 9092 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,876] INFO Property socket.receive.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,876] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,876] INFO Property socket.send.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,876] WARN Property version is not valid (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,877] INFO Property zookeeper.connect is overridden to zookeeper:2181 (kafka.utils.VerifiableProperties)
kafka_1 | [2016-10-24 04:51:15,877] INFO Property zookeeper.connection.timeout.ms is overridden to 6000 (kafka.utils.VerifiableProperties)
hdfsyarn_1 | Starting secondary namenodes [0.0.0.0]
kafka_1 | [2016-10-24 04:51:17,180] INFO [Kafka Server 1], starting (kafka.server.KafkaServer)
kafka_1 | [2016-10-24 04:51:17,225] INFO [Kafka Server 1], Connecting to zookeeper on zookeeper:2181 (kafka.server.KafkaServer)
kafka_1 | [2016-10-24 04:51:17,310] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
kafka_1 | [2016-10-24 04:51:17,399] INFO Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,399] INFO Client environment:host.name=730908f89582 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,399] INFO Client environment:java.version=1.6.0_37 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,399] INFO Client environment:java.vendor=Sun Microsystems Inc. (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,399] INFO Client environment:java.home=/usr/lib/jvm/java-6-openjdk-amd64/jre (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:java.class.path=:/opt/kafka_2.10-0.8.2.2/bin/../core/build/dependant-libs-2.10*/*.jar:/opt/kafka_2.10-0.8.2.2/bin/../examples/build/libs//kafka-examples*.jar:/opt/kafka_2.10-0.8.2.2/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/opt/kafka_2.10-0.8.2.2/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/opt/kafka_2.10-0.8.2.2/bin/../clients/build/libs/kafka-clients*.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/jopt-simple-3.2.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/kafka-clients-0.8.2.2.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/kafka_2.10-0.8.2.2-javadoc.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/kafka_2.10-0.8.2.2-scaladoc.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/kafka_2.10-0.8.2.2-sources.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/kafka_2.10-0.8.2.2-test.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/kafka_2.10-0.8.2.2.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/log4j-1.2.16.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/lz4-1.2.0.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/scala-library-2.10.4.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/slf4j-api-1.7.6.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/slf4j-log4j12-1.6.1.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/snappy-java-1.1.1.7.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/zkclient-0.3.jar:/opt/kafka_2.10-0.8.2.2/bin/../libs/zookeeper-3.4.6.jar:/opt/kafka_2.10-0.8.2.2/bin/../core/build/libs/kafka_2.10*.jar (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:java.library.path=/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk-amd64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:os.version=4.7.8-100.fc23.x86_64 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,400] INFO Client environment:user.dir=/ (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2016-10-24 04:51:17,401] INFO Initiating client connection, connectString=zookeeper:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@6d0021f (org.apache.zookeeper.ZooKeeper)
spark-master_1 | 16/10/24 04:51:18 INFO master.Master: I have been elected leader! New state: ALIVE
spark-slave_1 | 16/10/24 04:51:18 INFO server.Server: jetty-8.y.z-SNAPSHOT
streamsets_1 | 2016-10-24 04:51:18,743 [user:] [pipeline:] [thread:main] INFO Main - -----------------------------------------------------------------
streamsets_1 | 2016-10-24 04:51:19,082 [user:] [pipeline:] [thread:main] INFO Main - Build info:
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - Version : 2.1.0.0
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - Date : 2016-10-13T03:47Z
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - Built by : root
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - Repo SHA : 906a26a3985b7c88f48bbff08a55ffca119d736c
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - Source MD5 : d51c7bc4de82af111590d79142dd5ae
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - -----------------------------------------------------------------
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - Runtime info:
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - Java version : 1.8.0_102-b14
streamsets_1 | 2016-10-24 04:51:19,083 [user:] [pipeline:] [thread:main] INFO Main - SDC ID : 74cc1c90-d257-456b-b0e8-2e29acf6e607
streamsets_1 | 2016-10-24 04:51:19,084 [user:] [pipeline:] [thread:main] INFO Main - Runtime dir : /opt/streamsets-datacollector-2.1.0.0
streamsets_1 | 2016-10-24 04:51:19,084 [user:] [pipeline:] [thread:main] INFO Main - Config dir : /etc/sdc
streamsets_1 | 2016-10-24 04:51:19,084 [user:] [pipeline:] [thread:main] INFO Main - Data dir : /data
streamsets_1 | 2016-10-24 04:51:19,084 [user:] [pipeline:] [thread:main] INFO Main - Log dir : /logs
streamsets_1 | 2016-10-24 04:51:19,084 [user:] [pipeline:] [thread:main] INFO Main - -----------------------------------------------------------------
streamsets_1 | 2016-10-24 04:51:19,084 [user:] [pipeline:] [thread:main] INFO Main - Security Manager : ENABLED, policy file: file:///etc/sdc/sdc-security.policy
streamsets_1 | 2016-10-24 04:51:19,084 [user:] [pipeline:] [thread:main] INFO Main - -----------------------------------------------------------------
streamsets_1 | 2016-10-24 04:51:19,084 [user:] [pipeline:] [thread:main] INFO Main - Starting ...
elasticsearch_1 | [2016-10-24 04:51:19,293][INFO ][node ] [Blizzard] initialized
elasticsearch_1 | [2016-10-24 04:51:19,293][INFO ][node ] [Blizzard] starting ...
cassandra_1 | INFO 04:51:19 Initialized prepared statement caches with 31 MB (native) and 31 MB (Thrift)
streamsets_1 | 2016-10-24 04:51:19,451 [user:] [pipeline:] [thread:main] INFO Main - -----------------------------------------------------------------
streamsets_1 | 2016-10-24 04:51:19,452 [user:] [pipeline:] [thread:main] INFO Main - Kerberos enabled: false
streamsets_1 | 2016-10-24 04:51:19,452 [user:] [pipeline:] [thread:main] INFO Main - -----------------------------------------------------------------
streamsets_1 | 2016-10-24 04:51:19,452 [user:] [pipeline:] [thread:main] INFO Main - Starting ...
spark-slave_1 | 16/10/24 04:51:19 INFO server.AbstractConnector: Started [email protected]:8081
spark-slave_1 | 16/10/24 04:51:19 INFO util.Utils: Successfully started service 'WorkerUI' on port 8081.
spark-slave_1 | 16/10/24 04:51:19 INFO ui.WorkerWebUI: Started WorkerWebUI at http://172.21.0.7:8081
kafka_1 | [2016-10-24 04:51:19,624] INFO Opening socket connection to server demo_zookeeper_1.demo_default/172.21.0.3:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
spark-slave_1 | 16/10/24 04:51:19 INFO worker.Worker: Connecting to master spark-master:7077...
hdfsyarndata1_1 | 0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts.
hdfsyarndata1_1 | 0.0.0.0: starting namenode, logging to /usr/local/hadoop/logs/hadoop-root-namenode-b8f9577f4d51.out
kafka_1 | [2016-10-24 04:51:19,754] INFO Socket connection established to demo_zookeeper_1.demo_default/172.21.0.3:2181, initiating session (org.apache.zookeeper.ClientCnxn)
zookeeper_1 | 2016-10-24 04:51:19,778 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /172.21.0.13:38022
zookeeper_1 | 2016-10-24 04:51:19,784 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /172.21.0.13:38022
zookeeper_1 | 2016-10-24 04:51:19,812 [myid:] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.1
hdfsyarn_1 | 0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts.
hdfsyarn_1 | 0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-root-secondarynamenode-e2c548d76327.out
zookeeper_1 | 2016-10-24 04:51:20,564 [myid:] - INFO [SyncThread:0:ZooKeeperServer@617] - Established session 0x157f50639e20000 with negotiated timeout 6000 for client /172.21.0.13:38022
kafka_1 | [2016-10-24 04:51:20,567] INFO Session establishment complete on server demo_zookeeper_1.demo_default/172.21.0.3:2181, sessionid = 0x157f50639e20000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
kafka_1 | [2016-10-24 04:51:20,569] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
zookeeper_1 | 2016-10-24 04:51:20,799 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x157f50639e20000 type:create cxid:0x4 zxid:0x3 txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode = NoNode for /brokers
zookeeper_1 | 2016-10-24 04:51:21,273 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x157f50639e20000 type:create cxid:0xa zxid:0x7 txntype:-1 reqpath:n/a Error Path:/config Error:KeeperErrorCode = NoNode for /config
zookeeper_1 | 2016-10-24 04:51:21,552 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x157f50639e20000 type:create cxid:0x10 zxid:0xb txntype:-1 reqpath:n/a Error Path:/admin Error:KeeperErrorCode = NoNode for /admin
spark-master_1 | 16/10/24 04:51:21 INFO master.Master: Registering worker 172.21.0.7:40225 with 8 cores, 30.4 GB RAM
spark-slave_1 | 16/10/24 04:51:21 INFO worker.Worker: Successfully registered with master spark://172.21.0.2:7077
hdfsyarn_1 | starting yarn daemons
hdfsyarn_1 | starting resourcemanager, logging to /usr/local/hadoop/logs/yarn--resourcemanager-e2c548d76327.out
elasticsearch_1 | [2016-10-24 04:51:22,296][INFO ][transport ] [Blizzard] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/172.21.0.4:9300]}
elasticsearch_1 | [2016-10-24 04:51:22,590][INFO ][discovery ] [Blizzard] elasticsearch/7fTAemgXQLuX09eGrtFYFw
kafka_1 | [2016-10-24 04:51:22,753] INFO Log directory '/kafka/kafka-logs-1' not found, creating it. (kafka.log.LogManager)
kafka_1 | [2016-10-24 04:51:23,023] INFO Loading logs. (kafka.log.LogManager)
kafka_1 | [2016-10-24 04:51:23,100] INFO Logs loading complete. (kafka.log.LogManager)
kafka_1 | [2016-10-24 04:51:23,101] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
kafka_1 | [2016-10-24 04:51:23,203] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
kafka_1 | [2016-10-24 04:51:23,393] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
kafka_1 | [2016-10-24 04:51:23,394] INFO [Socket Server on Broker 1], Started (kafka.network.SocketServer)
hdfsyarndata1_1 | localhost: Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
hdfsyarndata1_1 | localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-root-datanode-b8f9577f4d51.out
hdfsyarn_1 | localhost: Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
hdfsyarn_1 | localhost: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-e2c548d76327.out
zookeeper_1 | 2016-10-24 04:51:24,899 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /172.21.0.13:46068
zookeeper_1 | 2016-10-24 04:51:24,909 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /172.21.0.13:46068
hdfsyarndata1_1 | Starting secondary namenodes [0.0.0.0]
hdfsyarn_1 | chown: missing operand after '/usr/local/hadoop/logs'
hdfsyarn_1 | Try 'chown --help' for more information.
hdfsyarn_1 | starting historyserver, logging to /usr/local/hadoop/logs/mapred--historyserver-e2c548d76327.out
zookeeper_1 | 2016-10-24 04:51:25,091 [myid:] - INFO [SyncThread:0:ZooKeeperServer@617] - Established session 0x157f50639e20001 with negotiated timeout 30000 for client /172.21.0.13:46068
kafka_1 | Error while executing topic command replication factor: 1 larger than available brokers: 0
kafka_1 | kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
kafka_1 | at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
kafka_1 | at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171)
kafka_1 | at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93)
kafka_1 | at kafka.admin.TopicCommand$.main(TopicCommand.scala:55)
kafka_1 | at kafka.admin.TopicCommand.main(TopicCommand.scala)
kafka_1 |
zookeeper_1 | 2016-10-24 04:51:25,279 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x157f50639e20001
zookeeper_1 | 2016-10-24 04:51:25,536 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /172.21.0.13:46068 which had sessionid 0x157f50639e20001
kafka_1 | [2016-10-24 04:51:26,284] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
elasticsearch_1 | [2016-10-24 04:51:26,369][INFO ][cluster.service ] [Blizzard] new_master [Blizzard][7fTAemgXQLuX09eGrtFYFw][cf7c3c54488d][inet[/172.21.0.4:9300]]{master=true}, reason: zen-disco-join (elected_as_master)
elasticsearch_1 | [2016-10-24 04:51:26,477][INFO ][http ] [Blizzard] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/172.21.0.4:9200]}
elasticsearch_1 | [2016-10-24 04:51:26,478][INFO ][node ] [Blizzard] started
zookeeper_1 | 2016-10-24 04:51:26,834 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /172.21.0.13:47544
zookeeper_1 | 2016-10-24 04:51:26,835 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /172.21.0.13:47544
zookeeper_1 | 2016-10-24 04:51:26,968 [myid:] - INFO [SyncThread:0:ZooKeeperServer@617] - Established session 0x157f50639e20002 with negotiated timeout 30000 for client /172.21.0.13:47544
kibana_1 | Starting Kibana
elasticsearch_1 | [2016-10-24 04:51:27,021][INFO ][gateway ] [Blizzard] recovered [2] indices into cluster_state
kafka_1 | Error while executing topic command replication factor: 1 larger than available brokers: 0
kafka_1 | kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
kafka_1 | at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
kafka_1 | at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171)
kafka_1 | at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93)
kafka_1 | at kafka.admin.TopicCommand$.main(TopicCommand.scala:55)
kafka_1 | at kafka.admin.TopicCommand.main(TopicCommand.scala)
kafka_1 |
zookeeper_1 | 2016-10-24 04:51:27,055 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x157f50639e20002
zookeeper_1 | 2016-10-24 04:51:27,135 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /172.21.0.13:47544 which had sessionid 0x157f50639e20002
cassandra_1 | INFO 04:51:27 Initializing system.IndexInfo
kafka_1 | [2016-10-24 04:51:27,874] INFO 1 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
zookeeper_1 | 2016-10-24 04:51:27,884 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x157f50639e20000 type:setData cxid:0x1a zxid:0x13 txntype:-1 reqpath:n/a Error Path:/controller_epoch Error:KeeperErrorCode = NoNode for /controller_epoch
zookeeper_1 | 2016-10-24 04:51:28,624 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x157f50639e20000 type:delete cxid:0x27 zxid:0x15 txntype:-1 reqpath:n/a Error Path:/admin/preferred_replica_election Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election
hdfsyarndata1_1 | 0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts.
hdfsyarndata1_1 | 0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-root-secondarynamenode-b8f9577f4d51.out
cassandra_1 | INFO 04:51:29 Initializing system.batches
cassandra_1 | INFO 04:51:29 Initializing system.paxos
cassandra_1 | INFO 04:51:29 Initializing system.local
cassandra_1 | INFO 04:51:29 Initializing system.peers
cassandra_1 | INFO 04:51:29 Initializing system.peer_events
cassandra_1 | INFO 04:51:29 Initializing system.range_xfers
cassandra_1 | INFO 04:51:29 Initializing system.compaction_history
cassandra_1 | INFO 04:51:29 Initializing system.sstable_activity
cassandra_1 | INFO 04:51:29 Initializing system.size_estimates
cassandra_1 | INFO 04:51:29 Initializing system.available_ranges
cassandra_1 | INFO 04:51:29 Initializing system.views_builds_in_progress
cassandra_1 | INFO 04:51:29 Initializing system.built_views
cassandra_1 | INFO 04:51:29 Initializing system.hints
cassandra_1 | INFO 04:51:29 Initializing system.batchlog
cassandra_1 | INFO 04:51:29 Initializing system.schema_keyspaces
cassandra_1 | INFO 04:51:29 Initializing system.schema_columnfamilies
cassandra_1 | INFO 04:51:29 Initializing system.schema_columns
cassandra_1 | INFO 04:51:29 Initializing system.schema_triggers
cassandra_1 | INFO 04:51:29 Initializing system.schema_usertypes
cassandra_1 | INFO 04:51:29 Initializing system.schema_functions
cassandra_1 | INFO 04:51:29 Initializing system.schema_aggregates
cassandra_1 | INFO 04:51:29 Not submitting build tasks for views in keyspace system as storage service is not initialized
kafka_1 | [2016-10-24 04:51:30,246] INFO Registered broker 1 at path /brokers/ids/1 with address 172.21.0.1:9092. (kafka.utils.ZkUtils$)
kafka_1 | [2016-10-24 04:51:30,329] INFO [Kafka Server 1], started (kafka.server.KafkaServer)
kafka_1 | [2016-10-24 04:51:30,990] INFO New leader is 1 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
hdfsyarndata1_1 | starting yarn daemons
hdfsyarndata1_1 | starting resourcemanager, logging to /usr/local/hadoop/logs/yarn--resourcemanager-b8f9577f4d51.out
cassandra_1 | INFO 04:51:32 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
hdfsyarndata1_1 | localhost: Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
hdfsyarndata1_1 | localhost: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-b8f9577f4d51.out
hdfsyarndata1_1 | chown: missing operand after '/usr/local/hadoop/logs'
hdfsyarndata1_1 | Try 'chown --help' for more information.
hdfsyarndata1_1 | starting historyserver, logging to /usr/local/hadoop/logs/mapred--historyserver-b8f9577f4d51.out
cassandra_1 | INFO 04:51:40 Initializing key cache with capacity of 100 MBs.
cassandra_1 | INFO 04:51:40 Initializing row cache with capacity of 0 MBs
cassandra_1 | INFO 04:51:40 Initializing counter cache with capacity of 50 MBs
cassandra_1 | INFO 04:51:40 Scheduling counter cache save to every 7200 seconds (going to save all keys).
cassandra_1 | INFO 04:51:45 Global buffer pool is enabled, when pool is exahusted (max is 512.000MiB) it will allocate on heap
cassandra_1 | INFO 04:51:50 Populating token metadata from system tables
cassandra_1 | INFO 04:51:50 Token metadata:
cassandra_1 | INFO 04:51:50 Initializing system_schema.keyspaces
cassandra_1 | INFO 04:51:50 Initializing system_schema.tables
cassandra_1 | INFO 04:51:50 Initializing system_schema.columns
cassandra_1 | INFO 04:51:50 Initializing system_schema.triggers
cassandra_1 | INFO 04:51:50 Initializing system_schema.dropped_columns
cassandra_1 | INFO 04:51:50 Initializing system_schema.views
cassandra_1 | INFO 04:51:50 Initializing system_schema.types
cassandra_1 | INFO 04:51:50 Initializing system_schema.functions
cassandra_1 | INFO 04:51:50 Initializing system_schema.aggregates
cassandra_1 | INFO 04:51:50 Initializing system_schema.indexes
cassandra_1 | INFO 04:51:50 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
cassandra_1 | INFO 04:51:50 Completed loading (0 ms; 4 keys) KeyCache cache
cassandra_1 | INFO 04:51:51 No commitlog files found; skipping replay
cassandra_1 | INFO 04:51:51 Populating token metadata from system tables
cassandra_1 | INFO 04:51:51 Token metadata:
cassandra_1 | INFO 04:51:51 Cassandra version: 3.7
cassandra_1 | INFO 04:51:51 Thrift API version: 20.1.0
cassandra_1 | INFO 04:51:51 CQL supported versions: 3.4.2 (default: 3.4.2)
cassandra_1 | INFO 04:51:51 Initializing index summary manager with a memory pool size of 397 MB and a resize interval of 60 minutes
cassandra_1 | INFO 04:51:52 Starting Messaging Service on /172.21.0.6:7000 (eth0)
cassandra_1 | WARN 04:51:52 No host ID found, created 13123b4e-6c80-4dd3-ac66-15b46eeafc0b (Note: This should happen exactly once per node).
cassandra_1 | INFO 04:51:52 Loading persisted ring state
cassandra_1 | INFO 04:51:52 Starting up server gossip
cassandra_1 | INFO 04:51:53 This node will not auto bootstrap because it is configured to be a seed node.
cassandra_1 | INFO 04:51:53 Generated random tokens. tokens are [1450212280348239702, 4359768934484422274, 4142971895749821201, 4127054699048347215, -5392056824737439473, 6244268626577182375, -7050042549095694783, -5344866533706198663, 7189830912249172852, -6088472078750448058, 2642895758481233318, 7372309841939691017, -8650445361302244213, 6524558201996304341, -3166114679550971814, -7162403406308487068, -8907884592642426057, 809619657357802028, 8877959974439748610, -2063852807540754328, -2833075859170642846, -2917449264831571576, 6932849612756460146, -3921938354104484125, -8066744855866199010, -3858661454062144079, 6305162965016012619, 5653302954678280733, 1482284844814064112, -6139767591158576415, -8385443990277234836, 5544510170404185168, -302936880932170333, -1160204587169014607, 8182406280321475730, 5682341434561983604, 2459919243995914607, -3954813967956656677, 5654874606778158133, -5711750245783371310, 7626705672478194531, -3825827982786506421, -2304600592264738483, -2423663797197590652, -7487184226071479612, 7141502932488660276, -5262851344649042304, 9031229957767468899, 5651352744564969302, 8939095682477032708, -5341762319027103973, -1622222764570533161, -578389910935837671, 3752894287584600591, 2977716520922695470, -8773004325426555709, -2601915173133955618, 8991397374461720542, -1300442649606499674, 4498929910996442287, -7537467787048647966, 2484230029755087334, 8490494847959824977, 8979825840810800771, 7717888619166464217, 3547440037675750150, -2432359539522310688, -6396588428946440376, -9062179244541949077, -9171430731833284459, -1336721665369442471, -8778863273951757155, -8780810365998558906, -1751727768387850799, -1268741017795650461, -8183889836154034069, 372730507663763430, -375952278508710967, 9067651783284747666, 1658104211427298775, 5279062880931090108, -374485595564320294, -6065619704045172816, -6933679785349503371, -3425784260032196055, 6603313395511577486, -1291982780546327136, -2273860511986940659, 625147367574262946, 695910214218991016, 1216333193448353482, 4208259084493305227, -723517807867943303, 7917028905751414127, -4933105360866608630, -4182591706179106561, -1162776900532663986, 1578339110277161855, -5058165664444833309, 1069432962186341313, -2678441256729347760, -4862440718664046325, 3419768196780350972, -5325960233373630374, -8152298107969896617, -8472765772453632234, 8290711667025650643, 3215280167028981518, -5559809466511662828, 4484030338263489383, -8260198734150467006, 8273923165842258354, -5144769166561299899, 8475960460687057883, -6969715511620591257, 7149050218516670143, -439277766325789594, -5180544775717086365, -2513794873234360908, -1545768096973966135, 3871054666163200040, 7419975168913357131, 6688579527271079267, -4637355934708009292, 4404319304657273749, 6161881088885343333, 4264264083446653977, 2752941676962885031, -2370006260757231569, 1711890487691060561, 3467389633130034431, 8296012912051817205, -365082721923460296, -753502648899270870, 8313928734939885121, 1490818143100451509, 4311084180393727428, 7884863774986663899, 6755198797217400954, -4730499850100179835, 8054467683444177905, -652545653173605351, 627374896228023078, -5742270073395538820, 2067601541589336177, -5575111228142950049, 2517251240202108361, -4429868321177401876, 4554219089800817366, 4274502748673057782, -3169010689066884483, -7211078024419656018, -4528542200084166322, -8302032515424215320, 1183231819240645265, 624791223985943971, 3729975481912386638, -101227616392473890, -8401886067006132812, 6156526516509676198, 2230067466357118562, 3806620734685963618, -6972428894364277991, -4274823186323842932, -6920557848931479787, 7360158799283561413, 592889669911630178, 3651907553638434883, -6775109836393147298, 2130063049200612120, -4133725233290946656, 4741462706169941167, -8397380649381799115, -3540731226482585502, 7605241204578942556, 5404185139809850021, 7511214834062240913, 1646329712553867862, 1734081165553959521, 2297940250134167190, -6118677813662900984, -7970107527206564794, -5994810104943512901, -489721546422383575, 1369631058276026299, 8231553787297419960, -5177647185024768938, -3443038569500607741, 3396114045794140083, -3433100048758093825, 7787858431285883214, 9075650471831686561, -8688465740890951325, -341182847681625524, 3976010207570713590, -3815541904174303556, -7496480182182728884, -7819688123315013532, -7707805571961365333, 8224285303136404643, 2653622646769088119, 7328674741517289569, 9180995365250193021, 6472007582860091591, 5756694736294655492, -2845695063317502056, 4234854237796129612, 5423307731648815089, 6634325338299106693, -3000565342999082176, -7192868287951860324, -7654500522227718229, 8531111788651376771, -3022745580116426353, 7956992130294742328, 3380778082756677564, 4369738773125553208, 8901049782066583428, -4831107386217784606, 8401313224300981710, -7412300325849418755, -8436542521565408779, 8292796407040012020, -3669367470888129989, -779648966415599922, -6998193605165944726, 5224784342361507471, -5431545430962269842, 9177044290217548764, -7155863811803004736, -4063552278276985902, -8006225439313809564, 2051058939235905056, -1875487852229532437, 5894682259078033478, 747610502403226711, 6796376682990317123, 4042432013991125275, -3401182231262894888, -6698523555761456142, 3808539076714551791, -4935150557619832543, -5423845531237798508, -7961993575763232806, -4208089230894709739, -4792060053035552207, 5778211877322464363, -6828832411872936330, 1347117495232509308, 4158418366671176145, -6881442445183279236, -6079561530322586529, 2691497384837738783, 1594733399261283364, 5334395922906372364, 5852896753162881823]
cassandra_1 | INFO 04:51:53 Create new Keyspace: KeyspaceMetadata{name=system_traces, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=2}}, tables=[org.apache.cassandra.config.CFMetaData@3c8cde4e[cfId=c5e99f16-8677-3914-b17e-960613512345,ksName=system_traces,cfName=sessions,flags=[COMPOUND],params=TableParams{comment=tracing sessions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(),partitionColumns=[[] | [client command coordinator duration request started_at parameters]],partitionKeyColumns=[ColumnDefinition{name=session_id, type=org.apache.cassandra.db.marshal.UUIDType, kind=PARTITION_KEY, position=0}],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[ColumnDefinition{name=client, type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, position=-1}, ColumnDefinition{name=command, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=session_id, type=org.apache.cassandra.db.marshal.UUIDType, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=coordinator, type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, position=-1}, ColumnDefinition{name=request, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, position=-1}, ColumnDefinition{name=duration, type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, ColumnDefinition{name=parameters, type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type), kind=REGULAR, position=-1}],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@1421eeb8[cfId=8826e8e9-e16a-3728-8753-3bc1fc713c25,ksName=system_traces,cfName=events,flags=[COMPOUND],params=TableParams{comment=tracing events, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [activity source source_elapsed thread]],partitionKeyColumns=[ColumnDefinition{name=session_id, type=org.apache.cassandra.db.marshal.UUIDType, kind=PARTITION_KEY, position=0}],clusteringColumns=[ColumnDefinition{name=event_id, type=org.apache.cassandra.db.marshal.TimeUUIDType, kind=CLUSTERING, position=0}],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[ColumnDefinition{name=activity, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=session_id, type=org.apache.cassandra.db.marshal.UUIDType, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=thread, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=event_id, type=org.apache.cassandra.db.marshal.TimeUUIDType, kind=CLUSTERING, position=0}, ColumnDefinition{name=source, type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, position=-1}, ColumnDefinition{name=source_elapsed, type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
cassandra_1 | INFO 04:52:10 Not submitting build tasks for views in keyspace system_traces as storage service is not initialized
cassandra_1 | INFO 04:52:10 Initializing system_traces.events
cassandra_1 | INFO 04:52:10 Initializing system_traces.sessions
cassandra_1 | INFO 04:52:10 Create new Keyspace: KeyspaceMetadata{name=system_distributed, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=3}}, tables=[org.apache.cassandra.config.CFMetaData@410dfa3c[cfId=759fffad-624b-3181-80ee-fa9a52d1f627,ksName=system_distributed,cfName=repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [coordinator exception_message exception_stacktrace finished_at parent_id range_begin range_end started_at status participants]],partitionKeyColumns=[ColumnDefinition{name=keyspace_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=columnfamily_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=1}],clusteringColumns=[ColumnDefinition{name=id, type=org.apache.cassandra.db.marshal.TimeUUIDType, kind=CLUSTERING, position=0}],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[ColumnDefinition{name=status, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=id, type=org.apache.cassandra.db.marshal.TimeUUIDType, kind=CLUSTERING, position=0}, ColumnDefinition{name=coordinator, type=org.apache.cassandra.db.marshal.InetAddressType, kind=REGULAR, position=-1}, ColumnDefinition{name=finished_at, type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, position=-1}, ColumnDefinition{name=participants, type=org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal.InetAddressType), kind=REGULAR, position=-1}, ColumnDefinition{name=exception_stacktrace, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=parent_id, type=org.apache.cassandra.db.marshal.TimeUUIDType, kind=REGULAR, position=-1}, ColumnDefinition{name=range_end, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=range_begin, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=exception_message, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=keyspace_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=started_at, type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, position=-1}, ColumnDefinition{name=columnfamily_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=1}],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@ad31174[cfId=deabd734-b99d-3b9c-92e5-fd92eb5abf14,ksName=system_distributed,cfName=parent_repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(),partitionColumns=[[] | [exception_message exception_stacktrace finished_at keyspace_name started_at columnfamily_names options requested_ranges successful_ranges]],partitionKeyColumns=[ColumnDefinition{name=parent_id, type=org.apache.cassandra.db.marshal.TimeUUIDType, kind=PARTITION_KEY, position=0}],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimeUUIDType,columnMetadata=[ColumnDefinition{name=requested_ranges, type=org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal.UTF8Type), kind=REGULAR, position=-1}, ColumnDefinition{name=exception_message, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=keyspace_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=successful_ranges, type=org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal.UTF8Type), kind=REGULAR, position=-1}, ColumnDefinition{name=started_at, type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, position=-1}, ColumnDefinition{name=finished_at, type=org.apache.cassandra.db.marshal.TimestampType, kind=REGULAR, position=-1}, ColumnDefinition{name=options, type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type), kind=REGULAR, position=-1}, ColumnDefinition{name=exception_stacktrace, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=parent_id, type=org.apache.cassandra.db.marshal.TimeUUIDType, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=columnfamily_names, type=org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal.UTF8Type), kind=REGULAR, position=-1}],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@5ebdb429[cfId=5582b59f-8e4e-35e1-b913-3acada51eb04,ksName=system_distributed,cfName=view_build_status,flags=[COMPOUND],params=TableParams{comment=Materialized View build status, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.UUIDType),partitionColumns=[[] | [status]],partitionKeyColumns=[ColumnDefinition{name=keyspace_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=view_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=1}],clusteringColumns=[ColumnDefinition{name=host_id, type=org.apache.cassandra.db.marshal.UUIDType, kind=CLUSTERING, position=0}],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[ColumnDefinition{name=status, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=keyspace_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=view_name, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=1}, ColumnDefinition{name=host_id, type=org.apache.cassandra.db.marshal.UUIDType, kind=CLUSTERING, position=0}],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
kibana_1 | {"name":"Kibana","hostname":"1d365d3720ec","pid":1,"level":30,"msg":"Found kibana index","time":"2016-10-24T04:52:13.475Z","v":0}
kibana_1 | {"name":"Kibana","hostname":"1d365d3720ec","pid":1,"level":30,"msg":"Listening on 0.0.0.0:5601","time":"2016-10-24T04:52:13.702Z","v":0}
cassandra_1 | INFO 04:52:15 Not submitting build tasks for views in keyspace system_distributed as storage service is not initialized
cassandra_1 | INFO 04:52:15 Initializing system_distributed.parent_repair_history
cassandra_1 | INFO 04:52:15 Initializing system_distributed.repair_history
cassandra_1 | INFO 04:52:15 Initializing system_distributed.view_build_status
cassandra_1 | INFO 04:52:18 Node /172.21.0.6 state jump to NORMAL
cassandra_1 | INFO 04:52:18 Create new Keyspace: KeyspaceMetadata{name=system_auth, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=1}}, tables=[org.apache.cassandra.config.CFMetaData@5fdae2c1[cfId=5bc52802-de25-35ed-aeab-188eecebb090,ksName=system_auth,cfName=roles,flags=[COMPOUND],params=TableParams{comment=role definitions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(),partitionColumns=[[] | [can_login is_superuser salted_hash member_of]],partitionKeyColumns=[ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=salted_hash, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR, position=-1}, ColumnDefinition{name=member_of, type=org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal.UTF8Type), kind=REGULAR, position=-1}, ColumnDefinition{name=can_login, type=org.apache.cassandra.db.marshal.BooleanType, kind=REGULAR, position=-1}, ColumnDefinition{name=is_superuser, type=org.apache.cassandra.db.marshal.BooleanType, kind=REGULAR, position=-1}],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@751a7391[cfId=0ecdaa87-f8fb-3e60-88d1-74fb36fe5c0d,ksName=system_auth,cfName=role_members,flags=[COMPOUND],params=TableParams{comment=role memberships lookup table, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}],clusteringColumns=[ColumnDefinition{name=member, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0}],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=member, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0}],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@5f27e638[cfId=3afbe79f-2194-31a7-add7-f5ab90d8ec9c,ksName=system_auth,cfName=role_permissions,flags=[COMPOUND],params=TableParams{comment=permissions granted to db roles, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | [permissions]],partitionKeyColumns=[ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}],clusteringColumns=[ColumnDefinition{name=resource, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0}],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[ColumnDefinition{name=resource, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0}, ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=permissions, type=org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal.UTF8Type), kind=REGULAR, position=-1}],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@79454af9[cfId=5f2fbdad-91f1-3946-bd25-d5da3a5c35ec,ksName=system_auth,cfName=resource_role_permissons_index,flags=[COMPOUND],params=TableParams{comment=index of db roles with permissions granted on a resource, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@b56fdcde, extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[ColumnDefinition{name=resource, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}],clusteringColumns=[ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0}],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[ColumnDefinition{name=resource, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, position=0}, ColumnDefinition{name=role, type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING, position=0}],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
logstash_1 | {:timestamp=>"2016-10-24T04:52:21.692000+0000", :message=>"Pipeline main started"}
cassandra_1 | INFO 04:52:23 Not submitting build tasks for views in keyspace system_auth as storage service is not initialized
cassandra_1 | INFO 04:52:23 Initializing system_auth.resource_role_permissons_index
cassandra_1 | INFO 04:52:23 Initializing system_auth.role_members
cassandra_1 | INFO 04:52:23 Initializing system_auth.role_permissions
cassandra_1 | INFO 04:52:23 Initializing system_auth.roles
cassandra_1 | INFO 04:52:23 Waiting for gossip to settle before accepting client requests...
cassandra_1 | INFO 04:52:31 No gossip backlog; proceeding
cassandra_1 | INFO 04:52:31 Netty using native Epoll event loop
cassandra_1 | INFO 04:52:32 Using Netty Version: [netty-buffer=netty-buffer-4.0.36.Final.e8fa848, netty-codec=netty-codec-4.0.36.Final.e8fa848, netty-codec-haproxy=netty-codec-haproxy-4.0.36.Final.e8fa848, netty-codec-http=netty-codec-http-4.0.36.Final.e8fa848, netty-codec-socks=netty-codec-socks-4.0.36.Final.e8fa848, netty-common=netty-common-4.0.36.Final.e8fa848, netty-handler=netty-handler-4.0.36.Final.e8fa848, netty-tcnative=netty-tcnative-1.1.33.Fork15.906a8ca, netty-transport=netty-transport-4.0.36.Final.e8fa848, netty-transport-native-epoll=netty-transport-native-epoll-4.0.36.Final.e8fa848, netty-transport-rxtx=netty-transport-rxtx-4.0.36.Final.e8fa848, netty-transport-sctp=netty-transport-sctp-4.0.36.Final.e8fa848, netty-transport-udt=netty-transport-udt-4.0.36.Final.e8fa848]
cassandra_1 | INFO 04:52:32 Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...
cassandra_1 | INFO 04:52:32 Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it
cassandra_1 | INFO 04:52:33 Scheduling approximate time-check task with a precision of 10 milliseconds
cassandra_1 | INFO 04:52:33 Created default superuser role 'cassandra'
streamsets_1 | 2016-10-24 04:52:58,841 [user:] [pipeline:] [thread:main] INFO StandaloneAndClusterRunnerProviderImpl - Pipeline execution mode is: STANDALONE
streamsets_1 | 2016-10-24 04:52:58,849 [user:admin] [pipeline:localtohdfscopycopy] [thread:main] INFO StandaloneRunner - Pipeline localtohdfscopycopy with rev 0 is in state: RUNNING
streamsets_1 | 2016-10-24 04:52:58,924 [user:admin] [pipeline:localtohdfscopycopy] [thread:runner-pool-2-thread-1] INFO StandaloneRunner - Pipeline 'localtohdfscopycopy::0' has status: 'DISCONNECTED'
streamsets_1 | 2016-10-24 04:52:58,928 [user:admin] [pipeline:localtohdfscopycopy] [thread:runner-pool-2-thread-1] INFO StandaloneRunner - Preparing to start pipeline 'localtohdfscopycopy::0'
streamsets_1 | 2016-10-24 04:52:58,930 [user:admin] [pipeline:localtohdfscopycopy] [thread:runner-pool-2-thread-1] INFO StandaloneRunner - Starting pipeline localtohdfscopycopy 0
streamsets_1 | 2016-10-24 04:52:59,583 [user:admin] [pipeline:localtohdfscopycopy] [thread:ProductionPipelineRunnable-localtohdfscopycopy] INFO SpoolDirSource - Ordering: Lexicographically Ascending File Names
streamsets_1 | 2016-10-24 04:52:59,602 [user:admin] [pipeline:localtohdfscopycopy] [thread:ProductionPipelineRunnable-localtohdfscopycopy] INFO StagePipe - Starting memory collector for Directory_01
streamsets_1 | 2016-10-24 04:53:00,110 [user:] [pipeline:] [thread:main] INFO Reflections - Reflections took 319 ms to scan 2 urls, producing 180 keys and 798 values
streamsets_1 | 2016-10-24 04:53:00,123 [user:admin] [pipeline:localtohdfscopycopy] [thread:ProductionPipelineRunnable-localtohdfscopycopy] WARN LoginUgiProviderFactory - Could not find implementation of LoginUgiProviderFactory. Using default implementation meant for Apache Hadoop.
streamsets_1 | 2016-10-24 04:53:01,481 [user:] [pipeline:] [thread:main] INFO Reflections - Reflections took 52 ms to scan 2 urls, producing 180 keys and 798 values
streamsets_1 | 2016-10-24 04:53:01,750 [user:admin] [pipeline:localtohdfscopycopy] [thread:ProductionPipelineRunnable-localtohdfscopycopy] INFO HdfsTargetConfigBean - Authentication Config: Using Simple
streamsets_1 | Running on URI : 'http://92c3cbc1d07b:18630'
streamsets_1 | 2016-10-24 04:53:01,861 [user:] [pipeline:] [thread:main] INFO WebServerTask - Running on URI : 'http://92c3cbc1d07b:18630'
streamsets_1 | 2016-10-24 04:53:02,278 [user:admin] [pipeline:localtohdfscopycopy] [thread:ProductionPipelineRunnable-localtohdfscopycopy] INFO StagePipe - Starting memory collector for HadoopFS_01
streamsets_1 | 2016-10-24 04:53:02,287 [user:admin] [pipeline:localtohdfscopycopy] [thread:ProductionPipelineRunnable-localtohdfscopycopy] WARN SpoolDirSource - Ignoring file 'UNSW89898.csv' in spool directory as is lesser than offset file 'UNSW89898.csv'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment