| lars@DESKTOP-I4MRDRU:~/projects/work/hbase-example/hbase-example$ mvn -q exec:java -Dexec.mainClass="com.larsgeorge.ScanExample" | |
| WARNING: An illegal reflective access operation has occurred | |
| WARNING: Illegal reflective access by org.apache.hadoop.hbase.unsafe.HBasePlatformDependent (file:/home/lars/.m2/repository/org/apache/hbase/thirdparty/hbase-unsafe/4.1.1/hbase-unsafe-4.1.1.jar) to method java.nio.Bits.unaligned() | |
| WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.hbase.unsafe.HBasePlatformDependent | |
| WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations | |
| WARNING: All illegal access operations will be denied in a future release | |
| 13:27:14.153 [ReadOnlyZKClient-107-21-193-76.compute-1.amazonaws.com:2181@0x39905349-SendThread()] ERROR org.apache.zookeeper.client.StaticHostProvider - Unable to resolve address: 107-21-193-76.compute-1.amazonaws.com:2181 | |
| java.net.UnknownHostException: 107-21-193-76.compute-1.amazonaws.com: Name or service |
This document contains the necessary resources to set up and configure an Okera cluster with the Sales Demo dataset, users, roles and access policies.
Attached are three files:
sales_data_sample.cvs - This is the sales demo dataset.salesdemo.sql - The DDL statements to create schema and access policies
| -- cleanup previous demo | |
| -- drop database and permissions | |
| drop database if exists marketing cascade INCLUDING PERMISSIONS; | |
| drop database if exists demo cascade INCLUDING PERMISSIONS; | |
| -- ensure roles are all created cleanly | |
| drop role if exists analyst_role; | |
| create role if not exists analyst_role; | |
| grant role analyst_role to group analyst_group; | |
| -- create US role for old view row filter demo. | |
| -- TODO remove once we are only using 2.1 to demo |
| 2016-11-25T12:39:20.420+0100: 177.558: [SoftReference, 0 refs, 0.0000547 secs]2016-11-25T12:39:20.420+0100: 177.558: [WeakReference, 2766 refs, 0.0004900 secs]2016-11-25T12:39:20.421+0100: 177.558: [FinalReference, 589 refs, 0.0007352 secs]2016-11-25T12:39:20.422+0100: 177.559: [PhantomReference, 0 refs, 182 refs, 0.0000369 secs]2016-11-25T12:39:20.422+0100: 177.559: [JNI Weak Reference, 0.0000666 secs] 177.562: [G1Ergonomics (Concurrent Cycles) do not request concurrent cycle initiation, reason: still doing mixed collections, occupancy: 8925478912 bytes, allocation request: 0 bytes, threshold: 7730941095 bytes (45.00 %), source: end of GC] | |
| 177.562: [G1Ergonomics (Mixed GCs) do not continue mixed GCs, reason: reclaimable percentage not over threshold, candidate old regions: 94 regions, reclaimable: 850530200 bytes (4.95 %), threshold: 5.00 %] | |
| , 0.0895976 secs] | |
| [Parallel Time: 82.9 ms, GC Workers: 33] | |
| [GC Worker Start (ms): Min: 177473.8, Avg: 177474.2, Max: 177474.6, Diff: 0.8] | |
| [Ext Root Scann |
| # See: https://www.websequencediagrams.com/ | |
| title HBase Metrics Call Sequence Diagramm | |
| MetricsSystem->MetricsSystem: onTimerEvent() | |
| activate MetricsSystem | |
| MetricsSystem->MetricsSystem: publishMetrics() | |
| MetricsSystem->MetricsSystem: sampleMetrics() | |
| MetricsSystem->+MetricsBufferBuilder: constructor | |
| MetricsBufferBuilder->-MetricsSystem: MetricsBufferBuilder |
| IMPORTANT DISCLAIMER: Many of the found "properties" aren't really useful or mistakenly listed, | |
| like those starting with "org.apache.hadoop.*". The tool simply greps a pattern in the matching | |
| files, and some of them are not used as properties. On the other hand, all named properties from | |
| the given configuration file are removed, so quite a few are valid. Again, some of those are still | |
| hidden on purpose, since it is not useful to tinker with them unless in extreme situations, or | |
| during test runs. Be VERY CAREFUL of what you do. I warned ya! | |
| $ sh target/bin/run-finder -d /projects/opensource/hbase/hbase-trunk-rw-git/ -u -s \ | |
| --exclude **/generated --exclude **/generated/** --exclude **/target \ | |
| --exclude **/test/** -c hbase-common/src/main/resources/hbase-default.xml -o /dev/null |
| /projects/private/hadoop-config-differ [git:master]$ sh target/bin/run-differ /projects/opensource/hbase/hbase-r1130916/src/main/resources/hbase-default.xml r1130916 /projects/opensource/hbase/hbase-0.92-rw/src/main/resources/hbase-default.xml 0.92 /projects/opensource/hbase/hbase-0.94-rw/src/main/resources/hbase-default.xml 0.94 /projects/opensource/hbase/hbase-0.96-rw/hbase-common/src/main/resources/hbase-default.xml 0.96 /projects/opensource/hbase/hbase-0.98-rw/hbase-common/src/main/resources/hbase-default.xml 0.98 /projects/opensource/hbase/hbase-trunk-rw-git/hbase-common/src/main/resources/hbase-default.xml 0.99 | |
| ========================================================= | |
| Start | |
| ========================================================= | |
| Checking differences across versions... | |
| Added or renamed keys in 0.92: | |
| added: Property{key='dfs.support.append', value='true', description='Does HDFS allow appends to files? This is an hdfs config. set in here so the hdfs client will do append support. You must ensure tha |
| $ time dd if=/dev/zero bs=1G count=1 | hadoop fs -put - 1g.bin | |
| 1+0 records in | |
| 1+0 records out | |
| 1073741824 bytes (1.1 GB) copied, 64.4632 s, 16.7 MB/s | |
| real 1m5.239s | |
| user 0m9.560s | |
| sys 0m3.394s | |
| $ time hadoop fs -get 1g.bin - > /dev/null |
| #... | |
| #Default values | |
| hadoop.tasklog.taskid=null | |
| hadoop.tasklog.iscleanup=false | |
| hadoop.tasklog.noKeepSplits=4 | |
| hadoop.tasklog.totalLogFileSize=100 | |
| hadoop.tasklog.purgeLogSplits=true | |
| hadoop.tasklog.logsRetainHours=12 | |
| log4j.appender.TLA=org.apache.log4j.net.SyslogAppender |
| HBase 1: | |
| 506 wget http://apache.easy-webs.de//hbase/hbase-0.90.0/hbase-0.90.0.tar.gz | |
| 507 tar -zxvf hbase-0.90.0.tar.gz | |
| 508 cd hbase-0.90.0 | |
| 522 cp -pR conf conf.2 | |
| 523 vim conf/hbase-site.xml | |
| 524 cp conf/hbase-site.xml conf.2/ | |
| 525 vim conf.2/hbase-site.xml |