Created
February 3, 2017 06:00
-
-
Save Jammink2/336bd2b3d8fa68e2fb02b7060db1e4a7 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#setting up your vm: | |
vagrant init hashicorp/precise64 | |
add the following to your Vagrantfile (comment this out at first) | |
#config.ssh.username = 'root' | |
#config.ssh.password = 'vagrant' | |
#config.ssh.insert_key = 'true' | |
vagrant up | |
#change sudo root password on your virtual box to 'vagrant' | |
sudo passwd root | |
#edit /etc/ssh/sshd_config | |
PermitRootLogin yes | |
#this doesn't actually work. once connected to the box, do sudo su | |
# see also http://stackoverflow.com/questions/25758737/vagrant-login-as-root-by-default | |
sudo apt-get update | |
sudo apt-get dist-upgrade | |
When prompted, install grub to first partition: | |
[ ] /dev/sda (85899 MB; VBOX_HARDDISK) | |
#installing dependencies | |
##start here## | |
echo 'vm.swappiness = 0' >> /etc/sysctl.conf | |
rm /bin/sh ; ln -sf /bin/bash /bin/sh | |
ensure you have: | |
curl | |
jdk-1.7.0_XX (Oracle JDK v1.7.0) | |
sudo add-apt-repository ppa:webupd8team/java | |
sudo apt-get update | |
sudo apt-get install oracle-java7-installer | |
sudo apt-get install oracle-java7-set-default | |
#create a soft link for YARN support | |
sudo ln -s /usr/bin/java /bin/java | |
#after installing JDK; before starting splice | |
sudo nano ~/.bash_profile (change this command to linux equivalent) | |
export JAVA_HOME=`/usr/libexec/java_home` | |
# install, or verify installation of curl, jdk (see above), nscd, ntp, openssh, openssh-clients, openssh-server, patch, rlwrap, wget | |
#install ntp - https://blogging.dragon.org.uk/setting-up-ntp-on-ubuntu-14-04/ | |
sudo apt-get update | |
sudo apt-get dist-upgrade | |
sudo apt-get install ntp ntpdate | |
sudo apt-get --ignore-missing install curl ntp openssh-clients openssh-server, patch, rlwrap, wget | |
#ensure all necessary services are started | |
#service nscd start && service ntpd start && service sshd start | |
1. Downloading Splice Machine: | |
# this seems to work | |
wget https://s3.amazonaws.com/splice-releases/2.0.1.18/standalone/splice_machine-2.0.1.18_standalone.tar.gz | |
2. Optional - move downloaded tarball | |
3. Install Splice Machine | |
#create a splicemachine subdirectory and install Splice machine | |
tar -xzf splice_machine-2.0.1.18_standalone.tar.gz | |
4. Start Using Splice Machine | |
# http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/StandaloneInstall.html#Start | |
cd splicemachine | |
./bin/start-splice.sh | |
./bin/sqlshell.sh | |
See link above | |
# http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/StandaloneInstall.html#Start | |
for more splice commands, sample data, importing that data, sample queries on that data, etc. | |
##restart from here Import and Query the Sample Data | |
on http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/StandaloneInstall.html | |
(include demo steps here) | |
## next command | |
splice> run 'demodata/sql/loadall.sql'; | |
#### caveats | |
a number of the dependencies don't appear to exist as they are documented. I'm probably missing some of them | |
can we bundle together different dependencies for different OSes? Installing dependencies manually is incredibly error prone | |
this as taken 2+ hours, drained most of my battery, traversed about 20 web pages, and I'm still not up and running with a basic standalone version on ubuntu vm. As an open source community hacker, I wouldn't touch this thing with a 50 foot pole. | |
http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/Intro.InstallationGuide.html | |
- this is terrible. there is no 'sshd' | |
can I get a reasonable error message (missing dependencies, etc) when ./bin/start-splice.sh fails ?? | |
splice log doesn't tell anything useful; initializing database never actually completes | |
http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/StandaloneInstall.html | |
Starting SM gives me this: Waiting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . | |
if loading with less than 10k memory | |
then | |
splice db never actually loads | |
Tailing splice.log gives me this: | |
"Splice Server is ready" message never actually appears. Although it says that splice is running. I get "Waiting....." with the following error from Splice Log: | |
java.lang.NullPointerException | |
at com.splicemachine.compactions.PoolSlotBooker.run(PoolSlotBooker.java:56) | |
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) | |
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) | |
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) | |
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) | |
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) | |
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) | |
at java.lang.Thread.run(Thread.java:745) | |
Where are the troubleshooting instructions? | |
This was a memory issue. | |
solution: | |
Vagrant Halt | |
Changed Vagrantfile | |
vb.memory = "10000" #later 14000 - | |
vagrant up | |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | |
eventually memory maxes out for any of data ingestion on a vm - will try cloud version next! | |
Feeding wrong values to database table of ints - indecipherable error message (repro this) | |
- need to do a survey of error conditions | |
suggestion - benchmark - will be good to have a dedicated engineer on benchmarks; I'll write the thing up and SEO it | |
bug: | |
if | |
splice> run 'demodata/sql/loadall.sql'; | |
then | |
Causes a crash, breaks connection to database: | |
splice> run 'demodata/sql/loadall.sql'; | |
splice> /home/vagrant/splicemachine/CREATE TABLE T_HEADER | |
( | |
TRANSACTION_HEADER_KEY BIGINT NOT NULL, | |
CUSTOMER_MASTER_ID BIGINT, | |
TRANSACTION_DT DATE NOT NULL, | |
STORE_NBR SMALLINT NOT NULL, | |
EXCHANGE_RATE_PERCENT DOUBLE PRECISION, | |
GEOCAPTURE_FLG VARCHAR(1), | |
PRIMARY KEY(TRANSACTION_HEADER_KEY) | |
); | |
ERROR 42X01: Syntax error: Encountered "/" at line 1, column 1. | |
Issue the 'help' command for general information on Splice command syntax. | |
Any unrecognized commands are treated as potential SQL commands and executed directly. | |
Consult your DBMS server reference documentation for details of the SQL syntax supported by your server. | |
splice> CREATE TABLE T_DETAIL | |
( | |
TRANSACTION_HEADER_KEY BIGINT NOT NULL, | |
TRANSACTION_DETAIL_KEY BIGINT NOT NULL, | |
CUSTOMER_MASTER_ID BIGINT, | |
TRANSACTION_DT DATE NOT NULL, | |
ORIGINAL_SKU_CATEGORY_ID INTEGER, | |
PRIMARY KEY(TRANSACTION_HEADER_KEY, TRANSACTION_DETAIL_KEY) | |
); | |
ERROR X0Y32: table 'T_DETAIL' already exists in schema 'SPLICE'. | |
splice> CREATE TABLE CUSTOMERS | |
( | |
CUSTOMER_MASTER_ID BIGINT PRIMARY KEY | |
); | |
ERROR X0Y32: table 'CUSTOMERS' already exists in schema 'SPLICE'. | |
splice> -- replace yourPath with the appropriate full path to the demodata directory (i.e. '/vagrant/user/splicemachine/demodata/data/theader.csv') | |
call SYSCS_UTIL.IMPORT_DATA('SPLICE' , 'T_HEADER' , null , '/home/vagrant/splicemachine/demodata/data/theader.csv' , ',' , null , null , 'yyyy-MM-dd' , null , 0 , null , null , null); | |
ERROR XIE0M: Table 'SPLICE.T_HEADER' does not exist. | |
splice> call SYSCS_UTIL.IMPORT_DATA('SPLICE' , 'T_DETAIL' , null , '/home/vagrant/splicemachine/demodata/data/tdetail.csv' , ',' , null , null , 'yyyy-MM-dd' , null , 0 , null , null , null); | |
ERROR 08006: A network protocol error was encountered and the connection has been terminated: the requested command encountered an unarchitected and implementation-specific condition for which there was no architected message (additional information may be available in the derby.log file on the server) | |
ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: A communications error has been detected: {0}.::SQLSTATE: XJ001Java exception: 'Permission denied: user=root, path="file:/home/vagrant/splicemachine/demodata/data":1002:vboxsf:drwxr-xr-x: org.apache.hadoop.security.AccessControlException'. | |
splice> call SYSCS_UTIL.IMPORT_DATA('SPLICE' , 'CUSTOMERS' , null , '/home/vagrant/splicemachine/demodata/data/customers.csv' , ',' , null , null , null , null , 0 , null , null , null); | |
ERROR 08003: No current connection. | |
splice> create index tdidx1 on t_detail(original_sku_category_id,transaction_dt,customer_master_id); | |
ERROR 08003: No current connection. | |
splice> create index thidx2 on t_header(customer_master_id, transaction_dt); | |
ERROR 08003: No current connection. | |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | |
unclear error message | |
if | |
splice> create table test (i int); | |
and | |
splice> | |
insert into test values a,b,c,d,5; | |
then | |
ERROR 42X04: Column 'A' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE statement then 'A' is not a column in the target table. | |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment