Skip to content

Instantly share code, notes, and snippets.

@Jammink2
Created February 3, 2017 06:00
Show Gist options
  • Save Jammink2/336bd2b3d8fa68e2fb02b7060db1e4a7 to your computer and use it in GitHub Desktop.
Save Jammink2/336bd2b3d8fa68e2fb02b7060db1e4a7 to your computer and use it in GitHub Desktop.
#setting up your vm:
vagrant init hashicorp/precise64
add the following to your Vagrantfile (comment this out at first)
#config.ssh.username = 'root'
#config.ssh.password = 'vagrant'
#config.ssh.insert_key = 'true'
vagrant up
#change sudo root password on your virtual box to 'vagrant'
sudo passwd root
#edit /etc/ssh/sshd_config
PermitRootLogin yes
#this doesn't actually work. once connected to the box, do sudo su
# see also http://stackoverflow.com/questions/25758737/vagrant-login-as-root-by-default
sudo apt-get update
sudo apt-get dist-upgrade
When prompted, install grub to first partition:
[ ] /dev/sda (85899 MB; VBOX_HARDDISK)
#installing dependencies
##start here##
echo 'vm.swappiness = 0' >> /etc/sysctl.conf
rm /bin/sh ; ln -sf /bin/bash /bin/sh
ensure you have:
curl
jdk-1.7.0_XX (Oracle JDK v1.7.0)
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer
sudo apt-get install oracle-java7-set-default
#create a soft link for YARN support
sudo ln -s /usr/bin/java /bin/java
#after installing JDK; before starting splice
sudo nano ~/.bash_profile (change this command to linux equivalent)
export JAVA_HOME=`/usr/libexec/java_home`
# install, or verify installation of curl, jdk (see above), nscd, ntp, openssh, openssh-clients, openssh-server, patch, rlwrap, wget
#install ntp - https://blogging.dragon.org.uk/setting-up-ntp-on-ubuntu-14-04/
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install ntp ntpdate
sudo apt-get --ignore-missing install curl ntp openssh-clients openssh-server, patch, rlwrap, wget
#ensure all necessary services are started
#service nscd start && service ntpd start && service sshd start
1. Downloading Splice Machine:
# this seems to work
wget https://s3.amazonaws.com/splice-releases/2.0.1.18/standalone/splice_machine-2.0.1.18_standalone.tar.gz
2. Optional - move downloaded tarball
3. Install Splice Machine
#create a splicemachine subdirectory and install Splice machine
tar -xzf splice_machine-2.0.1.18_standalone.tar.gz
4. Start Using Splice Machine
# http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/StandaloneInstall.html#Start
cd splicemachine
./bin/start-splice.sh
./bin/sqlshell.sh
See link above
# http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/StandaloneInstall.html#Start
for more splice commands, sample data, importing that data, sample queries on that data, etc.
##restart from here Import and Query the Sample Data
on http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/StandaloneInstall.html
(include demo steps here)
## next command
splice> run 'demodata/sql/loadall.sql';
#### caveats
a number of the dependencies don't appear to exist as they are documented. I'm probably missing some of them
can we bundle together different dependencies for different OSes? Installing dependencies manually is incredibly error prone
this as taken 2+ hours, drained most of my battery, traversed about 20 web pages, and I'm still not up and running with a basic standalone version on ubuntu vm. As an open source community hacker, I wouldn't touch this thing with a 50 foot pole.
http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/Intro.InstallationGuide.html
- this is terrible. there is no 'sshd'
can I get a reasonable error message (missing dependencies, etc) when ./bin/start-splice.sh fails ??
splice log doesn't tell anything useful; initializing database never actually completes
http://doc.splicemachine.com/Administrators/InstallingSpliceMachine/StandaloneInstall.html
Starting SM gives me this: Waiting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
if loading with less than 10k memory
then
splice db never actually loads
Tailing splice.log gives me this:
"Splice Server is ready" message never actually appears. Although it says that splice is running. I get "Waiting....." with the following error from Splice Log:
java.lang.NullPointerException
at com.splicemachine.compactions.PoolSlotBooker.run(PoolSlotBooker.java:56)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Where are the troubleshooting instructions?
This was a memory issue.
solution:
Vagrant Halt
Changed Vagrantfile
vb.memory = "10000" #later 14000 -
vagrant up
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
eventually memory maxes out for any of data ingestion on a vm - will try cloud version next!
Feeding wrong values to database table of ints - indecipherable error message (repro this)
- need to do a survey of error conditions
suggestion - benchmark - will be good to have a dedicated engineer on benchmarks; I'll write the thing up and SEO it
bug:
if
splice> run 'demodata/sql/loadall.sql';
then
Causes a crash, breaks connection to database:
splice> run 'demodata/sql/loadall.sql';
splice> /home/vagrant/splicemachine/CREATE TABLE T_HEADER
(
TRANSACTION_HEADER_KEY BIGINT NOT NULL,
CUSTOMER_MASTER_ID BIGINT,
TRANSACTION_DT DATE NOT NULL,
STORE_NBR SMALLINT NOT NULL,
EXCHANGE_RATE_PERCENT DOUBLE PRECISION,
GEOCAPTURE_FLG VARCHAR(1),
PRIMARY KEY(TRANSACTION_HEADER_KEY)
);
ERROR 42X01: Syntax error: Encountered "/" at line 1, column 1.
Issue the 'help' command for general information on Splice command syntax.
Any unrecognized commands are treated as potential SQL commands and executed directly.
Consult your DBMS server reference documentation for details of the SQL syntax supported by your server.
splice> CREATE TABLE T_DETAIL
(
TRANSACTION_HEADER_KEY BIGINT NOT NULL,
TRANSACTION_DETAIL_KEY BIGINT NOT NULL,
CUSTOMER_MASTER_ID BIGINT,
TRANSACTION_DT DATE NOT NULL,
ORIGINAL_SKU_CATEGORY_ID INTEGER,
PRIMARY KEY(TRANSACTION_HEADER_KEY, TRANSACTION_DETAIL_KEY)
);
ERROR X0Y32: table 'T_DETAIL' already exists in schema 'SPLICE'.
splice> CREATE TABLE CUSTOMERS
(
CUSTOMER_MASTER_ID BIGINT PRIMARY KEY
);
ERROR X0Y32: table 'CUSTOMERS' already exists in schema 'SPLICE'.
splice> -- replace yourPath with the appropriate full path to the demodata directory (i.e. '/vagrant/user/splicemachine/demodata/data/theader.csv')
call SYSCS_UTIL.IMPORT_DATA('SPLICE' , 'T_HEADER' , null , '/home/vagrant/splicemachine/demodata/data/theader.csv' , ',' , null , null , 'yyyy-MM-dd' , null , 0 , null , null , null);
ERROR XIE0M: Table 'SPLICE.T_HEADER' does not exist.
splice> call SYSCS_UTIL.IMPORT_DATA('SPLICE' , 'T_DETAIL' , null , '/home/vagrant/splicemachine/demodata/data/tdetail.csv' , ',' , null , null , 'yyyy-MM-dd' , null , 0 , null , null , null);
ERROR 08006: A network protocol error was encountered and the connection has been terminated: the requested command encountered an unarchitected and implementation-specific condition for which there was no architected message (additional information may be available in the derby.log file on the server)
ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: A communications error has been detected: {0}.::SQLSTATE: XJ001Java exception: 'Permission denied: user=root, path="file:/home/vagrant/splicemachine/demodata/data":1002:vboxsf:drwxr-xr-x: org.apache.hadoop.security.AccessControlException'.
splice> call SYSCS_UTIL.IMPORT_DATA('SPLICE' , 'CUSTOMERS' , null , '/home/vagrant/splicemachine/demodata/data/customers.csv' , ',' , null , null , null , null , 0 , null , null , null);
ERROR 08003: No current connection.
splice> create index tdidx1 on t_detail(original_sku_category_id,transaction_dt,customer_master_id);
ERROR 08003: No current connection.
splice> create index thidx2 on t_header(customer_master_id, transaction_dt);
ERROR 08003: No current connection.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
unclear error message
if
splice> create table test (i int);
and
splice>
insert into test values a,b,c,d,5;
then
ERROR 42X04: Column 'A' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE statement then 'A' is not a column in the target table.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment