How to make pallet-hadoop-example work without #<JSchException com.jcraft.jsch.JSchException: Auth fail>

config-example.clj

  ;; We can define our compute service here... (eval at the REPL)
  (def ec2-service
    (compute-service "aws-ec2"
                     :identity "__YOUR_AWS_ACCESS_KEY__"
                     :credential "__YOUR_AWS_SECRET_KEY__"
                     ;;
                     ;; Add this environment configuration to make it work
                     ;;      
                     :environment
                     {:user {:username "ubuntu"                     ;; change the admin user to "ubuntu"
                             :private-key-path "./ec2key_rsa"       ;; path to the keys we just created
                             :public-key-path "./ec2key_rsa.pub"}}
                     ))

Instructions.md

First, generate a set of SSH keys in the local directory (./ec2key_rsa) to use for the experiment with the ssh-keygen command:

pallet-hadoop-example$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/mjul/.ssh/id_rsa): ./ec2key_rsa
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in ./ec2key_rsa.
Your public key has been saved in ./ec2key_rsa.pub.

This creates the keys for SSH.

Now, make sure they are kept private so ssh will not complain: chmod 600 ./ec2key_rsa*

Now you are ready to open a REPL and get going:

Follow the pallet-hadoop-example README with this modification: For some reason, to run on Ubuntu, we need to explicitly set the name of the admin user to "ubuntu" (in the proper map :environment :user map, see config-example.clj below. We also set the path of the keys we just generated.

Now, at the REPL eval the config and the line to set up the machines.

Follow, the config and when it tells you to SSH into the job-tracker server, use the right key and user like so (insert the correct server DNS or IP number instead of jobtracker.com):

ssh -i ./ec2key_rsa.pub [email protected]