Skip to content

Instantly share code, notes, and snippets.

@rasheedamir

rasheedamir/Jenkins CI on EC2.md

Forked from jsuwo/BankAccount.java
Last active August 29, 2015 14:06
Show Gist options
  • Save rasheedamir/f32b382b67ab2cf8a880 to your computer and use it in GitHub Desktop.
Save rasheedamir/f32b382b67ab2cf8a880 to your computer and use it in GitHub Desktop.
Download ZIP
Jenkins CI on EC2
Raw
Jenkins CI on EC2.md

The practice of automated continuous deployment ensures that the latest checked in code is deployed, running, and accessible to various roles within an organization.

Jenkins is a popular continuous integration server that is easy to install and configure

Continuous Delivery is about automation of the software development process. It combines four core disciplines:

  • continuous integration,
  • continuous testing,
  • continuous deployment and
  • continuous inspection

Continuous integration has become part of the software development process in most companies. Apart from building software artefacts it also contains version control and dependency management. Continuous testing has become more and more popular as a result of better integration in CI servers like Jenkins, CruiseControl and Bamboo. Continuous deployment on the other hand has become a bottleneck for releasing software since development and operation teams are historically separated within most organizations.

How do we measure code quality? Measuring quality is about applying the concept of Continuous Inspection. Just like continuous integration makes sure your project builds and all tests pass, continuous inspection is about looking at your code and notify you when the quality decreases. Both Jenkins and Sonar have the ability to give a trend of what is going on over time.

Continuous Integration with Jenkins on Amazon EC2

Follow these tutorials:

Tools

  • Jenkins
  • Apache Tomcat Server
  • MySQL
  • Apache HTTP Server
  • Java
  • Maven
  • Nodejs & NPM
  • Bower
  • Git
  • Grunt

Connect Server

  • Given that OS is Ubuntu AMI [Amazon Machine Image] (these are tested on Ubuntu Server 14.04 LTS)
  • Download and save key pair (amazon server key pair i.e. pem file) to some location on your computer.
  • Change permission so, that the pem file is not accessible to others run: chmod 700 <file-name>.pem
  • Make a SSH script file (create a text file with extension .sh - you can name it e.g. connect--.sh). And copy following content in it. ssh -i <path-to-your-pem-file> ubuntu@<server-name> (e.g. ssh -i /home/myproject/myproject-dev.pem [email protected])
  • Make the script executable. Run this command in the terminal: chmod +x connect-<project-name>-<environment-name>.sh
  • Open the terminal and run the script: ./connect-<project-name>-<environment-name>.sh
  • If everything goes well then you should be able to connect the server!

Installing Jenkins

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
echo "deb http://pkg.jenkins-ci.org/debian binary/" | sudo tee -a /etc/apt/sources.list.d/jenkins.list
sudo apt-get update
sudo apt-get install jenkins
Verify jenkins is running: ps -ef | grep jenkins

Installing and Configuring Apache

Installing Apache

sudo apt-get install apache2
sudo a2enmod proxy
sudo a2enmod proxy_http
  • Try to access the dashboard of jenkins; simply put the server name in the browser and see what you get

Configuring Jenkins

Update Jenkins Configuration

Edit sudo vim /etc/apache2/sites-available/jenkins.conf and add following:

<VirtualHost *:80>
	ServerName HOSTNAME
	ProxyRequests Off
	<Proxy *>
		Order deny,allow
		Allow from all
	</Proxy>
	ProxyPreserveHost on
	ProxyPass / http://localhost:8080/
</VirtualHost>

Enable Jenkins Configuration

sudo a2ensite jenkins
sudo service apache2 reload

Enable Jenkins Security

Manage Jenkins --> Configure Global Security

  • Click on Enable Security ** Jenkin's own user database *** (un-select allow users to signup)
  • Logged in users can do anything
  • Click Save

This will take us to login page! Jenkins does allow to signup first user; click on Jenkins and it will show you signup page. It only allow's to sign up first time and never afterwards!

Installing MySQL

To install MySQL, run the following command from a terminal prompt:

  • sudo apt-get install mysql-server

During the installation process you will be prompted to enter a password for the MySQL root user.

  • username / password : root / root

Once the installation is complete, the MySQL server should be started automatically. You can run the following command from a terminal prompt to check whether the MySQL server is running:

  • sudo netstat -tap | grep mysql

When you run this command, you should see the following line or something similar:

  • tcp 0 0 localhost:mysql *:* LISTEN 2759/mysqld

Create an empty schema and a fmu user. Grant this fmu user permissions to create, update and delete objects for this schema. The charset of the database has to be set to "UTF-8" and the language (database and user) to "English".

  • Here is a MYSQL Script
  • To run the script: sudo mysql -u root -p < create_database.sql
  • To verify the user has been created: SELECT User FROM mysql.user; you should see user with name fmu
  • To verify the database has been created: show databases; you should see database with name fmu

Installing Tomcat

  • Create directory "tools" in home directory: sudo mkdir tools

  • Move to tools directory: cd tools

  • Run this command to download tomcat as tar (zip) file:

`sudo wget http://www.us.apache.org/dist/tomcat/tomcat-7/v7.0.55/bin/apache-tomcat-7.0.55.tar.gz`

or

`sudo wget http://apache.mivzakim.net/tomcat/tomcat-7/v7.0.57/bin/apache-tomcat-7.0.57.tar.gz`

  • Then unzip the file: sudo tar xzf apache-tomcat-7.0.55.tar.gz

  • Move to tomcat directory: cd apache-tomcat-7.0.55

  • Change tomcat default port from 8080 to 8181: cd conf then sudo nano server.xml then change <Connector port="8080" protocol... to <Connector port="8181" protocol... then save it by pressing CTRL+X. ENSURE! that port 8181 is open on the server; confirm from the guy who had setup the instance

  • Increase permgem space. Change directory to cd apache-tomcat-7.0.55\bin then create a new file named setenv.sh in the same directory using the following command: sudo nano setenv.sh then the file would be created and opened in terminal. Paste the following piece of code export JAVA_OPTS="-Dfile.encoding=UTF-8 -Xms256m -Xmx1024m -XX:PermSize=512m -XX:MaxPermSize=512m" then close the file by pressing Ctrl + x and then press Y and Enter

  • To start the tomcat execute: sudo ./startup.sh

  • To verify tomcat is running then execute: ps -ef | grep tomcat you should see an output like

root     20537     1 49 21:43 pts/0    00:00:05 /usr/bin/java -Djava.util.logging.config.file=/home/ubuntu/tools/apache-tomcat-7.0.55/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dfile.encoding=UTF-8 -Xms256m -Xmx1024m -XX:PermSize=512m -XX:MaxPermSize=512m -Djava.endorsed.dirs=/home/ubuntu/tools/apache-tomcat-7.0.55/endorsed -classpath /home/ubuntu/tools/apache-tomcat-7.0.55/bin/bootstrap.jar:/home/ubuntu/tools/apache-tomcat-7.0.55/bin/tomcat-juli.jar -Dcatalina.base=/home/ubuntu/tools/apache-tomcat-7.0.55 -Dcatalina.home=/home/ubuntu/tools/apache-tomcat-7.0.55 -Djava.io.tmpdir=/home/ubuntu/tools/apache-tomcat-7.0.55/temp org.apache.catalina.startup.Bootstrap start

Installing Java, Maven, Git, Nodejs & NPM, Bower, Grunt

Installation

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer maven git-core

sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs

sudo npm install -g bower
sudo npm install -g grunt-cli

Verification

Verify!

javac -version
mvn -version
git --version
npm --version
node --version
bower --version
grunt --version

Configuring Jenkins

Install NodeJS Plugin

From the Jenkins dashboard, navigate to 'Manage Jenkins' and then 'Manage Plugins'.

Add JDK Information

  • From the Jenkins dashboard, navigate to Manage Jenkins and then Configure System.
  • Click Add JDK and enter Name and JAVA_HOME.
  • Click Apply

Get path to javac (go to terminal and run following):

readlink -f /usr/bin/javac
or
java -version

Add Maven Information

  • From the Jenkins dashboard, navigate to Manage Jenkins and then Configure System.
  • Click Add Maven and enter Name and MAVEN_HOME.
  • Click Apply

Get path to maven (go to terminal and run following):

readlink -f /usr/bin/mvn
or
mvn -version

Add Git Information

  • From the Jenkins dashboard, navigate to Manage Jenkins and then Manage Plugins.
  • Click on the Available tab and search using the browser (CTRL+F) for git plugin.
  • Check Git Plugin and click Download now and install after restart. The install can take a while.
  • From the Jenkins dashboard, navigate to Manage Jenkins and then Configure System.
  • Add Git information Name & Path to git executable
  • Add Git user and email info under section Git Plugin.

Get path to git:

???

Configure a Jenkins Job

  • From the Jenkins dashboard, navigate to New Job.
  • Enter a 'Job Name' and select Build a maven2/3 project. Click OK.
  • Enter a Description.
  • Select Discard Old Builds.
  • Enter an amount by Max # of builds to keep.
  • Select Git and enter a Repository URL. If you wanted to track a git repo on Github, this is where you would enter the git URL.
  • Check Poll SCM and add * * * * * to Schedule. * * * * * is a cron expression which means every minute.
  • Under Build add a Maven goal in Goals and options.
???
  • Click Save.

Helpful Jenkins Plugins

  • Cloudbees Folders Plugin : This plugin allows users to create "folders" to organize jobs. Users can define custom taxonomies (like by project type, organization type etc). Folders are nestable and you can define views within folders.

Junk!

Manage Jenkins -> Configure System

  • JDK
  • Git
  • Maven
  • E-mail Notification ??

Continuous Delivery in the Cloud

New Item

  • Provide name
  • Select git
  • Open terminal
  • 'sudo su - jenkins'
  • 'ssh-keygen -t dsa' (Accept defaults...)
  • cd .ssh
  • cat id_dsa.pub
  • git ls-remote -h [email protected]:rasheedamir/fmu.git HEAD

Deployment with Apache Tomcat

The Tomcat manager application allows remote deployment to an instance of Tomcat. In this case, you need to configure Tomcat to allow access to the manager application through the plain text interface. This is accomplished by assigning the manager-script role to the credentials that will be performing deployments. This configuration depends on which Realm implementation you are using. The following example is utilizing the default MemoryRealm which reads an XML formatted file stored at $CATALINA_BASE/conf/tomcat-users.xml.

<tomcat-users>
	<user password="tomcat" roles=" manager-script" username="tomcat">
</user></tomcat-users>

Next, you need to configure your job in Jenkins to utilize the manager interface in order to deploy your recently built application. This is configured using the Post Build task plugin.

You can use wget, curl, or a similar tool to call the manager HTTP interface. It is necessary to first undeploy the application before deploying the new build. This avoids issues with deploying to an existing context path. The location of the newly built application war file can be determined by examining the output from a previous Jenkins job execution for that job.

wget "http://tomcat:password@localhost:8080/manager/text/undeploy?path=/spring-travel" -O - -q
wget "http://tomcat:password@localhost:8080/manager/text/deploy?path=/spring-travel&war=file:<path to="" war="">" -O - -q </path>

Easy Installation

Jenkins comes as a WAR file (there are also a wide variety of Linux packages available and a Windows installer) that you can drop into your favourite JEE container (Tomcat, Jetty, Glassfish etc.) or you can start the WAR file directly with 'java -jar jenkins.war'. That’s it! Now that it is up and running, you can start configuring jobs and installing plugins. It will use a folder in your $HOME to persist the configuration and store the actual jobs. No database, no special installation, no special registration. Clean and easy.

@rasheedamir
Copy link
Author

  • Add public key to github

@Juansasa
Copy link

Configure Jenkins with following

Install plugins
Install nodeJs, github plugins from Jenkins > Manage jenkins > manage plugins > available > search
Download and install after restart
Restart with: "sudo service jenkins restart” from amazon command-line

Configure plugins

Navigate to Jenkins > Manage Jenkins > Configure System
configure JDK with name of your choice for example: Java 7
Uncheck “install automatically” and point JAVA_HOME to java installation folder, for example: /usr/lib/jvm/java-7-oracle
Repeat with Git <name: Git, Path to Git executable: /usr/bin/git>, Maven <name: Maven, MAVEN_HOME: /usr/share/maven> and NodeJS <name: NodeJs, Installation directory: /usr/lib>

Generate SSH keys for Jenkins
Login to amazon and run the command sudo su - jenkins. The ‘-’ specifies a login shell, and will switch you to jenkins’ home directory (for me this was /var/lib/jenkins’).
Create a .ssh directory in the jenkins home directory: mkdir .ssh.
go to this folder: cd .ssh/
Create the public private key pair: ssh-keygen -t rsa -C "[email protected]". Do not to set a password, otherwise the jenkins user will not be able to connect to the git repo in an automated way without additional plugin.
Add the public key .pub to your Git repo.
restart jenkins: /restart or sudo service jenkins restart
verify by entering jenkins sudo su - jenkins, and run git ls-remote -h [email protected]:rasheedamir/fmu.git HEAD

@Juansasa
Copy link

Jenkins should be configured first of all and all other steps should begin while logged in as jenkins user to make sure jenkins have correct privileges to write and execute jobs freely.

  • In short before proceeding to install and creating directories, first log into the amazon instance, then login as "jenkins user" with :
    "sudo su - jenkins"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment