Cesare Rossi crossi202

Configure Proxy Settings to View Terradue Services Web GUIs Hosted on a Public Cloud infrastructure

Because of the usage of internal names to configure the services (e.g., Hadoop), you will need to perform the following actions:

Setup of a SSH tunnel with node,
Configure a browser proxy tool (for example FoxyProxy)

After that you will be able to view the Terradue Services Web GUIs.

Part 1. Setup of a SSH tunnel using dynamic port forwarding

Application release

Introduction

This procedure will guide you on the usage of automation tools, in order to release an Application developed within the Terradue Cloud Platform.

Prerequisites

Successfully performed the Application packaging and Github usage procedure.

MELODIES - Exploiting Open Data

Open Data Tools - Session 2: Working with Data resources

Brussels, 13rd October 2016

Prerequisites

You successfully performed the Open Data Tools - Session 1: Working with Cloud resources.

Get the API key

MELODIES - Exploiting Open Data

Open Data Tools - Session 1: Working with Cloud resources

Brussels, 13rd October 2016

Prerequisites

You have successfully performed the SSH key pair and VPN Setup (Training Preparatory Step 1/2),
You have successfully performed the Cloud Dashboard access and Sandbox deployment (Training Preparatory Step 2/2).

Get the Application

Generate and install the SSH key

SSH keys are a way to identify trusted computers without involving passwords. You can generate an SSH key and add the public key to your Terradue Cloud Platform account by following the procedures below.

Generate a new SSH key

Go to https://www.terradue.com,
Type your credentials,
Click on the top-right botton showing your username,
Click on Profile,

Installing CentOS 6.7 on Hetzner with partitions larger than 2TB

Install the CentOS 6.7 64 bit minimal

Start the server in Rescue mode [1],
Type

$ installimage

Application packaging and Github usage

Introduction

This procedure will guide you on the usage of automation tools, in order to build an Application suitable for the Terradue Cloud Platform (see these examples). Such applications have the main advantage to be self-contained. In other words, a single package (both in RPM and Docker image formats) contains the application resources and the dependencies specification. For this step, we will also add a new repository under your Github organization.

Prerequisites

A Github account. If you don't have it, you can request a new one here https://github.com/join,
An application name for the Github repository under your Github organization. We will add the dcs- prefix in order to be homogeneous with the existing repositories.

Setting up the number of mappers per job

Rationale

The number of mappers per job is a function of the number of blocks across all the files used as input for the mapreduce job. It could be necessary to setup explicitily the number of mappers per job when, for instance, the inputs are just references to files (and the input file containing the references occupies just one HDFS block).

Procedure

Add the following property to the mapred-site.xml configuration file on all the tasktrackers nodes of the Hadoop cluster:

	<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
	<feed xmlns="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:georss="http://www.georss.org/georss" xmlns:gml="http://www.opengis.net/gml"
	xmlns:owc="http://www.opengis.net/owc/1.0" xmlns:os="http://a9.com/-/spec/opensearch/1.1/"
	xml:lang="en">
	<link rel="profile" href="http://www.opengis.net/spec/owc-atom/1.0/req/core"
	title="This file is compliant with version 1.0 of OGC Context"/>
	<link rel="profile" href="http://www.opengis.net/tb13/eoc"
	title="This file is compliant with Testbed-13 EOC Thread for Application Packing"/>
	<id>http://www.opengis.net/tb13/eoc/examples/app_pkg/dcs-stemp-s2</id>

	<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
	<feed xmlns="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:georss="http://www.georss.org/georss" xmlns:gml="http://www.opengis.net/gml"
	xmlns:owc="http://www.opengis.net/owc/1.0" xmlns:os="http://a9.com/-/spec/opensearch/1.1/"
	xml:lang="en">
	<link rel="profile" href="http://www.opengis.net/spec/owc-atom/1.0/req/core"
	title="This file is compliant with version 1.0 of OGC Context"/>
	<link rel="profile" href="http://www.opengis.net/tb13/eoc"
	title="This file is compliant with Testbed-13 EOC Thread for Application Packing"/>
	<id>http://www.opengis.net/tb13/eoc/examples/app_pkg/dcs-stemp-s2</id>