setup mac for contrail web ui dev sandbox

setup-mac-contrail-ui-dev.md

Install brew, if it doesn't exist

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install basics via brew

$ brew install redis wget n

Alternatively, you can use nvm and containerized redis server to replace n and local redis, respectively.

Install Node.JS via n (a node version manager, similar to nvm)

$ n 0.10.35

Install a python-version XML parser

$ sudo easy_install pip && pip install --user lxml

Alternative way to install pip:

$ curl https://bootstrap.pypa.io/get-pip.py | sudo python

Go to your preferred work directory, create a sandbox folder

$ mkdir sandbox

Go into the created sandbox folder, fork below repos

$ cd sandbox
$ git clone https://github.com/Juniper/contrail-web-core.git
$ git clone https://github.com/Juniper/contrail-webui-third-party.git
$ git clone https://github.com/Juniper/contrail-web-controller.git
$ git clone https://github.com/Juniper/contrail-web-server-manager.git
$ git clone https://github.com/Juniper/contrail-web-storage.git

In forked contrail-web-core, install all 3rd-party dependencies via makefile

$ cd contrail-web-core
$ make fetch-pkgs-dev

Update config/config.global.js

• Make orchestration modules consume remote API services
• At the begining of the file, create a varible of remote API server IP

var contrail_ip = "10.84.11.2"; // For users who don't have access to the internal servers,
                                 // use "10.1.1.200"

NOTE: For those who will use "10.1.1.200", check also the usage of openVPN.

• Make below orchestration modules point to the remote server(which pulls and returns real data)

  • networkManager
  • imageManager
  • computerManager
  • identityManager
  • storageManager
  • cnfg
  • analytics

  An example:

  config.networkManager.ip = contrail_ip;

• Normally you'll point jobServer and redis to your local machine and other IP with your Contrail Installation.

• For each feature, update the feature pkg path in config/config.global.js

• By default in config, only webController is given as featurePkg

• Similarly add config.featurePkg.serverManager and config.featurePkg.webStorage if you need those as well.

Update config/userAuth.js

Check admin_user and admin_password. If they aren't filled, please fill them with valid credentials:

var auth = {};
auth.admin_user = 'admin';
auth.admin_password = 'contrail123';
auth.admin_token = '';
auth.admin_tenant_name = 'admin';

To build web controller

$ make dev-env REPO=webController

To build all the repos

$ make dev-env REPO=webController,serverManager,webStorage

Similarly to create the test-environment on each repo.

$ make test-env REPO=webController,serverManager,webStorage

Start redis

$ redis-server /usr/local/etc/redis.conf 

See the app in action

• start local server instances

Note: each server needs run in its own process

$ node jobServerStart.js
$ node webServerStart.js

• open app at http://localhost:8080

Setting up git review

• Install git review

$ pip install --user git-review

• Create your account in http://review.opencontrail.org/
• Add your ssh public key id_rsa.pub to review.opencontrail.org

NOTE: if you run into permission issues using pip to install git-review, try below alternative:

$ brew install git-review

Add gerrit remote to each repo

$ git remote add gerrit ssh://<username>@review.opencontrail.org:29418/Juniper/<repo-name>.git

Once changes are committed, issue git review.

Usage of openVPN

For developers with no access to internal servers, they need to use openVPN to access another project installation on AWS.

• Open https://52.9.112.220:943/?src=connect in browser and login with your username/password (ask the manager for the credentials)
• After log in, download and install the openVPN client.
• Once installed, provide your username/password to the client and connect to 52.9.112.220
• When connected, you can access the application on 10.1.1.200:8080 with defautl credentials (admin/secret123). For development, point your local servers to this IP address.

Cassandra is also needed. It can be ran using docker:

docker run --name cassandra -d -p 9042:9042 --restart unless-stopped cassandra

Moreover, config.dns.server_ip also has to be changed for some of the errors related connecting to 127.0.0.1:8092 to go away:

config.dns.server_ip = [contrail_ip];

Additionally, favicon file paths require modifications. Change the following two variables to resolve more errors:

/* Logo File: Use complete path of logo file location */
config.logo_file =
    "/home/grzegorz/go/src/github.com/Juniper/sandbox/contrail-web-core/webroot/img/tf-logo.png";

/* Favicon File: Use complete path of favicon file location */
config.favicon_file =
    "/home/grzegorz/go/src/github.com/Juniper/sandbox/contrail-web-core/webroot/img/tf-favicon.ico";

When working on modern testbeds, change the identityManager version to v3.0:

config.identityManager.apiVersion = ["v3.0"];

When running node webServerStart.js, I see one more error:

03/19/2019 12:10:27 PM - error: Configured/Default MOTD file read error code 34

I did not manage to solve this one

Step - 1:

• Install ruby, redis, docker, pyenv, nvm, pip
• Install node version: 0.10.48 with the help of nvm to manage multiple nodeJS (0.10.48 preferred)
• Install python latest version using pyenv to manage multiple python versions (3.8.1 preferred)
• Install XML Parser using: sudo easy_install pip && pip install --user lxml
• docker run --name cassandra -d -p 9042:9042 --restart unless-stopped cassandra

Challenges:
Python version will not change, so do this:

• pyenv global 3.8.1
• eval "$(pyenv init -)"

Step - 2:
a. create a sandbox folder and get inside that folder
b. $ git clone https://github.com/Juniper/contrail-web-core.git
$ git clone https://github.com/Juniper/contrail-webui-third-party.git
$ git clone https://github.com/Juniper/contrail-web-controller.git

**Challenges:**
    i. The repo mentioned above are moved to tf-*, make sure you clone the copy of tf-* from readme documentation.
    ii. Rename the tf-* to contrail-*, so that Symlinks target this folders.

Step - 3:
Update config.global.js inside config/ folder of contrail-web-core. It should look like below:

var config = {};
var contrail_ip = '10.84.11.6'
// var contrail_ip = '10.219.130.175'

config.orchestration = {};
/****************************************************************************
 * Specify Orchestration Model
 * Available models are:
 *  - openstack
 *  - cloudstack
 * If you do not want to specify any model, set it to 'none'
 *
 *****************************************************************************/
config.orchestration.Manager = 'openstack';

/****************************************************************************
 * This boolean flag indicates to communicate with Orchestration
 * modules(networkManager, imageManager, computeManager,
 * storageManager), should the webServer communicate using the
 * ip/port/authProtocol/apiVersion as specified in this file, or as returned
 * from auth catalog list.
 * Note: config.identityManager is not controlled by this boolean
 * flag.
 *
 * true  - These values should be taken from this config
 *         file.
 * false - These values should be taken from auth catalog list
 *
 * Default: false
 *****************************************************************************/
config.orchestrationModuleEndPointFromConfig = false;

/****************************************************************************
 * This boolean flag indicates to communicate with contrail services Api Server
 * and Analytics Server, should the webServer communicate using the
 * ip/port/authProtocol as specified in this file, or as returned  from auth
 * catalog list.
 *
 * true  - These values should be taken from this config
 *         file.
 * false - These values should be taken from auth catalog list
 *
 * Note: If this flag is set as false, we must have two services ApiServer and
 * OpServer, variables as defined in config.endpoints to be provisioned in
 * keystone
 *
 * Default: true
 ****************************************************************************/
config.contrailEndPointFromConfig = true;

/****************************************************************************
 * This boolean flag specifies wheather region list should be taken from config
 * file or from keystone endpoint
 * true  - If set as true, then keystone endpoint is taken from
 *         config.regions
 * false - If set as false, then keystone endpoint is taken from
 *         config.identityManager
 *
 ****************************************************************************/
config.regionsFromConfig = false;

/****************************************************************************
 * Below are the configs for Api Server and analytics Service type & name as
 * provisioned in keystone
 *
 * apiServiceType - Service Type for apiServer, default value is apiServer
 * opServiceType  - Service Type for analytics, default value is opServer
 *
 * Note: If there are multiple api server or analytices nodes in a specific
 *       region, then provision service type/name as ApiServer0, ApiServer1,
 *       ApiServer2 etc, similarly for analytics node: OpServer0, OpServer1,
 *       OpServer2 etc.
 *
 ****************************************************************************/
config.endpoints = {};
config.endpoints.apiServiceType = 'ApiServer';
config.endpoints.opServiceType = 'OpServer';

/****************************************************************************
 * Mapping to region name with keystone endpoint
 *
 * For example:
 * config.regions.RegionOne = 'http://nodeIp:5000/v2.0';
 * config.regions.RegionTwo = 'http://nodeIp:5000/v3';
 *
 ****************************************************************************/
config.regions = {};
config.regions.RegionOne = 'http://127.0.0.1:5000/v2.0';

/****************************************************************************
 * This boolean flag indicates if orchestrationModuleEndPointFromConfig or
 * contrailEndPointFromConfig is set as false,
 * then to take IP/Port/Protocol/Version information from auth catalog,
 * should publicURL OR internalURL will be used.
 *
 * true  - publicURL in endpoint will be used to retrieve IP/Port/Protocol/
 *         Version information
 * false - internalURL in endpoint will be used to retrieve
 *         IP/Port/Protocol/Version information
 *
 * NOTE: if config.orchestrationModuleEndPointFromConfig is set as true, then
 *       this flag does not have any effect on orchestration modules.
 *       if config.contrailEndPointFromConfig is set as true, then this flag
 *       does not have effect on contrail modules as defined in config.endpoints
 *
 *****************************************************************************/
config.serviceEndPointTakePublicURL = true;

/****************************************************************************
 * Below are the config options for all Orchestration Modules below:
 *  - networkManager
 *  - imageManager
 *  - computeManager
 *  - identityManager
 *  - storageManager
 *  - cnfg
 *  - analytics
 *
 * Options:
 * ip:
 *      IP to connect to for this Server.
 * port:
 *      Port to connect to for this server
 * authProtocol:
 *      Specify authProtocol either 'http' or 'https'
 * apiVersion:
 *      REST API Version for this server to connect to.
 *      Specify a list of Versions in array notation.
 *      Below are the supported list of apiVersion for the modules as of now:
 *      imageManager    -   ['v1', 'v2']
 *      computeManager  -   ['v1.1', 'v2']
 *      identityManager -   ['v2.0']
 *      storageManager  -   ['v1']
 *
 *      Not applicable for cnfg/analytics as of now
 * strictSSL:
 *      If true, requires certificates to be valid
 * ca:
 *      An authority certificate to check the remote host against,
 *      if you do not want to specify then use ''
 *****************************************************************************/
config.networkManager = {};
config.networkManager.ip = contrail_ip;
config.networkManager.port = '9696'
config.networkManager.authProtocol = 'http';
config.networkManager.apiVersion = [];
config.networkManager.strictSSL = false;
config.networkManager.ca = '';

config.imageManager = {};
config.imageManager.ip = contrail_ip;
config.imageManager.port = '9292';
config.imageManager.authProtocol = 'http';
config.imageManager.apiVersion = ['v1', 'v2'];
config.imageManager.strictSSL = false;
config.imageManager.ca = '';

config.computeManager = {};
config.computeManager.ip = contrail_ip;
config.computeManager.port = '8774';
config.computeManager.authProtocol = 'http';
config.computeManager.apiVersion = ['v1.1', 'v2'];
config.computeManager.strictSSL = false;
config.computeManager.ca = '';

config.identityManager = {};
config.identityManager.ip = contrail_ip;
config.identityManager.port = '5000';
config.identityManager.authProtocol = 'http';
/******************************************************************************
 * Note: config.identityManager.apiVersion is not controlled by boolean flag
 * config.orchestrationModuleEndPointFromConfig. If specified apiVersion here,
 * then these API versions will be used while using REST API to identityManager.
 * If want to use with default apiVersion(v2.0), then can specify it as
 * empty array.
 ******************************************************************************/
config.identityManager.apiVersion = ['v3'];
config.identityManager.strictSSL = false;
config.identityManager.ca = '';
config.identityManager.urlPrefix = '';

config.storageManager = {};
config.storageManager.ip = contrail_ip;
config.storageManager.port = '8776';
config.storageManager.authProtocol = 'http';
config.storageManager.apiVersion = ['v1'];
config.storageManager.strictSSL = false;
config.storageManager.ca = '';

// VNConfig API server and port.
config.cnfg = {};
config.cnfg.server_ip = [contrail_ip];
config.cnfg.server_port = '8082';
config.cnfg.authProtocol = 'http';
config.cnfg.strictSSL = false;
config.cnfg.ca = '';
config.cnfg.statusURL = "/global-system-configs";

// Analytics API server and port.
config.analytics = {};
config.analytics.server_ip = [contrail_ip];
config.analytics.server_port = '8081';
config.analytics.authProtocol = 'http';
config.analytics.strictSSL = false;
config.analytics.ca = '';
config.analytics.statusURL = "/analytics/uves/bgp-peers";

//DNS API Server and port.
/* Please note: being introspect port, SSL options for dns should come from
   config.introspect.ssl configuration
 */
config.dns = {};
config.dns.server_ip = [contrail_ip];
config.dns.server_port = '8092';
config.dns.statusURL = "/Snh_PageReq?x=AllEntries%20VdnsServersReq";

// vcenter related parameters
config.vcenter = {};
config.vcenter.server_ip = [contrail_ip];         //vCenter IP
config.vcenter.server_port = '443';             //Port
config.vcenter.authProtocol = 'https';          //http or https
config.vcenter.datacenter = 'vcenter';          //datacenter name
config.vcenter.dvsswitch = 'vswitch';           //dvsswitch name
config.vcenter.strictSSL = false;               //Validate the certificate or ignore
config.vcenter.ca = '';                         //specify the certificate key file
config.vcenter.wsdl = '/Users/yourname/workspace/sandbox/contrail-web-core/webroot/js/vim.wsdl';

/*****************************************************************************
 * The below configurations descibe the SSL options for connecting to different
 * introspect port.
 *
 * enabled:
 *      Boolean flag to enable or disable ssl while connecting to different
 *      introspect port
 * key:
 *      Private key to use for SSL
 * cert:
 *      Public x509 certificate to use
 * ca:
 *      A string, Buffer or array of strings or Buffers of trusted certificates
 *      in PEM format. These are used to authorize connections.
 * strictSSL:
 *      If true, the server certificate is verified against the list of
 *      supplied CAs
 *****************************************************************************/
config.introspect = {};
config.introspect.ssl = {};
config.introspect.ssl.enabled = false;
config.introspect.ssl.key = '';
config.introspect.ssl.cert = '';
config.introspect.ssl.ca = '';
config.introspect.ssl.strictSSL = false;

/* Job Server */
config.jobServer = {};
config.jobServer.server_ip = '127.0.0.1';
config.jobServer.server_port = '3000';

/* Upload/Download Directory */
config.files = {};
config.files.download_path = '/tmp';

/* Cassandra Server */
config.cassandra = {};
config.cassandra.server_ips = [contrail_ip];
config.cassandra.server_port = '9042';
config.cassandra.enable_edit = false;
config.cassandra.use_ssl = false;
config.cassandra.ca_certs = '';

/* KUE Job Scheduler */
config.kue = {};
config.kue.ui_port = '3002'

/* IP List to listen on */
config.webui_addresses = ['0.0.0.0'];

/* Is insecure access to WebUI?
 * If set as false, then all http request will be redirected
 * to https, if set true, then no https request will be processed, but only http
 * request
 */
config.insecure_access = false;

// HTTP port for NodeJS Server.
config.http_port = '8080';

// HTTPS port for NodeJS Server.
config.https_port = '8143';

// Activate/Deactivate Login.
config.require_auth = false;

/* Number of node worker processes for cluster. */
config.node_worker_count = 1;

/* Number of Parallel Active Jobs with same type */
config.maxActiveJobs = 10;

/* Redis DB index for Web-UI */
config.redisDBIndex = 3;

/* Retry time for reading servers list recursively */
config.CONTRAIL_SERVICE_RETRY_TIME = 300000; //5 minutes

/* WebUI Redis Server */
config.redis_server_port = '6379';
config.redis_server_ip = ['127.0.0.1'];
config.redis_dump_file = '/var/lib/redis/dump-webui.rdb';
config.redis_password = '';

/* Logo File: Use complete path of logo file location */
config.logo_file = '/Users/yourname/workspace/sandbox/contrail-web-core/webroot/img/tf-logo.png';

/* Favicon File: Use complete path of favicon file location */
config.favicon_file = '/Users/yourname/workspace/sandbox/contrail-web-core/webroot/img/tf-favicon.ico';

config.featurePkg = {};
/* Add new feature Package Config details below */
config.featurePkg.webController = {};
config.featurePkg.webController.path = '/Users/yourname/workspace/sandbox/contrail-web-controller';
config.featurePkg.webController.enable = true;

/* Enable/disable Stat Query Links in Sidebar*/
config.qe = {};
config.qe.enable_stat_queries = false;

/* Configure level of logs, supported log levels are:
 debug, info, notice, warning, error, crit, alert, emerg
 */
config.logs = {};
config.logs.level = 'debug';

/******************************************************************************
 * Boolean flag getDomainProjectsFromApiServer indicates wheather the project
 * list should come from API Server or Identity Manager.
 * If Set
 *      - true, then project list will come from API Server
 *      - false, then project list will come from Identity Manager
 * Default: false
 *
 ******************************************************************************/
config.getDomainProjectsFromApiServer = false;
/*****************************************************************************
 * Boolean flag L2_enable indicates the default forwarding-mode of a network.
 * Allowed values : true / false
 * Set this flag to true if all the networks are to be L2 networks,
 * set to false otherwise.
 *****************************************************************************/
config.network = {};
config.network.L2_enable = false;

/******************************************************************************
 * Boolean flag getDomainsFromApiServer indicates wheather the domain
 * list should come from API Server or Identity Manager.
 * If Set
 *      - true, then domain list will come from API Server
 *      - false, then domain list will come from Identity Manager
 * Default: true
 * NOTE: if config.identityManager.apiVersion is set as v2.0, then this flag
 *       does not have any effect, in that case the domain list is retrieved
 *       from API Server.
 *
 *****************************************************************************/
config.getDomainsFromApiServer = false;

config.jsonSchemaPath = "/Users/yourname/workspace/sandbox/contrail-web-core/src/serverroot/configJsonSchemas";
config.connectedAppsInfo = {};
//This is global flag which enables links (display links in footer)
// to all the connectedApps mentioned, however we can
// override this by using enable flag (like config.connectedAppsInfo.AppFormix.enable)
// in the app object
config.connectedAppsInfo.enable = false;
config.connectedAppsInfo.AppFormix = {};
config.connectedAppsInfo.AppFormix.url = '';
config.connectedAppsInfo.AppFormix.enable = false;

config.server_options = {};
config.server_options.ciphers =
    'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:' +
    'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:' +
    'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:' +
    'ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:' +
    'ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:' +
    'AES256-SHA';
config.server_options.key_file = '/Users/yourname/workspace/sandbox/contrail-web-core/keys/cs-key.pem';
config.server_options.cert_file = '/Users/yourname/workspace/sandbox/contrail-web-core/keys/cs-cert.pem';

// Export this as a module.
module.exports = config;

Step - 4:
a. Get inside contrail-web-core repo
First Time users, do:
- make fetch-pkgs-dev
- make dev-env REPO=webController
Returning users should clear the cache:
i make clean
ii. make clear-cache-dev
iii. rm -rf /tmp/cache/yourname/webui_third_party/
(Don't get confused, it is webui_third_party only, dont change to contrail_webui_third_party here)
iv. make fetch-pkgs-dev
v. make dev-env REPO=webController

Step - 5:
Make sure you are in 0.10.48 versions and
a. Start redis server locally: redis-server --port 6379 in a separate terminal
b. Get inside contrail-web-core folder
- Start jobserver: node jobServerStart.js
- Start webserver: node webServerStart.js

Step - 6:
Your application should be running in https://localhost:8143