Skip to content

Instantly share code, notes, and snippets.

@juanpaulo
Last active November 24, 2015 08:31
Show Gist options
  • Save juanpaulo/ac866ab24a3bbf3ebe9f to your computer and use it in GitHub Desktop.
Save juanpaulo/ac866ab24a3bbf3ebe9f to your computer and use it in GitHub Desktop.
Full Stack Toronto (Day 1)

A DevOps Pipeline

Steve Pereira (@SteveElsewhere)

Source: https://speakerdeck.com/stevepereira/a-devops-pipeline

Why would we ship 100 times a day?

  • Creating value
  • Safer and more consistent

Pipeline Characteristics

  • Natural start and end
  • Starts with source control
  • Initiate with each change
  • Run in parallel and/or sequentially
  • Success means code in production

Continuous Delivery (you're able to ship in production if you want to)

  • Change
  • Validate
  • Stabilize
  • Release

Process

  • Code
  • Test
  • Build
  • Deploy

Deploy directly > CI > CDelivery > CDeployment

Why is it crucial?

Docker - High change, high turnover

Overview

  1. Run unit tests
  2. Build the image that we use to build our application
  3. Run the build container and compiling our application
  4. Build the image that we run and deploy
  5. Push the final image to a docker registry

Quality Assurance (Test Env)

  • Static Analysis
  • Integration
  • Functional
  • Regression
  • Cross-Device

Quality Assurance (Production)

  • Smoke
  • Security
  • Performance
  • Exploratory
  • Acceptance

Quality Assurance (Post-production)

  • Smoke
  • A/B

CI Onsite

  • Jenkins
  • Gitlab CI
  • Go.CD
  • Strider
  • Team Foundation Server (Windows)

CI Online

  • Shippable
  • Codeship
  • Circle CI
  • Travis
  • Appveyor

Deployment

  • Pull from hub
  • Config management

Where we go from Here

  • Automation
  • Age of the bots
  • ChatOps (Collabo-Tomation)
  • ChatOps Visibility

Advices

  • Think about the edges

Other considerations

  • Scheduling
  • Storage
  • Service-discovery
  • Logging
  • Monitoring
  • Networking
  • Healthcheck
  • Rollback
  • Graceful failure
  • Secrets
  • Scaling
  • Cron

Change as little as possible

Aim for specifics

Think about security

Start Now

Evolving High Availability at Braintree

Lionel Barrow (@LionelBarrow)

Source: http://eventmobi.com/api/events/9565/documents/download/f8d4ebbd-c26f-4b6d-beb4-29f03204e50a.pdf/as/full%20stack%20toronto.pdf

Point being...

  • Very write-heavy load
  • Many requests require us to call out somewhere
  • Dropped requests directly mean a loss of money for the custome

Basically

  • More or less standard web architecture
  • Sharded relational database (Postgres)
  • Tiered load balancers and proxies
  • Some stuff irrelevant to this talk (e.g. )

What is high availability?

The first enemy: chaos

  • Individual things break: have more than one
  • Use indirection to make many things appear as one
  • Monitor things to make sure they're healthy
  • Take automated action to repair damage

Load balancing motivation

  • We want redundant web servers
  • We could have each client connect to a different server,
  • ...

Load balancing

  • Need two components: a health metric and a decision-maker in the load balancer
  • Health metrics are simple. LB decision-making can be harder, but not too bad.
  • Alternative strategies such as round-robin and ...

How we do it

  • Litmus paper - simple, extensible health check endpoint (github.com/braintree/litmus_paper)
  • Big brother - load balancing plugin that reads from litmus paper (github.com/braintree/big_brother)

Philosophy vs Chaos

  • The system quickly becomes complex.
  • Normal conditions are what is easy; problem conditions are what matter.
  • Individual components must be simple...

How we do it

  • Load balancers form a pacemaker cluster. At any given time one is the leader.
  • LBs route using Linux IPVS
  • All LBs advertise the same virtual IP
  • ...

The second enemy: change

  • System doesn't just need to stay up - it needs evolve
    • Deploy new code
    • Change database schema
    • Change network topology
  • Maintenance page is not OK!
  • Most changes are deployments and migrations
  • Rule of thumb: do as much as possible beforehand
    • Push new code to all servers before running it
    • Use...

Change and hope

  • We can make changes without dropping requests
  • No two versions of the code run simultaneously
  • Major caveats:
    • Op must be fast
    • Capacity must be >> ...

Why rip out the broxy?

  • Operational problems
  • Serializing requests in and out of Redis is easy at first, then hard
  • Mixing push and pull workflows creates operational complexity
  • Rails dispatchers don't even listen to HTTP
  • Complexity in the broxy code base
  • Wanted to do rate-limiting and monitoring, but broxy doesn't have all the information it needs
  • Failure creates self-reinforcing performance problems
  • It got less useful
  • Starting a new Rails
  • ...
  • The alternatives got better
  • We started using PgBouncer for connection pooling to Postgres
  • HAproxy to replace the web front end component
  • The work to replace it completed this spring and took ~3 months.

Change v2

  • Most changes now have to follow the general pattern of: add the new, migrate to the new, remove the old
  • This is significantly more engineering work but also much safer and more reliable
  • Only possible if deployments are very common and...
  • The ...

Postgres is awesome

  • No read/write lock when adding/removing/renaming a column
  • CREATE INDEX ... CONCURRENTLY;
  • Transactional schema changes!

Bringing it all together

  • HA is not easy, but it is possible.
  • Most things are straightforward; some are not.
  • Prefer simple, understandable components.
  • Think a lot about failure cases.

Making the Switch Ruby on Rails to Node.js + Express

Kendrick Coleman (@KendrickColeman)

Source: http://eventmobi.com/api/events/9565/documents/download/23ef4f8f-cd27-4843-ba8b-d66cc3ad304f.pdf/as/Making-The-Switch-Ruby-On-Rails-To-NodeJS-Express.pdf

Why did i love rails so much

  • completely OSS
  • support was easy
  • batteries included
  • scaffolding

Why did i hate rails?

  • SQLite vs MySQL vs PostgreSQL
  • NoSQL Support outside of core
  • Model Association!
  • Object migrations
  • Bootstrap wonkiness
  • Deployments

Why did i switch?

  • the ruby language was getting pooed on
    • rails were only good for small deployments or POCs
    • backend devs were switching to other languages
    • scaling concerns
    • javascript is ubiquitious
    • i needed to branch out
    • learn about this thing called "callback hell"

My journey

  • There was never a comprehensive tutorial for the "Rails-minded"
  • Series of trial and error
  • Culminatin of countless blog posts
  • I need consistency

Where's my scaffolding

  • opted to not start with Sails or other frameworks
  • no batteries included
  • need to build out the entire CRUD & REST structure

In the beginning... there was npm

  • just like installing rails via gem, install express via npm
$ npm install -g express
...

Install dependencies

  • mongoose, body-parser, method-override

Create the controller

  • DO NOT STICK EVERYTHING into index.js. Leave index.js for front page stuff.
  • Create a new file in the routes folder...

ES6 Easy Wins

Ryan Christiani (@RChristiani)

History

  • 1999 - ES3
  • ES4 Abandoned
  • 2009 - ES5: A subset of the features proposed from ES4
  • 2015 - ES6/ECMAScript 2015 released
  • 2016 - ES7/ECMAScript 2016 to be released

let

  • The let keyword allows us to block scope our variables
  • Currently we have function scope
  • A variable created with the let keyword will be bound to the block it was created in
  • ...
  • Again, using let will scope in to that block.

const

  • read-only value, or constant value
  • also block scoped.

Template strings

  • backticks and ${}

Arrow Functions

let add = (a,b) => {
  return a + b;
};
  • No need for the function keyword
let add = (a,b) => { return a + b };
OR
let add = (a,b) => a + b;
  • Lexical scope (parent scope)

Functional Programming

Spread Operator

let number = [2,3,4,5];
console.log(...numbers);
// 2,3,4,5;

Rest Parameters

let numbers = [2,3,4,5,6,7];
let add = (...nums) => {
  return nums.reduce(
    (prev,num) => prev += num
  );
}

add(...numbers);
???

Modules

  • native modules
  • import and export keywords

Import

import React from 'react';

Export

export default function() { ... }
`import React from 'react'`;

Destructuring

import { Router, Route, Link } from 'react-router';

class

  • can use extends, constructor, super, etc.

Promises

  • native implementation of Promises
  • AJAX, asynchronous stuff
let promise1 = new Promise(...);
let promise2 = new Promise(...);

Promise.all([promise1,promise2])
  .then(data => {
     ...
  }
 ...

node.js

BABEL

Gulp

More resources

http://leanpub.com/understandinges6 http://leanpub.com/exploring-es6 http://babeljs.io/docs/learn-es2015

Evolving Web Applications with Module Based Design Components

Mike Kivikoski (@mkivikoski)

Source: https://speakerdeck.com/mkivikoski/evolving-web-applications-with-modular-design-components

Keep everyone in sync with styles => Module Design Component

  • Lego as metaphor
  • ...

Benefits

  • Cohesion
  • UI/UX Patterns
  • Responsive Design
  • Testing
  • Accessibility
  • Performance

Good & bad examples

Why does this happen?

  • Company merges/acquisitions
  • Segregated departments
  • Legacy code
  • ...

Stop designing screens

  • Start designing components

Atomic Design

Atomic Design

Build faster (and easier)

  • Sketch layouts with pre-build components

Benefits

  • Decrease design time
  • Faster testing of ideas
  • Quicker Stakeholder approval

How to start

  • Use .classes instead of IDs
  • Single Responsibility Principle
  • Allow classes to combine
  • Use variables
  • Review everything

Harry Roberts, @csswizardry

Use variables for colors

  • color palettes
  • @gray-100, ...
  • @primary, @secondary, @tertiary

Media queries

// use a media query
.nav-screen-small{
  display: block
  
  @media (min-width: $screen-medium){
    display: none;
  }
}

// or create a sass mixin...
...

File organization

  • Files based on components, not screens
    • alert.less
    • buttons.less
    • forms.less

Use a grid

  • (e.g. bootstrap grid)

Check outputted CSS

  • Ensure outputted CSS is slim
  • Ensure CSS follows conventions

Show errors

Review everything

  • Variables used
  • Writing syntax
  • Component is reusable
  • Reused component follows convention

Styleguide CSS

  • Connect it to the...
    • KSS
    • pattern lab
    • fabricator

Living Styleguide

  • salesforce design system

Documentation

  • Writing/syntax examples
  • Show each combo
  • Write for a nOOb
  • Don't let it die
  • Let it evolve

Writing Example

  • good vs. bad

Component

  • e.g. Currency Filter
  • How to use, How to install

Making changes

  • Don't make one offs
  • Change components globally
  • Create new class, if needed

Refactoring

  • Don't be afraid
  • Refactor everything
  • Don't take it personal
  • Refactor often

Regression testing

  • BackstopJS
  • PhantomCSS
  • Resemble.js
  • PhantomJS
  • SlimerJS

When to start regression

  • After initial components are created
  • Otherwise, you may go mad
  • Try to test components, not screens

Deeper dive

  • css-tricks.com
  • davidwalsh.name

Frameworks

  • Bootstrap
  • Foundation
  • Bourbon

Communication

  • Make the styleguide public to team
  • Give people freedom

Play nice with others

  • The name 'LEGO' is an abbreviation of the two Danish words "leg godt", meaning "play well".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment