Skip to content

Instantly share code, notes, and snippets.

@krfong916

krfong916/bottomline-architecture.md

Last active April 4, 2021 18:09
Show Gist options
  • Save krfong916/9583789cf071133774a0a12a04220040 to your computer and use it in GitHub Desktop.
Save krfong916/9583789cf071133774a0a12a04220040 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
bottomline-architecture.md

Architecture

Cloudfront

  • a Content Delivery Network (CDN). A CDN delivers content through nodes (data centers) that are globally distributed
  • When a user requests content that we're serving from CloudFront (our application), the user is routed to the edge location that provides the lowest latency
  • User requests content and is routed to the nearest CDN -> low latency, best performance
  • If the content is not at the edge location (content DNE at node) CloudFront retrieves the content from the origin we define (our S3 bucket)

Lambda@Edge

  1. For headers
  2. For client redirects

Suppose the Use Case of Headers

First, what are security headers and why are they important?

  • They protect against XSS, code injection, clickjacking
  • How?
  • For example:
  • X-XSS-Protection is a header that browsers respect to stop loading pages when they detect a cross-site scripting attack

Our use case: Security Header Lambda function

  1. A user requests our website
  2. If the object is cached, CloudFront returns the object (Our single-page application). If not
  3. Cloudfront requests the object from S3
  4. When S3 returns the object, our ModifyResponseHeaders lambda function triggers. Explicitly, we add security headers after the origin (S3) returns but before the cache. The resulting output is then cached for future requests at our CDN.

Suppose the Use Case of Client Redirects

Route53

IAM Roles

S3

Design Decisions

Front-End

Performance

We care about a few things when it comes to performance. Milliseconds and kilobytes.

  • Ship only the javascript and styling over the network that's necessary.
  • No Typescript - front end doesn't need it, it's a simple interface. However, the backend is a different story
  • Remove render-blocking resources i.e. optimizing our CSS and Font load.

Metrics

We measure in milliseconds

  • First Paint
  • First Contentful Paint
  • First Web Font
  • Visually Complete
  • Lighthouse Score

Async CSS

  • Synchronously loaded stylesheets block rendering, async
  • We want to remove minimum set of blocking CSS.
  • We should design with the expectati that our desired font will not load and that we must rely on our fallbacks
  • We link out to Google Font APIs, on a high-latency connection this could be bad
  • However, we preload and preconnect to the google font apis. Preconnect fetches the origin headers (our bottleneck - we're latency-bound)

Async Fonts

We must have a font display strategy for making content rendering fast. We use google fonts - how can we make the load fast?
We should want to asynchronously load all our fonts (non-blocking load so rendering can happen)

preloading the css file increases its priority preconnect warms up the fonts.gstatic.com origin

FP (First Paint): Is the critical path affected? Sequence of steps browser goes through to convert html, css, javascript, render pixels on the screen FCP (First Contentful Path): how fast can we read something? FWF (First Web Font): At what point has the first web font loaded? Visually Complete (VC): When has everything settled? like Last Web Font Lighthouse Score: Gimme the Metrics!

What's fastest? Comparatively:

  • sync loading: synchronous blocking, must wait until css file is loaded from the third part origin. Files will contain @font-face at-rules with no font-display descriptors (how a font face is displayed based on whether and when it is downloaded and ready to use)
  • async w/ display swap: makes font load async so we'll begin with the fallback then render when the font-face has been loaded. We won't see FOIT (flash of invisible text) the browser will render a fallback font until our font face is successfully loaded
  • An aside: when using a web font, we're bound to see a FOUC (flash of unstyled text), we must use a websafe font to minimize the discrepancy between sizing of websafe and web font. https://meowni.ca/font-style-matcher/
  • preconnect: warmup the fonts' origin (warmup means to load code into a new instance before any live requests reach the instance. Warmup requests reduce request and response latency i.e. initialize device readiness)
  • preload: preload the css file to increase browser rendering priority

We don't use any one strategy by itself, we use them in combination

<link rel="stylesheet"
     href="$CSS&display=swap"
     media="print" onload="this.media='all'" />

load CSS in non-blocking fashion, when the moment the file arrives, tell the browser to apply it to all contents of the page - style it The browser will place the async CSS file on lowest priority of rendering because it can load later (as opposed to sync - we must load now! Rendering will block until loaded)

Async loading is a good idea but reducing the priority of the css file has slowed rendering othe custom font - let's speed it that part up by increasing the priority

High-latency networks will slow our TTFB (time to first byte). Loading the font itself will not be slow, connecting to the origin, fonts.gstatic.com, will be

So we use a preconnect to grab headers from the fonts' origin

https://csswizardry.com/2020/05/the-fastest-google-fonts/

Performance Testing

APIs

  • We choose to expose REST APIs for our application. Our REST API style resembles SQL over HTTP (basically) - as we name resources, not actions get(users/:id) -> SELECT * FROM users WHERE users.id = id.
  • We also use nested endpoints. Why? Because

Server Code

  • We use the Dependency Inversion Principle in order to program to an interfaces and not an implementation

Testing

The role of test can be described in many ways, here are a few that we find important and use on Bottomline:

  • Tests should give a deeper understanding of the code's design
  • Tests should be behavior-driven, and not implementation-driven

What Do We Gain?

Testing gives us the ability to understand and think more clearly about code, it requires us to break our problem into smaller tasks, and it can complement documentation of what code does is or is supposed to do. Note: a higher percentage of test coverage doesn't mean safety.

Our Testing Strategy:

  • We perform an integration test to make sure that we have hooked up everything correctly

    • the integration test involves the development database because we want to ensure we have a happy path correct
    • the integration test can be specific to the platform (web integration, mobile integration)
      • consequently this will test our controllers
      • our integration tests also verifies our the freedom of our design decisions: we isolate our use cases, domain objects (entities), and services away from the device or platform we use
      • we don't directly test our controllers because they're free from logic, they're coordinators of others doing the work - so yes, we could test in isolation, but the cost to value isn't as great as what we could get out of an integration test

  • Our unit tests really live at the heart of our use case and domain objects

  • We test aggregates as they are - We wouldn't break their grouping in our use cases, so we don't break their grouping in our tests

    • they contain invariant checking, state transitions, calculations, algos etc.
    • we expect them to change when our domain changes

  • We directly instantiate the SUT (system under test)

    • Tests are specs. If the requirements changes, some tests have to change.

TDD pains and lessons-learned:

  • writing low-level unit tests that enforce poor encapsulation is a PAIN
  • what do I mean by this?
  • Writing low-level unit tests left me thinking, how should I implement the thing whileI'm writing the test
    • and then I implemented the thing with more or less the exact same code
  • What's the result?
    • I have to maintain code in two places and refactor in two places
    • the tests will fail when the implementation changes
    • the test reveals state that would otherwise be private
  • When writing a test we have to ask: are writing the test to help us flesh out implementation details, or are we testing how the object behaves? We want behavior.
    • and by behavior: I mean either the system-under-test is a delegator or it's a pure function (usually)

Notes:

  • Integration testing: making sure everything is hooked up right
    • can't test business logic without involving external dependencies
    • two kinds of dependencies:
      • ones we have control over: db/filesystem
      • we don't gave control: email SMTP, messaging bus <- those out
  • We can directly test our db because our app is the only one that has access to it, the only client - and not observable by anyone else. If db accessible by others - then we need to isolate that part and treat as an external dependency
  • we should program to interfaces
  • we should test end result, and not implementation details

Address

Security

We use a statically built SPA. We use nonces to allow script and style tags to be loaded at runtime in the client's browsers. From a web security perspective, caching nonces in Cloudfront now contributes to a threat vector. The issue is that we cache our application - therefore nonces are not being use the way that their designed (number used once). This may be an architectural issue. One consideration is to look through each script/style tag in our bundle and generate a nonce via a node.js server on each request. What are the implications? This may mean that we have to move away from serving our application being statically served from s3. Another consideration is to hash our files instead. At this time, a third consideration is to specify host-sources in our csp meta tag.

CI

Audit

https://web.dev/learn/ https://www.troyhunt.com/locking-down-your-website-scripts-with-csp-hashes-nonces-and-report-uri/

CloudWatch is AWS logging service. If we have a lambda deployed in a region and we need to debug, we have to play a guessing game of the region location of the CloudWatch logs

Sources

Explore Later

Server libraries

  • Joi: validation and support for functional error handling
  • Sequelize ORM. Perf isn't our concern, yet. No need to reinvent the wheel as well. "It's an ORM", yeah so what.

Server Test Env

Compose container networking Dockerfile and docker-compose Docker and live reloading 1 Docker and live reloading 2

Sessions

UUIDs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment