Skip to content

Instantly share code, notes, and snippets.

@rollwagen

rollwagen/dynamodb_data_modelling.md

Created October 7, 2021 08:06
Show Gist options
  • Save rollwagen/7a1c991f2b3418b1e3a038f4fb727d0c to your computer and use it in GitHub Desktop.
Save rollwagen/7a1c991f2b3418b1e3a038f4fb727d0c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
dynamodb_data_modelling.md

Dynamo DB Data Models and Modelling

It’s a bad idea to model your data in DynamoDB the same way you model your data in a relational database. The entire point of using a NoSQL datastore is to get some benefit you couldn’t get with a relational database. If you model the data in the same way, you not only won’t get that benefit but you will also end up with a solution that’s worse than using the relational database!

from Alex DeBrie: The DynamoDB book

General / Note on Keys

  • Primary key:

    • Simple primary key (partition key) or
    • Composite primary key (partition key + sort key)
    • primary keys are critical: all your access patterns will be driven off your primary key; really think about your primary keys
    • for item based actions (write, update, delete) you must provide the entire primary key (if composite both partition key and sort key)
    • for queries must provide primary, sort key is optional

  • Secondary Indexes

    • define as required/needed by access patterns

Data Modelling

Basics / Process

    1. Start with an ERD (Entity Relationship Diagram)
    • what entities do exist in the application and how do they relate to each other (one-to-one, one-to-many, etc)
    • Entities
    • Relationships
    • Attributes
    • Collections?
    1. Define your access patterns
    • how will the entities be used, how will they be fetched, how will they be manipulated...write down the access patterns
    • examples:
      • Shopping: Get user profile, Get single order and order items, Get orders for user by status
      • Social: Follow User, View Followers for User
    1. Design your primary keys & secondary indexes (local/global)
    • can be tricky to start; however, don't be afraid to iterate on this at this stage
    • try to start with a 'core' entity (e.g. Order or User for a shopping system)
    • define/map how the access patterns are addressed; see also 'Filtering patterns' in slides/video

Data Modeling Guidelines and Tools

  • Denormalize
  • Build collections (set of items same partition key but different sort key)
  • Primary Patterns in Tables (can write)
  • Secondary Pattrerns in GSIs
  • Iterate

Relationship strategies / One-to-many relationsships

Denormalize - Attribute (list or map)

  • consider e.g. when the number is 'bound' e.g. a user can have a max of five addresses
  • example see slide #50

Primary key + query

  • use case / relationsship 'User has many Orders' (not bound)
  • example see slide #54

Secondary index + query

Filtering

  • applied after key query expression
    • First - reads the entiry set / all the data
    • Second - filters the data / applies
  • consumption (cost!) associated to 1.
  • bulk of 'filtering' should be done using keys

Note on coming from a relational world

  • 'Normalization' - 3rd normal form etc...forget about it; denormalize so it's in the shape you want / need it
  • 'JOINs': do not exist
  • One entity type per table vs multiple entities/types per table in DynamoDB
    • e.g. Primary Key 'PK, Sort Key 'SK' as generic names and then prefix values with "USER#joe" (PK) and "#PROFILE" (SK) for user item
    • consider doing an entity chart (approx at 22:30 in below video, slide #46)

References / Links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment