100 days mongodb

100-days-mongodb.md

100 days mongodb

Data Modeling Methodology

Overall

Input

• Scenarios
• Business Domain Experts
• Production Logs & Stats
• Data Modeling Experts

Workload

• size data
• quantify ops
• qualify ops

Relationships

• identify
• quantify
• embed or link

Apply Schema Design Patterns

• recognize
• apply

Schema

• Queries
• Indexes
• Data sizing
• Operations
• Assumptions

Recap

Methodologies

Performance criteria

• Sharding : because of data size
• Read & Writes
• Ton of operations
• Operations per second
• Required latency
• Pinning attributes queries

=> More collections

1st: Identify your workload 2nd: Embed or Link

Identify Workload

Relationship

Embed vs Link

One to Many

Embeded in One Side

Embeded in Many Side

One to Many Reference in the One Side

One to Many Reference in the Many Side

Many To Many Relationship

Many to Many, Embed in main Side

Many to Many, Reference in main Side

Many to Many, Reference in secondary side

One to One

One to One Relationship

One to One Embed

One to One Reference

Sample

From

To

One to Zillions Relationship

Solution

Patterns

Reusable solutions of knowledge

• Data Modeling
• Schema Design

Topics

• Optimize large documents with Subset Pattern
• Use the Computed Pattern to avoid the repetitive computations
• Structure similar fields with the Attribute Pattern
• Handle changes to your deployment with no downtime with a Schema versioning Pattern

Datatypes

ObjectId

mongosh.exe
test> x = ObjectId()
ObjectId("62a025dca5ad6127dbacd972")
test> x.toString()
62a025dca5ad6127dbacd972
test> x.getTimestamp()
ISODate("2022-06-08T04:30:20.000Z")
test> x.toString().length;
24
test>
ObjectId("62a02802a5ad6127dbacd975")
test> ObjectId(Date.now()/1000)
ObjectId("62a0280ea5ad6127dbacd976")
test> ObjectId(Date.now()/1000)
ObjectId("62a02811a5ad6127dbacd977")
test> db.examples.insert({"_id":ObjectId(), "title":"A", "artist":"B", "room": "r", "spot":"s", "on_display":true, "in_house": true, "events":[{"k":"e1","v": new Date()}]});
DeprecationWarning: Collection.insert() is deprecated. Use insertOne, insertMany, or bulkWrite.
{
  acknowledged: true,
  insertedIds: { '0': ObjectId("62a02b56a5ad6127dbacd979") }
}
test> db.examples.find({})
[
  {
    _id: ObjectId("62a02b56a5ad6127dbacd979"),
    title: 'A',
    artist: 'B',
    room: 'r',
    spot: 's',
    on_display: true,
    in_house: true,
    events: [ { k: 'e1', v: ISODate("2022-06-08T04:53:42.776Z") } ]
  }
]

Handling Duplication Staleness

Good things come at a cost

Duplication

Staleness

Referential Integrity

Attribute Pattern

Solution

Another case

Recap

Extended Reference Pattern

Too Many Joins Nightmares

Solution

Recap

Subset Pattern

MongoDB tries to optimize the use of RAM by pulling in memory the documents that it needs from the disk to the RAM. When there is no more available, it evicts pages that contains the document it doesn't need anymore to make room for more document it needs to process at the moment

Solution

Split Collections

Computed Pattern

Mathematical Operations

Fan Out Operations

It means to do many tasks to represent one logical tasks

1. Fan Out of Reads
2. Fan Out of Writes

Shared photos

Role up Operations

Role up Operations: we merged data together. Eg: grouping categories in a parent category will be a roll-up, grouping time-based data from small intervals to large ones -> reporting for hourly, daily, monthly, yearly summaries.

Recap

Bucket Pattern

Problems:

10M devices send data every minutes. How do we store?

Maximum : 64KB/document

Example Collaboration Platform

Row Oriented vs Column Oriented

Recap

Schema versioning Pattern

Issues

• Memories of modifying the schema of your relational database
• Performing complex tasks under pressure, so you could limit the amount of downtime for your users?

Solution

Recap

Tree Patterns

Questions

List of patterns

Parent References

Who are the ancestors of Node X ?

Who reports to Y ?

Changes all categories under N to P ?

Child References

Array of Ancestors

Materialized Paths

Solution

Mixed Patterns

Summary

Accessment

Initial model

{
  "_id": "<objectId>",
  "name": "<string>",
  "role": "<string>",
  "department": {
    "name": "<string>",
    "id": "<objectId>"
  }
}

Issue

Polymorphic Pattern

Grouping Objects Together

Sample

Single View

Summary

Approximation Pattern

Outlier Pattern

Eg: a favorite single may have more than 1 million followers in the app

More