storage.md

Storages

  docker run -d -p 5672:5672 rabbitmq # message
  docker run -d -p 27017:27017  mongo #result
  docker run -d -p 6379:6379 redis

Note : Use PK ,FK - Star Schema Design ie., StudentID in school

1. sqlite3 : To Stores Relational Data
2. Redis
   • Store Cache Data (key-val)
   • Message Bus between application (pubsub)
3. Mongo
   • Schema Less Data
   • when no installer are present
4. Kafka (pubsub)

Properties

1. docker images and Cloud
2. Light Weight
3. cli

Sqlite3

uses

1. Stores Relational Data

Adv

1. Store Data as Relational

2. Use ORM for complex operations

3. Supports CLI operation

4. Adminer (opensource Editor - Can be run in docker)

     import sqlite3 as db
     con =db.connect("file.sqlite3")
     with con:
        con.execute("create table T (id int) ")
        con.executeMany("insert into T (id) values('?')" , data=[(1)])
        con.execute("select * from T")
   

Redis

uses

1. Stores Cache Data
2. Store Messag Bus

Adv

1. pubsub and key-value
2. Light Weight

pubsub

  import redis
  host='redis-14713.c292.ap-southeast-1-1.ec2.cloud.redislabs.com'
  #pub
  r = redis.Redis(host=host,port=14713,password='<pass>',decode_responses=True)
  r.publish("topic", "hi")
  #sub
  sub = r.pubsub()  
  sub.subscribe('topic')
  for m in sub.listen():print(m)

KeyVal

  import redis
  host='redis-14713.c292.ap-southeast-1-1.ec2.cloud.redislabs.com'
  r = redis.Redis(host=host,port=14713,password='<pass>',decode_responses=True)
  r.set('key','value')
  r.get('key').decode()

Mongo

uses

1. Stores Non Relation Data
2. Schema Less Data

Adv

1. Save directly as dictionary

   import requests,json
   url = "https://data.mongodb-api.com/app/data-iunna/endpoint/data/v1/action/"
   headers = {
       'Content-Type': 'application/json',
       'api-key': '<api-key>',
   }
   
   #ADD
   payload = {"collection": "T","database": "Test","dataSource": "Cluster0"}
   payload.update({"filter": {"1": "A" }})
   response = requests.post( url=url+"find", headers=headers, data=json.dumps(payload))
   print(response.text)
   
   #FIND
   payload = {"collection": "T","database": "Test","dataSource": "Cluster0"}
   payload.update({"documents": [ { "B": "2"}]})
   response = requests.post( url=url+"insertMany", headers=headers, data=json.dumps(payload))
   print(response.text)