Skip to content

Instantly share code, notes, and snippets.

View j-thepac's full-sized avatar

Deepak j-thepac

View GitHub Profile
@j-thepac
j-thepac / importantResources.md
Last active May 26, 2024 09:05
importantResources

Important Resources

Open Applications

  • OSX - open is a universal opener in OSX
  • Linux - has xdg-open (open bashrc > add "alias open = xdg-open" > save file >source ./bashrc)
  • Windows - use start

Online Virtual terminals

from fastapi import FastAPI,Request
from pydantic import BaseModel
import uvicorn
from smslottery import validate_bet
from fastapi.middleware.cors import CORSMiddleware
origins = ["*"]
class Data(BaseModel):
game:str
text:str
@j-thepac
j-thepac / ETL.md
Last active January 13, 2025 13:27

ETL

BigData:

  • OLTP (online Transactional processing)
  • OLAP(online analytical processing)

Data processing systems

  • OLTP (OnLine Transaction Processing) : is used for managing current day to day data information.
    • ACID
  • Atomicity(entire transaction happens at once or nothing happens)

System

  • Sop (std operation procedure)
  • Rca (root cause analysis)
  • Modular
  • Scalable
  • Scheme agnostic (No SqL) : No Schema
  • Model
  • Decouple
@j-thepac
j-thepac / kafka.md
Last active November 18, 2024 00:03

Kafka

kafkaSample

Docker YML

## docker-compose.yml (start kafka Broker and Zookeeper)

    #docker-compose up -d #detach mode
	version: '3'
	services:
 zookeeper:
@j-thepac
j-thepac / docker.md
Last active August 4, 2024 11:44
Docker

DOCKER

Env : https://labs.play-with-docker.com/

  • Docker = Container and Images
  • Docker Can be executed as
    • command line
    • dockerfile
    • docker-compose.yml

Image

Snapshot of the application containing all the libraries , supporting files etc.,

@j-thepac
j-thepac / Synapse.md
Last active February 4, 2025 11:59
Azure Synapse

Synapse

Note :

  • For JSON file make sure all new lines , blank spaces are removed.
  • Dataflow , Notebooks etc - names has to start with String.
  • blob = flat namespace , Gen2 = directories
  • Azure Active Directory (Entra ID) : Used to Create 1 Functional ID (secret ) called Serive Principle across all applications to communicate with each other
  • SP ( serivce Principle) : Managed by User ie., renew etc.,-
  • Managed identity (service principal) : managed by Azure

Kubernetes

Problem with VM:

  • Virtual machines comes with unwanted apps and services Eg: Browser
  • All process / Threads were not is user control .
  • Memory allocation was not in user control.
  • Syncing Virtual machines is not easy
  • Lot of configuration was needed
  • To delete and reconfigure was a nightmare.

Kubernetes

  • configure, automate, and manage applications using containers Image
@j-thepac
j-thepac / DP-900.md
Last active January 12, 2025 13:34
Material for Dp-900 Exam

DP 900

Azure Data Fundamentals: Explore core data concepts

  • Type: structured, semi-structured, or unstructured.
  • Data stores : File stores ,Databases
  • file Formats :
    • Delimited text files
    • JavaScript Object Notation (JSON)
    • Extensible Markup Language (XML)
  • Optimized File Format
@j-thepac
j-thepac / Coding Principles.md
Last active May 19, 2024 13:43
Coding Principles

Principles

  • Loose Coupling
  • High Cohesion
  • Change is Local
  • It is Easy to Remove

Smells

  • Rigidity ( A -> B -> C . something hardcoded in C )
  • Fragility
  • Immobility