Data Engineering Lifecycle - Mermaid Diagram

Data_Engineering_Lifecycle.md

flowchart LR
    %% Generation (outside)
    Generation[🔄 Generation]
    
    %% Core pipeline stages with single boxes
    subgraph Pipeline[ ]
        Ingestion[📥 Ingestion]
        Storage[(💾 Storage)]
        Transformation[⚙️ Transformation]
        Serving[📊 Serving]
    end
    
    %% Serving outputs (outside right)
    Analytics[📈 Analytics]
    MachineLearning[🤖 Machine Learning]
    ReverseETL[🔄 Reverse ETL]
    
    %% Flow connections
    Generation --> Ingestion
    Ingestion --> Transformation
    Transformation --> Serving
    
    %% Storage connections (foundation for all)
    Generation -.-> Storage
    Ingestion --> Storage
    Storage --> Transformation
    Storage -.-> Serving
    
    %% Serving to outputs
    Serving --> Analytics
    Serving --> MachineLearning
    Serving --> ReverseETL
    
    %% Styling for individual nodes
    classDef ingestion fill:#e8f5e8,stroke:#2e7d32,stroke-width:2px
    classDef storage fill:#fff3e0,stroke:#f57c00,stroke-width:3px
    classDef transformation fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
    classDef serving fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
    classDef generation fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
    classDef output fill:#fce4ec,stroke:#c2185b,stroke-width:2px
    
    class Ingestion ingestion
    class Storage storage
    class Transformation transformation
    class Serving serving
    class Generation generation
    class Analytics,MachineLearning,ReverseETL output

Loading