price-matching-v1

price-matching-v1.md

Mumzworld Price Matching Service v2

A sophisticated Laravel 12 API service that automatically monitors competitor prices and applies intelligent price matching rules for Mumzworld's e-commerce platform across UAE and KSA markets.

🚀 Quick Start

Prerequisites

• Docker & Docker Compose
• Git

Setup Instructions

1. Clone the Repository

   git clone <repository-url>
   cd mumzworld-price-matching-service-v2

2. Install Dependencies

   composer install

3. Environment Configuration

   cp .env.example .env

   Required Environment Variables:

   # Database
   DB_CONNECTION=mysql
   DB_HOST=mysql
   DB_PORT=3306
   DB_DATABASE=price_matching_db
   DB_USERNAME=sail
   DB_PASSWORD=password
   
   # Redis (Queues & Coordination)
   REDIS_HOST=redis
   REDIS_PASSWORD=null
   REDIS_PORT=6379
   
   # Price Matcher Configuration
   PRICE_MATCHER_CRAWL_TTL=3600
   PRICE_MATCHER_MATCH_LOCK_TTL=3000
   PRICE_MATCHER_DISPATCH_LOCK_TTL=3900
   PRICE_MATCHER_DEFAULT_MARGIN_CAP=0.15
   
   # External Crawler API
   CRAWLER_API_BASE_URL=http://crawler-service:3000
   CRAWLER_API_TIMEOUT=30
   CRAWLER_API_RETRY_ATTEMPTS=3
   
   # Queue Configuration
   QUEUE_CONNECTION=redis
   HORIZON_BALANCE=auto

4. Start Docker Services

   docker-compose up -d

5. Run Database Migrations

   docker-compose exec app php artisan migrate

6. Seed Database with Price Matching Data

   docker-compose exec app php artisan db:seed --class=PriceMatchingDataSeeder

7. Start Laravel Horizon (Queue Processing)

   docker-compose exec app php artisan horizon

8. Verify Installation

   # Check queue workers
   docker-compose exec app php artisan horizon:status
   
   # Check imported data
   docker-compose exec app php artisan tinker
   >>> App\Models\MumzworldProduct::count()
   >>> App\Models\Competitor::count()

🏗️ System Architecture

Tech Stack

• Framework: Laravel 12 (API-only)
• Runtime: PHP 8.3
• Database: MySQL 8.0
• Cache/Queues: Redis 7.0
• Queue Management: Laravel Horizon
• Containerization: Docker & Docker Compose
• External Integration: REST API crawler service

Project Structure

app/
├── Console/Commands/          # Artisan commands for price matching
├── Http/Controllers/          # API controllers (if needed)
├── Jobs/                      # Queue jobs for crawling & matching
├── Models/                    # Eloquent models for price data
├── Services/                  # Business logic services
└── Providers/                 # Service providers

database/
├── migrations/                # Database schema migrations
└── seeders/                   # CSV data import seeders

docs/
├── input-data/               # CSV data files for seeding
└── price-matching-locks.md   # Lock mechanism documentation

Queue Architecture

graph TB
    subgraph "Horizon Supervisors"
        CS[Crawler Supervisor<br/>40 workers<br/>crawls queue]
        MS[Match Supervisor<br/>10 workers<br/>matching queue]
        RS[Report Supervisor<br/>4 workers<br/>reports queue]
    end
    
    subgraph "Job Types"
        DCJ[DispatchCrawlCompetitorJob]
        DMJ[DispatchCrawlMumzworldJob]
        EPJ[ExecutePriceMatchingJob]
        GRJ[GeneratePriceMatchingReportJob]
    end
    
    CS --> DCJ
    CS --> DMJ
    MS --> EPJ
    RS --> GRJ

Loading

🔄 Core Data Flow

Price Matching Process

sequenceDiagram
    participant S as Scheduler
    participant DC as DispatchCrawlsCommand
    participant Q as Queue System
    participant R as Redis
    participant C as CrawlerAPI
    participant PM as PriceMatcherService
    participant DB as Database

    S->>DC: Hourly trigger
    DC->>DB: Get all SKU×Country products
    DC->>R: Set expected crawls
    DC->>Q: Dispatch external competitor jobs
    
    Q->>C: Crawl competitor URLs
    C-->>Q: Return competitor prices
    Q->>DB: Store competitor prices
    Q->>R: Mark crawl complete
    
    R->>R: Check if all crawls done
    R->>Q: Dispatch price matching job
    
    Q->>PM: Execute price matching
    PM->>DB: Get business rules
    PM->>PM: Apply matching logic
    PM->>DB: Log price match decision
    PM->>Q: Dispatch report generation
    
    Q->>DB: Generate CSV reports

Loading

Dual-Lock Mechanism

graph LR
    subgraph "Dispatch Lock (65min)"
        DL[Prevents duplicate<br/>SKU dispatching<br/>per hour]
    end
    
    subgraph "Match Lock (50min)"
        ML[Prevents concurrent<br/>price matching<br/>same SKU]
    end
    
    subgraph "Redis Keys"
        DK["dispatch:lock:{sku}:{country}"]
        MK["match:lock:{sku}:{country}"]
    end
    
    DL --> DK
    ML --> MK

Loading

📊 Database Schema

Core Tables

Table	Purpose
competitors	External competitors (Amazon, Noon) + Internal (Mumzworld)
mumzworld_products	Product catalog with cost/selling prices
competitor_product_urls	URLs to crawl for each SKU×competitor×country
competitor_prices	Crawled competitor pricing data
mumzworld_price_snapshots	Live mumzworld price snapshots

Business Rules Tables

Table	Purpose
price_match_exceptions	SKUs excluded from price matching
margin_caps	Minimum margin requirements per SKU
price_up_constraints	SKUs blocked from price increases
preferred_competitor_overrides	Priority competitor selection
price_match_logs	Complete audit trail with price direction tracking

Data Relationships

erDiagram
    competitors ||--o{ competitor_product_urls : has
    competitors ||--o{ competitor_prices : stores
    mumzworld_products ||--o{ competitor_product_urls : maps_to
    mumzworld_products ||--o{ price_match_logs : generates
    competitor_prices ||--o{ price_match_logs : influences

Loading

🎯 Business Logic

Price Matching Rules (Applied in Order)

1. Exception Check: Skip if SKU in exceptions list
2. Candidate Selection: Choose preferred competitor OR cheapest discounted price
3. Price-Up Guard: Block increases if SKU in price-up constraints
4. Margin Cap Validation: Ensure margin ≥ minimum required margin
5. Price Application: Update if all checks pass

Competitor Types

• External Competitors (is_internal = false): Amazon, FirstCry, Noon, etc.
• Internal Competitors (is_internal = true): Mumzworld UAE/KSA for live price validation

🔧 Available Commands

Price Matching Operations

# Dispatch crawl jobs for all products
php artisan price-matcher:dispatch-crawls

# Dispatch for specific SKUs/countries
php artisan price-matcher:dispatch-crawls --sku=SKU123 --country=uae

# Generate reports for specific batch
php artisan price-matcher:generate-report {batch_id}

# Check lock status (debugging)
php artisan price-matcher:check-locks
php artisan price-matcher:check-locks SKU123 uae

Data Management

# Import all CSV data
php artisan db:seed --class=PriceMatchingDataSeeder

# Import specific data types
php artisan db:seed --class=CompetitorSeeder
php artisan db:seed --class=MumzworldProductSeeder
php artisan db:seed --class=CompetitorProductUrlSeeder

📋 CSV Data Import

The system imports data from the following CSV files in docs/input-data/:

File	Purpose	Format
Price Match SKU - UAE + KSA.csv	Product catalog	SKU,Name,Brand,Country,Currency,Cost,Selling price
uae-sku-urls.csv	UAE competitor URLs	Mumzworld AE,amazon UAE,First cry uae,Noon UAE
ksa-sku-url.csv	KSA competitor URLs	SKU,Name,Amazon SA,Firstcry SA,Nahdionline SA,Noon sa
mumzworld-sku-country-url.csv	Mumzworld URLs	SKU,Name,Country,mumzworld url
exceptions.csv	Price match exceptions	SKU,Name,Brand,Country,Exception
margin-caps.csv	Margin requirements	SKU,Name,Brand,Country,Margin Cap
price-up.csv	Price increase blocks	SKU,Name,Brand,Country,Price Up Exception

📈 Monitoring & Reports

Generated Reports (CSV)

• matched_prices.csv: Successful price matches with direction analysis
• blocked_by_exception.csv: SKUs excluded from matching
• margin_violations.csv: Failed margin cap validations
• price_up_blocks.csv: Blocked price increases
• summary.csv: High-level metrics and performance

Queue Monitoring

Access Horizon dashboard at: http://localhost/horizon

• Monitor job throughput
• View failed jobs
• Track queue metrics
• Manage supervisor processes

🚀 Scaling & Performance

Queue Scaling

Adjust worker counts in config/horizon.php:

'crawler-supervisor' => [
    'maxProcesses' => 40,  // Increase for more crawl throughput
],
'match-supervisor' => [
    'maxProcesses' => 10,  // Scale for price matching load
],

Geographic Expansion

To add new countries (e.g., Egypt):

1. Add competitors: mumzworld_egy, amazon_egy, etc.
2. Update seeder data with new country URLs
3. No code changes required - system auto-scales

Performance Optimization

• Redis coordination minimizes database locks
• Chunked processing prevents memory issues
• Lock-based deduplication prevents wasted work
• Horizon auto-balancing optimizes resource usage

---

Built with ❤️ @ Mumzworld