Software engineering in a nutshell

software_engineering.md

• NodeJS and JavaScript
  • Event loop
  • Parallel programming
  • Async
  • Libuv and the thread pool
  • Streams and backpressure
  • Worker threads vs child processes
  • Memory model (V8 heap, garbage collection, memory leaks)
  • CommonJS vs ESM modules
• Clustering and process management (PM2, cluster module)
  • Error handling patterns (uncaughtException, unhandledRejection)
• Python
  • Async operations
  • How to multithread
  • How to parallelize async operations
  • Latest version
  • Typing
  • GIL (Global Interpreter Lock) and its implications
  • asyncio event loop internals
  • multiprocessing vs threading vs concurrent.futures
  • Context variables (contextvars module)
  • Decorators and metaclasses
  • Generator and coroutine internals (yield, send, throw)
  • Memory management (reference counting, cyclic GC)
  • Virtual environments and dependency management (pip, poetry, uv)
• JVM Languages (Java, Kotlin, Scala)
  • JVM memory model (heap, stack, metaspace)
  • Garbage collection strategies (G1, ZGC, Shenandoah)
  • Concurrency primitives (synchronized, volatile, Locks, Atomics)
  • Virtual threads (Project Loom)
  • Kotlin coroutines and structured concurrency
  • Scala futures and Akka actors
  • Spring Boot lifecycle and dependency injection
  • Micronaut compile-time DI vs Spring runtime DI
• Go
  • Goroutines and channels
  • Select statement and concurrency patterns
  • Context propagation and cancellation
  • Memory model and garbage collection
  • Interface-based polymorphism
• Distributed Systems
  • Event streams like Kinesis and other products in this vertical
  • Idempotency
  • CAP Theorem
  • Caching
  • Kubernetes
  • Consensus protocols (Raft, Paxos)
  • Consistency models (strong, eventual, causal, linearizable)
  • Distributed transactions (2PC, 3PC, saga pattern)
  • Vector clocks and logical timestamps (Lamport clocks)
  • Conflict resolution (last-writer-wins, CRDTs, merge functions)
  • Service discovery (DNS-based, consul, etcd)
  • Circuit breakers and bulkheads (resilience patterns)
  • Backpressure and flow control
  • Leader election
  • Gossip protocols
  • Sharding strategies (hash-based, range-based, consistent hashing)
  • Replication (single-leader, multi-leader, leaderless)
  • Split-brain and network partition handling
  • Exactly-once delivery semantics
  • Distributed locking (Redlock, ZooKeeper)
• Message Queues and Event Streaming
  • Kafka (partitions, consumer groups, offsets, log compaction)
  • RabbitMQ (exchanges, queues, bindings, dead-letter queues)
  • SQS and SNS (standard vs FIFO, visibility timeout, fan-out)
  • EventBridge (content-based routing, schema registry)
  • Pub/sub vs point-to-point vs event sourcing
  • At-least-once vs at-most-once vs exactly-once
  • Ordering guarantees and partitioning
  • Backpressure and consumer lag
• SQL Databases
  • ACID properties
  • Transaction isolation levels (read uncommitted, read committed, repeatable read, serializable)
  • MVCC (Multi-Version Concurrency Control)
  • Indexing (B-tree, B+ tree, hash index, GIN, GiST)
  • Composite indexes and index ordering
  • Query execution plans (EXPLAIN, ANALYZE)
  • Joins (nested loop, hash join, merge join)
  • Normalization vs denormalization tradeoffs
  • Partitioning (range, list, hash)
  • Connection pooling (PgBouncer, HikariCP)
  • Stored procedures and triggers (pros and cons)
  • Window functions and CTEs
  • PostgreSQL-specific (JSONB, advisory locks, LISTEN/NOTIFY, logical replication)
  • MySQL-specific (InnoDB vs MyISAM, replication modes)
  • Schema migration strategies (Flyway, Liquibase, zero-downtime migrations)
  • Read replicas and write-ahead log (WAL)
  • Vacuum and autovacuum (PostgreSQL)
• NoSQL Databases
  • DynamoDB (partition keys, sort keys, GSI, LSI, single-table design)
  • DynamoDB capacity modes (on-demand vs provisioned, burst capacity)
  • DynamoDB Streams and change data capture
  • DynamoDB transactions and conditional writes
  • MongoDB (document model, aggregation pipeline, sharding, WiredTiger)
  • Redis (data structures, persistence modes, pub/sub, Lua scripting)
  • Redis Cluster (hash slots, resharding, failover)
  • Cassandra (partition key, clustering key, gossip protocol, tunable consistency)
  • ElasticSearch (inverted index, shards, mapping, analyzers)
  • When to use SQL vs NoSQL (access patterns, data model, scale requirements)
  • Data modeling for NoSQL (denormalization, adjacency lists, materialized views)
• API Design
  • REST (resource modeling, HATEOAS, idempotency, pagination)
  • GraphQL (schema design, resolvers, N+1 problem, DataLoader, subscriptions)
  • gRPC (protobuf, streaming, load balancing, deadlines)
  • API versioning strategies (URL path, header, content negotiation)
  • Rate limiting (token bucket, sliding window, leaky bucket)
  • Authentication (OAuth 2.0, JWT, API keys, mTLS)
  • Authorization (RBAC, ABAC, policy engines)
  • OpenAPI / Swagger specification
  • Webhooks (delivery guarantees, retry policies, signature verification)
  • Backward compatibility and breaking changes
  • Pagination patterns (offset, cursor-based, keyset)
• Operating Systems
  • Unix principles
  • Docker and containerization
  • Process vs thread vs coroutine
  • Virtual memory, paging, and swap
  • File descriptors and I/O multiplexing (epoll, kqueue, io_uring)
  • Signals and inter-process communication (pipes, shared memory, sockets)
  • Linux namespaces and cgroups (container foundations)
  • systemd and process supervision
  • File systems (ext4, XFS, copy-on-write)
  • CPU scheduling (CFS, nice values, CPU affinity)
  • Memory-mapped files (mmap)
  • OOM killer and memory pressure
• Networking
  • Load balancers
  • Proxy servers
  • TCP/IP
  • Websockets
  • Protobuf
  • RPC
  • HTTP
  • REST
  • DNS
    • how subdomains work
  • SSL & HTTPS
  • HTTP/2 and HTTP/3 (multiplexing, QUIC, header compression)
  • TCP vs UDP (when to use each)
  • Connection pooling and keep-alive
  • TLS handshake and certificate chain
  • mTLS (mutual TLS) for service-to-service auth
  • CDN and edge computing
  • NAT, CIDR, and VPC networking
  • Service mesh (Istio, Envoy, sidecar pattern)
  • gRPC load balancing (L4 vs L7)
  • Server-Sent Events (SSE) vs WebSockets vs long polling
• Caching
  • Cache invalidation strategies (TTL, write-through, write-behind, cache-aside)
  • Cache eviction policies (LRU, LFU, FIFO)
  • Cache stampede / thundering herd prevention
  • Multi-level caching (L1 in-process, L2 distributed)
  • CDN caching (Cache-Control headers, Vary, ETags)
  • Application-level caching (Redis, Memcached)
  • Database query cache
  • Bloom filters for cache miss reduction
• Security
  • OWASP Top 10 (SQL injection, XSS, CSRF, SSRF)
  • Authentication flows (OAuth 2.0, OIDC, SAML)
  • JWT (structure, signing algorithms, refresh tokens, token rotation)
  • Password hashing (bcrypt, scrypt, argon2)
  • Secrets management (Vault, AWS Secrets Manager, GCP Secret Manager)
  • Encryption at rest and in transit
  • Certificate management and rotation
  • API security (rate limiting, input validation, CORS)
  • SQL injection and parameterized queries
  • Principle of least privilege
  • Security headers (CSP, HSTS, X-Frame-Options)
• Observability
  • Three pillars (metrics, logs, traces)
  • Structured logging
  • Distributed tracing (OpenTelemetry, W3C Trace Context)
  • Metrics types (counters, gauges, histograms)
  • Alerting strategies (SLO-based, anomaly detection, threshold)
  • Log aggregation (ELK, CloudWatch, Datadog)
  • APM (Application Performance Monitoring)
  • Health checks (liveness, readiness, startup probes)
  • Error tracking (Sentry, Bugsnag)
  • Runbooks and playbooks for incident response
  • SLI / SLO / SLA definitions and tracking
• Cloud & Infrastructure
  • AWS (Lambda, Step Functions, DynamoDB, S3, SQS, SNS, EventBridge, ECS, EKS)
  • GCP (Cloud Run, GKE, Cloud SQL, Pub/Sub, Cloud Functions, Firestore)
  • Infrastructure as Code (Terraform, Pulumi, CloudFormation, CDK)
  • CI/CD (GitHub Actions, CircleCI, GitLab CI, deployment strategies)
  • Container orchestration (Kubernetes pods, services, deployments, StatefulSets)
  • Kubernetes networking (Ingress, Service types, NetworkPolicy)
  • Helm charts and Kustomize
  • Serverless patterns (Lambda, API Gateway, event-driven)
  • Serverless tradeoffs (cold starts, timeout limits, vendor lock-in)
  • Auto-scaling (HPA, VPA, Karpenter, Lambda concurrency)
  • Cost optimization (reserved instances, spot, right-sizing, savings plans)
  • Multi-region and disaster recovery (RPO, RTO)
• Data Structures and Algorithms
  • Hash tables (collision resolution, load factor, resize strategy)
  • Trees (BST, AVL, Red-Black, B-tree, B+ tree, trie)
  • Graphs (BFS, DFS, Dijkstra, topological sort, union-find)
  • Heaps and priority queues
  • Sorting (quicksort, mergesort, radix sort, timsort)
  • Dynamic programming (memoization vs tabulation, state reduction)
  • Sliding window and two-pointer techniques
  • Binary search variations (on answer, rotated arrays, first/last occurrence)
  • String algorithms (KMP, Rabin-Karp, suffix arrays)
  • Probabilistic data structures (Bloom filter, HyperLogLog, Count-Min Sketch)
  • Consistent hashing
  • Skip lists
• Concurrency and Parallelism
  • Thread safety (mutexes, semaphores, read-write locks)
  • Lock-free data structures (CAS operations, atomic references)
  • Deadlock, livelock, and starvation
  • Producer-consumer pattern
  • Actor model (Akka, Erlang/OTP)
  • CSP (Communicating Sequential Processes, Go channels)
  • Async/await vs threads vs green threads
  • Thread pools and work stealing
  • Race conditions and data races
  • Memory ordering and happens-before relationships
  • Futures and promises
• Software Design
  • SOLID principles
  • Design patterns (factory, strategy, observer, decorator, adapter, proxy)
  • Domain-Driven Design (bounded contexts, aggregates, entities, value objects)
  • Clean Architecture / Hexagonal Architecture
  • Event sourcing and CQRS
  • Microservices patterns (API gateway, saga, strangler fig, sidecar)
  • Monolith vs microservices vs modular monolith tradeoffs
  • Dependency injection
  • Repository pattern and unit of work
  • Feature flags and trunk-based development
  • Error handling philosophies (exceptions vs result types vs error codes)
• Testing
  • Unit testing (isolation, mocking, stubbing, faking)
  • Integration testing (test containers, in-memory databases)
  • End-to-end testing (contract testing, Pact)
  • Property-based testing (Hypothesis, QuickCheck)
  • Load testing (k6, Locust, Artillery)
  • Chaos engineering (Chaos Monkey, Litmus)
  • Test pyramid vs test trophy
  • TDD and BDD
  • Mutation testing
  • Testing in serverless environments
• Performance Engineering
  • Profiling (CPU, memory, I/O)
  • Benchmarking methodology
  • Latency vs throughput
  • P50, P95, P99 percentiles
  • Connection pooling
  • Database query optimization
  • N+1 query problem
  • Batching and debouncing
  • Compression (gzip, brotli, zstd)
  • Lazy loading and pagination
  • Object pooling and memory allocation