Big Data AcademyQuery · move · process · govern
All 25 chapters available

Stage 03 · NoSQL, Search & Application Data

NoSQL and Distributed Database Fundamentals

A vendor-neutral course on NoSQL and distributed databases covering data models, CAP/PACELC, clocks, replication, quorums, sharding, consensus, conflict resolution, storage engines, indexing, transactions, security, observability, disaster recovery, cloud operations, and principled technology selection.

Browse curriculum Back to all courses
25chapters
125lesson paths
Beginner → Advancedlearning level
Publishedcourse state
Coverage baselineVendor-neutral distributed-data principles with modern document, key-value, wide-column, graph, search, and cloud patterns

Course brief

Learn the tradeoffs underneath NoSQL products before learning product commands.

A vendor-neutral course on NoSQL and distributed databases covering data models, CAP/PACELC, clocks, replication, quorums, sharding, consensus, conflict resolution, storage engines, indexing, transactions, security, observability, disaster recovery, cloud operations, and principled technology selection.

This syllabus deliberately separates foundations, data modeling, query behavior, internals, reliability, security, performance, operations, and production design so advanced material is not compressed into generic catch-all chapters.

By the end

You will be able to

  • Explain availability, consistency, latency, durability, ordering, and partition tolerance without reducing distributed systems to slogans
  • Choose document, key-value, wide-column, graph, search, cache, or relational approaches from access patterns and invariants
  • Reason about replication, quorums, partitioning, clocks, consensus, conflicts, transactions, and failure modes
  • Evaluate storage engines, indexes, compaction, TTL, caching, CDC, security, backup, observability, and cost as one operational system
  • Design and defend a polyglot data architecture using measurable workload requirements, migration strategy, SLOs, and failure testing

Complete syllabus

25 chapters · 125 lessons.

Every lesson path was reserved in the planned syllabus and is now linked because its lesson HTML is available. The sequence moves from foundations through advanced implementation, architecture, operations, reliability, security, tuning, and a production capstone.

01

Chapter 1

Why NoSQL Exists: Workloads, Scale, Flexibility, and Polyglot Persistence

5 lessons
01
Relational Databases vs NoSQL: Different Optimization Targets, Not Competing ReligionsPublished lesson · Chapter01/Lesson1.html
Open 02
Horizontal Scale, Global Distribution, Flexible Schemas, High Write Rates, and Low-Latency AccessPublished lesson · Chapter01/Lesson2.html
Open 03
Operational vs Analytical Workloads and Why One Database Rarely Optimizes Every Access PatternPublished lesson · Chapter01/Lesson3.html
Open 04
Polyglot Persistence: Assigning Systems by Invariant, Query Shape, Latency, and OwnershipPublished lesson · Chapter01/Lesson4.html
Open 05
Build a Decision Notebook: Workload, Data Volume, Consistency, Availability, Recovery, and Cost RequirementsPublished lesson · Chapter01/Lesson5.html
Open
02

Chapter 2

Distributed Systems Foundations: Nodes, Networks, Failure, and State

5 lessons
01
Processes, Nodes, Clusters, Partitions, Replicas, Coordinators, and Client RoutingPublished lesson · Chapter02/Lesson1.html
Open 02
Partial Failure: Why a Distributed System Can Be Both Working and Broken at the Same TimePublished lesson · Chapter02/Lesson2.html
Open 03
Latency, Timeouts, Retries, Duplicate Work, Backpressure, and the Fallacy of a Reliable NetworkPublished lesson · Chapter02/Lesson3.html
Open 04
Failure Domains: Process, Host, Rack, Zone, Region, Provider, and Human OperationsPublished lesson · Chapter02/Lesson4.html
Open 05
Define SLOs for Reads, Writes, Freshness, Durability, Recovery, and AvailabilityPublished lesson · Chapter02/Lesson5.html
Open
03

Chapter 3

CAP, PACELC, and Consistency/Availability Tradeoffs

5 lessons
01
CAP Precisely: Network Partitions Force a Consistency-or-Availability Choice for Conflicting OperationsPublished lesson · Chapter03/Lesson1.html
Open 02
Why CAP Does Not Mean Pick Two During Normal OperationPublished lesson · Chapter03/Lesson2.html
Open 03
PACELC: Latency vs Consistency Tradeoffs Even When the Network Is HealthyPublished lesson · Chapter03/Lesson3.html
Open 04
Strong, Eventual, Bounded-Staleness, Session, and Tunable Consistency ModelsPublished lesson · Chapter03/Lesson4.html
Open 05
Map Business Invariants to the Weakest Consistency Model That Still Preserves CorrectnessPublished lesson · Chapter03/Lesson5.html
Open
04

Chapter 4

Time, Ordering, Logical Clocks, and Versioning

5 lessons
01
Why Wall Clocks Are Dangerous: Drift, Skew, NTP, and Cross-Node OrderingPublished lesson · Chapter04/Lesson1.html
Open 02
Lamport Clocks and Happens-Before ReasoningPublished lesson · Chapter04/Lesson2.html
Open 03
Vector Clocks/Version Vectors and Detecting Concurrent UpdatesPublished lesson · Chapter04/Lesson3.html
Open 04
Hybrid Logical Clocks and Practical Time-Aware OrderingPublished lesson · Chapter04/Lesson4.html
Open 05
Use Versions, ETags, Timestamps, and Idempotency Keys Without Pretending Time Is PerfectPublished lesson · Chapter04/Lesson5.html
Open
05

Chapter 5

Replication Patterns: Leaders, Multi-Leader, and Leaderless Systems

5 lessons
01
Single-Leader Replication: Logs, Followers, Failover, and Read ScalingPublished lesson · Chapter05/Lesson1.html
Open 02
Synchronous vs Asynchronous Replication: Latency, Durability, and Data-Loss WindowsPublished lesson · Chapter05/Lesson2.html
Open 03
Multi-Leader Replication: Geographic Writes, Conflict Risk, and Topology ComplexityPublished lesson · Chapter05/Lesson3.html
Open 04
Leaderless/Dynamo-Style Replication: Coordinators, Replica Sets, Sloppy Quorums, and Hinted HandoffPublished lesson · Chapter05/Lesson4.html
Open 05
Choose a Replication Pattern from Write Locality, Failure Tolerance, and Conflict SemanticsPublished lesson · Chapter05/Lesson5.html
Open
06

Chapter 6

Quorums, Consistency Levels, Read Repair, and Anti-Entropy

5 lessons
01
Replication Factor, Read Quorum, Write Quorum, and the Meaning of R + W > NPublished lesson · Chapter06/Lesson1.html
Open 02
Tunable Consistency: Per-Operation Choices and Their Failure/Latency ConsequencesPublished lesson · Chapter06/Lesson2.html
Open 03
Read Repair, Merkle Trees, Anti-Entropy, and Convergence After Missed WritesPublished lesson · Chapter06/Lesson3.html
Open 04
Quorum Edge Cases: Network Partitions, Stale Replicas, Failed Coordinators, and TimeoutsPublished lesson · Chapter06/Lesson4.html
Open 05
Measure Staleness and Correctness Instead of Treating Consistency Labels as GuaranteesPublished lesson · Chapter06/Lesson5.html
Open
07

Chapter 7

Partitioning and Sharding: Range, Hash, Directory, and Consistent Hashing

5 lessons
01
Why Partition Data: Capacity, Throughput, Parallelism, and Failure IsolationPublished lesson · Chapter07/Lesson1.html
Open 02
Range Partitioning vs Hash Partitioning: Locality, Skew, and Query RoutingPublished lesson · Chapter07/Lesson2.html
Open 03
Consistent Hashing, Token Rings, Virtual Nodes, and RebalancingPublished lesson · Chapter07/Lesson3.html
Open 04
Hot Partitions, Hot Keys, Monotonic Identifiers, Celebrity Data, and Write ConcentrationPublished lesson · Chapter07/Lesson4.html
Open 05
Resharding, Split/Merge Operations, Online Rebalancing, and Capacity HeadroomPublished lesson · Chapter07/Lesson5.html
Open
08

Chapter 8

Membership, Failure Detection, Gossip, and Cluster Coordination

5 lessons
01
Static Membership vs Dynamic Discovery and Service RegistrationPublished lesson · Chapter08/Lesson1.html
Open 02
Heartbeats, Phi Accrual and Other Failure Detectors: Suspicion Is Not ProofPublished lesson · Chapter08/Lesson2.html
Open 03
Gossip Protocols: Epidemic Membership and Metadata DisseminationPublished lesson · Chapter08/Lesson3.html
Open 04
Split Brain, Fencing, Leases, Epochs/Terms, and Avoiding Two Active OwnersPublished lesson · Chapter08/Lesson4.html
Open 05
Operational Coordination: Topology Changes, Rolling Maintenance, and Convergence WindowsPublished lesson · Chapter08/Lesson5.html
Open
09

Chapter 9

Storage Engines: WAL, B-Trees, LSM Trees, Memtables, SSTables, and Compaction

5 lessons
01
Write-Ahead Logs and Durability Before Mutating Indexed StatePublished lesson · Chapter09/Lesson1.html
Open 02
B-Tree/B+Tree Storage: Read Locality, In-Place Updates, and Page ManagementPublished lesson · Chapter09/Lesson2.html
Open 03
LSM Trees: Memtables, Immutable Sorted Files, Write Amplification, and Read AmplificationPublished lesson · Chapter09/Lesson3.html
Open 04
Compaction Strategies, Tombstones, Bloom Filters, Caches, and Space AmplificationPublished lesson · Chapter09/Lesson4.html
Open 05
Choose Storage Behavior by Write Rate, Read Pattern, SSD Characteristics, and RetentionPublished lesson · Chapter09/Lesson5.html
Open
10

Chapter 10

Key-Value Databases and Data-Structure Stores

5 lessons
01
Keys as the API: Lookup Semantics, Namespaces, Prefixes, and Atomic Single-Key OperationsPublished lesson · Chapter10/Lesson1.html
Open 02
Values as Bytes, Documents, or Data Structures: Modeling Beyond Simple StringsPublished lesson · Chapter10/Lesson2.html
Open 03
TTL, Expiration, Eviction, Counters, Conditional Writes, and Compare-and-SetPublished lesson · Chapter10/Lesson3.html
Open 04
Caching vs System-of-Record Usage: Durability, Recovery, and Source-of-Truth BoundariesPublished lesson · Chapter10/Lesson4.html
Open 05
Use Cases: Sessions, Caches, Feature State, Rate Limits, Idempotency, and CoordinationPublished lesson · Chapter10/Lesson5.html
Open
11

Chapter 11

Document Databases and Aggregate-Oriented Modeling

5 lessons
01
Documents, Nested Objects, Arrays, Dynamic Fields, and BSON/JSON-Like RepresentationsPublished lesson · Chapter11/Lesson1.html
Open 02
Embed vs Reference: Aggregate Boundaries, Duplication, Update Frequency, and Read LocalityPublished lesson · Chapter11/Lesson2.html
Open 03
Document Growth, Size Limits, Large Arrays, Bucketing, and Schema VersioningPublished lesson · Chapter11/Lesson3.html
Open 04
Querying Nested Structures, Multikey Indexing, Aggregation Pipelines, and Denormalized ReadsPublished lesson · Chapter11/Lesson4.html
Open 05
Model One-to-One, One-to-Many, Many-to-Many, and Event Data with Workload EvidencePublished lesson · Chapter11/Lesson5.html
Open
12

Chapter 12

Wide-Column Databases and Query-First Partition Design

5 lessons
01
Partitions, Rows, Clustering Columns, Column Families/Tables, and Sparse DataPublished lesson · Chapter12/Lesson1.html
Open 02
Partition Key Design: Distribution, Bounded Partitions, and Single-Partition QueriesPublished lesson · Chapter12/Lesson2.html
Open 03
Clustering Order, Time Bucketing, Denormalized Tables, and Query-First ModelingPublished lesson · Chapter12/Lesson3.html
Open 04
Tombstones, TTL, Compaction, Repair, and Why Deletes Can Be Operationally ExpensivePublished lesson · Chapter12/Lesson4.html
Open 05
Use Cases: Telemetry, Time-Ordered Events, High-Write Services, and Multi-Region WorkloadsPublished lesson · Chapter12/Lesson5.html
Open
13

Chapter 13

Graph Databases: Property Graphs, Relationships, Traversal, and Path Queries

5 lessons
01
Nodes, Relationships/Edges, Labels, Properties, and Graph Schema ChoicesPublished lesson · Chapter13/Lesson1.html
Open 02
Adjacency, Index-Free Traversal Concepts, and When Relationships Are First-Class DataPublished lesson · Chapter13/Lesson2.html
Open 03
Path, Neighborhood, Reachability, Pattern Matching, and Variable-Length TraversalPublished lesson · Chapter13/Lesson3.html
Open 04
Supernodes, Dense Graphs, Partitioning Challenges, and Distributed Graph TradeoffsPublished lesson · Chapter13/Lesson4.html
Open 05
Use Cases: Identity, Fraud, Recommendations, Networks, Knowledge Graphs, and Dependency AnalysisPublished lesson · Chapter13/Lesson5.html
Open
14

Chapter 14

Search, Secondary Indexes, Inverted Indexes, and Vector Retrieval

5 lessons
01
Primary Access Paths vs Secondary Indexes and the Cost of Maintaining Alternate ViewsPublished lesson · Chapter14/Lesson1.html
Open 02
Inverted Indexes, Tokens, Analyzers, Relevance, Filters, Facets, and AggregationsPublished lesson · Chapter14/Lesson2.html
Open 03
Local vs Global Secondary Indexes in Distributed DatabasesPublished lesson · Chapter14/Lesson3.html
Open 04
Vector Embeddings, Approximate Nearest Neighbor Search, HNSW, and Hybrid RetrievalPublished lesson · Chapter14/Lesson4.html
Open 05
Choose Database-Native Search vs Dedicated Search/Vector InfrastructurePublished lesson · Chapter14/Lesson5.html
Open
15

Chapter 15

Access-Pattern-Driven Modeling, Denormalization, and Materialized Views

5 lessons
01
Start from Queries, Commands, Invariants, Cardinalities, and Change RatesPublished lesson · Chapter15/Lesson1.html
Open 02
Denormalization as Precomputed Work: Duplicated Data, Write Fan-Out, and Repair StrategiesPublished lesson · Chapter15/Lesson2.html
Open 03
Aggregate Boundaries, Bounded Contexts, and Data Ownership Across ServicesPublished lesson · Chapter15/Lesson3.html
Open 04
Materialized Views, Projection Tables, CQRS Read Models, and Eventually Consistent Derived DataPublished lesson · Chapter15/Lesson4.html
Open 05
Review a Relational Schema and Redesign It for Document, Wide-Column, and Key-Value WorkloadsPublished lesson · Chapter15/Lesson5.html
Open
16

Chapter 16

Transactions Beyond One Record: Atomicity, 2PC, Sagas, and Compensation

5 lessons
01
Single-Key/Single-Document Atomicity and Why It Is So ValuablePublished lesson · Chapter16/Lesson1.html
Open 02
Optimistic Concurrency, Compare-and-Set, Lightweight Transactions, and Conditional MutationsPublished lesson · Chapter16/Lesson2.html
Open 03
Two-Phase Commit: Coordinator State, Blocking Failure Modes, and Recovery ComplexityPublished lesson · Chapter16/Lesson3.html
Open 04
Sagas: Orchestration vs Choreography, Compensating Actions, and Business-Level AtomicityPublished lesson · Chapter16/Lesson4.html
Open 05
Design Invariants to Minimize Distributed Transactions Without Pretending They Never MatterPublished lesson · Chapter16/Lesson5.html
Open
17

Chapter 17

Consensus: Raft, Paxos, Quorums, Terms, Logs, and Linearizable Decisions

5 lessons
01
What Consensus Solves: Agreement Despite Failures Under Specific AssumptionsPublished lesson · Chapter17/Lesson1.html
Open 02
Raft Mental Model: Leaders, Terms, Log Replication, Commit Index, and ElectionsPublished lesson · Chapter17/Lesson2.html
Open 03
Paxos Mental Model: Proposers, Acceptors, Ballots, Quorums, and Chosen ValuesPublished lesson · Chapter17/Lesson3.html
Open 04
Linearizability, Leader Leases, Fencing Tokens, and Metadata/Control-Plane ConsensusPublished lesson · Chapter17/Lesson4.html
Open 05
Distinguish Consensus for Cluster Metadata from Replication of Application DataPublished lesson · Chapter17/Lesson5.html
Open
18

Chapter 18

Conflict Detection and Resolution: LWW, Merge Logic, CRDTs, and Idempotency

5 lessons
01
Lost Updates, Concurrent Writes, Duplicate Events, and Retry-Induced ConflictsPublished lesson · Chapter18/Lesson1.html
Open 02
Last-Write-Wins: Simple Semantics, Clock Dependence, and Silent Data LossPublished lesson · Chapter18/Lesson2.html
Open 03
Application-Level Merge: Domain Rules, Versions, and Human ResolutionPublished lesson · Chapter18/Lesson3.html
Open 04
CRDT Foundations: Counters, Sets, Registers, Monotonic State, and ConvergencePublished lesson · Chapter18/Lesson4.html
Open 05
Idempotent Commands, Deduplication Windows, Request IDs, and Exactly-Once IllusionsPublished lesson · Chapter18/Lesson5.html
Open
19

Chapter 19

TTL, Caching, Eviction, Expiration, and Ephemeral Data

5 lessons
01
Expiration Semantics: Lazy vs Active Expiry and What TTL Means Under ReplicationPublished lesson · Chapter19/Lesson1.html
Open 02
Cache-Aside, Read-Through, Write-Through, Write-Behind, and Refresh-Ahead PatternsPublished lesson · Chapter19/Lesson2.html
Open 03
LRU, LFU, Random, TTL-Aware Eviction, and Memory PressurePublished lesson · Chapter19/Lesson3.html
Open 04
Cache Stampedes, Thundering Herds, Hot Keys, Negative Caching, and Jittered ExpiryPublished lesson · Chapter19/Lesson4.html
Open 05
Decide What May Disappear, What Must Survive, and How to Reconstruct Ephemeral StatePublished lesson · Chapter19/Lesson5.html
Open
20

Chapter 20

Events, CDC, Streams, and Data Synchronization

5 lessons
01
Change Data Capture from Logs vs Application-Published EventsPublished lesson · Chapter20/Lesson1.html
Open 02
Outbox Pattern, Dual-Write Failure, Idempotent Consumers, and Ordering KeysPublished lesson · Chapter20/Lesson2.html
Open 03
Event Streams, Consumer Groups, Replay, Retention, and BackpressurePublished lesson · Chapter20/Lesson3.html
Open 04
Build Search Indexes, Caches, Warehouses, and Read Models from Change StreamsPublished lesson · Chapter20/Lesson4.html
Open 05
Reconciliation Jobs and Anti-Corruption Layers for Long-Lived Data SynchronizationPublished lesson · Chapter20/Lesson5.html
Open
21

Chapter 21

Security, Identity, Encryption, Multi-Tenancy, and Data Governance

5 lessons
01
Authentication, Authorization, Roles, Service Identities, and Least PrivilegePublished lesson · Chapter21/Lesson1.html
Open 02
TLS in Transit, Encryption at Rest, Key Management, and Client-Side/Field-Level EncryptionPublished lesson · Chapter21/Lesson2.html
Open 03
Tenant Isolation by Key, Collection/Table, Database, Cluster, or AccountPublished lesson · Chapter21/Lesson3.html
Open 04
Auditing, Data Classification, Retention, Legal Holds, Deletion, and Backup GovernancePublished lesson · Chapter21/Lesson4.html
Open 05
Protect Administrative Interfaces, Cluster Metadata, Network Paths, and SecretsPublished lesson · Chapter21/Lesson5.html
Open
22

Chapter 22

Backup, Restore, Repair, Disaster Recovery, and Durability Engineering

5 lessons
01
Logical vs Physical Backups, Snapshots, Continuous Backups, and Change-Log ArchivingPublished lesson · Chapter22/Lesson1.html
Open 02
Replication Is Not Backup: Operator Error, Corruption, Bad Deployments, and RansomwarePublished lesson · Chapter22/Lesson2.html
Open 03
RPO/RTO, Regional Failure, Quorum Loss, Restore Ordering, and Dependency RecoveryPublished lesson · Chapter22/Lesson3.html
Open 04
Test Restores, Integrity Checks, Reconciliation, and Repair After Partial RecoveryPublished lesson · Chapter22/Lesson4.html
Open 05
Write a Disaster-Recovery Runbook and Prove It with Game-Day ExercisesPublished lesson · Chapter22/Lesson5.html
Open
23

Chapter 23

Observability, Capacity, Benchmarking, and Performance Engineering

5 lessons
01
Golden Signals for Databases: Latency, Throughput, Errors, Saturation, and FreshnessPublished lesson · Chapter23/Lesson1.html
Open 02
Per-Partition/Shard Metrics, Replication Lag, Queue Depth, Compaction, Cache, and Storage SignalsPublished lesson · Chapter23/Lesson2.html
Open 03
Benchmark Workload Shapes, Not Marketing Numbers: Keys, Values, Zipf Skew, Concurrency, and DurabilityPublished lesson · Chapter23/Lesson3.html
Open 04
Tail Latency, Coordinated Omission, Warmup, GC, Network, Disk, and Client-Pool EffectsPublished lesson · Chapter23/Lesson4.html
Open 05
Capacity Models for Data Size, Replication, Indexes, Headroom, Growth, and Failure ScenariosPublished lesson · Chapter23/Lesson5.html
Open
24

Chapter 24

Cloud and Managed NoSQL: Autoscaling, Serverless, Global Tables, and Cost

5 lessons
01
Managed Control Planes vs Self-Managed Data Planes: Responsibility BoundariesPublished lesson · Chapter24/Lesson1.html
Open 02
Provisioned Throughput, Autoscaling, Serverless Billing, Storage, Egress, and Request PricingPublished lesson · Chapter24/Lesson2.html
Open 03
Global Replication, Multi-Region Writes, Conflict Semantics, and Regulatory Data ResidencyPublished lesson · Chapter24/Lesson3.html
Open 04
Backups, Upgrades, Observability, Private Networking, and Vendor-Specific Failure ModesPublished lesson · Chapter24/Lesson4.html
Open 05
Model Total Cost of Ownership Including Operations, Lock-In, Migration, and Developer TimePublished lesson · Chapter24/Lesson5.html
Open
25

Chapter 25

Technology Selection, Migration, Polyglot Architecture, and Capstone

5 lessons
01
Build a Database Selection Matrix from Queries, Invariants, Latency, Scale, Operations, and SkillsPublished lesson · Chapter25/Lesson1.html
Open 02
Avoid Cargo-Cult NoSQL: Recognize Workloads Better Served by Relational DatabasesPublished lesson · Chapter25/Lesson2.html
Open 03
Migrate Safely with Dual Reads/Writes, Backfills, CDC, Verification, Cutover, and RollbackPublished lesson · Chapter25/Lesson3.html
Open 04
Design a Polyglot Architecture with Explicit Ownership, Synchronization, SLOs, and Failure BoundariesPublished lesson · Chapter25/Lesson4.html
Open 05
Capstone: Defend a Distributed Data Architecture Under Partitions, Growth, Incidents, and Cost ConstraintsPublished lesson · Chapter25/Lesson5.html
Open