Curriculum planned

Stage 06 · Hadoop Ecosystem

Apache Pig

A complete Apache Pig course for understanding, maintaining, testing, tuning, and modernizing Pig Latin data pipelines. It covers Pig 0.18.0, local and distributed execution, Pig Latin types/operators, nested data, joins, grouping, UDFs, macros, parameters, loaders/storers, Avro/ORC/HCatalog/HBase integrations, MapReduce/Tez/Spark execution, diagnostics, security, performance, testing, migration to modern engines, and legacy-production operations.

Browse planned curriculum Back to all courses

26planned chapters

130reserved lesson paths

Legacy → Maintenance / Modernizationlearning level

Plannedcourse state

Coverage baselineApache Pig 0.18.0, released in 2025 with Hadoop 3, Tez 0.10, Hive 3, Spark 3, HBase 2, and Python 3 support; taught explicitly as a legacy/maintenance and modernization technology rather than a default choice for new analytics projects

Course brief

Read and reason about Pig Latin dataflows well enough to safely maintain legacy Hadoop jobs, then test and migrate them toward contemporary SQL/DataFrame pipelines with semantic equivalence and measured performance.

This syllabus deliberately separates foundations, data/model semantics, internals, reliability, security, performance, operations, and production design so advanced material is not compressed into generic catch-all chapters.

By the end

You will be able to

Read, write, debug, and test Pig Latin scripts using relations, schemas, nested data, joins, groups, functions, macros, and parameters
Understand how Pig compiles logical dataflows to MapReduce, Tez, or Spark execution and how optimizer choices affect cost
Integrate Pig with HDFS, HCatalog/Hive, Avro, ORC, HBase, custom loaders/storers, Java/Python UDFs, and shell streaming
Operate and troubleshoot secured legacy pipelines with counters, explain plans, skew handling, memory controls, and repeatable tests
Migrate Pig workloads to Spark, Hive, SQL, or DataFrame systems using golden outputs, edge-case tests, performance baselines, and rollback

Complete planned syllabus

26 chapters · 130 lesson paths.

Every lesson path is reserved now but intentionally not linked until its lesson HTML is actually published. The sequence moves from foundations through advanced implementation, architecture, operations, reliability, security, tuning, and a production capstone.

Chapter 1

Apache Pig in 2026: Historical Role, 0.18.0 Revival, Legacy Status, Workload Fit, and Modernization Strategy

5 lessons

Apache Pig in 2026: Historical Role, 0.18.0 Revival, Legacy Status, Workload Fit, and Modernization Strategy: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter01/Lesson1.html

Planned02

Apache Pig in 2026: Historical Role, 0.18.0 Revival, Legacy Status, Workload Fit, and Modernization Strategy: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter01/Lesson2.html

Planned03

Apache Pig in 2026: Historical Role, 0.18.0 Revival, Legacy Status, Workload Fit, and Modernization Strategy: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter01/Lesson3.html

Planned04

Apache Pig in 2026: Historical Role, 0.18.0 Revival, Legacy Status, Workload Fit, and Modernization Strategy: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter01/Lesson4.html

Planned05

Checkpoint Lab — Apache Pig in 2026: Historical Role, 0.18.0 Revival, Legacy Status, Workload Fit, and Modernization Strategy: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter01/Lesson5.html

Planned

Chapter 2

Installation and Execution Modes: Local, Hadoop MapReduce, Tez, Spark, Environment Variables, and Reproducible Lab Setup

5 lessons

Installation and Execution Modes: Local, Hadoop MapReduce, Tez, Spark, Environment Variables, and Reproducible Lab Setup: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter02/Lesson1.html

Planned02

Installation and Execution Modes: Local, Hadoop MapReduce, Tez, Spark, Environment Variables, and Reproducible Lab Setup: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter02/Lesson2.html

Planned03

Installation and Execution Modes: Local, Hadoop MapReduce, Tez, Spark, Environment Variables, and Reproducible Lab Setup: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter02/Lesson3.html

Planned04

Installation and Execution Modes: Local, Hadoop MapReduce, Tez, Spark, Environment Variables, and Reproducible Lab Setup: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter02/Lesson4.html

Planned05

Checkpoint Lab — Installation and Execution Modes: Local, Hadoop MapReduce, Tez, Spark, Environment Variables, and Reproducible Lab Setup: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter02/Lesson5.html

Planned

Chapter 3

Pig Latin Mental Model: Relations, Statements, Aliases, Lazy Dataflow Evaluation, DAGs, and Schema-on-Read

5 lessons

Pig Latin Mental Model: Relations, Statements, Aliases, Lazy Dataflow Evaluation, DAGs, and Schema-on-Read: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter03/Lesson1.html

Planned02

Pig Latin Mental Model: Relations, Statements, Aliases, Lazy Dataflow Evaluation, DAGs, and Schema-on-Read: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter03/Lesson2.html

Planned03

Pig Latin Mental Model: Relations, Statements, Aliases, Lazy Dataflow Evaluation, DAGs, and Schema-on-Read: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter03/Lesson3.html

Planned04

Pig Latin Mental Model: Relations, Statements, Aliases, Lazy Dataflow Evaluation, DAGs, and Schema-on-Read: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter03/Lesson4.html

Planned05

Checkpoint Lab — Pig Latin Mental Model: Relations, Statements, Aliases, Lazy Dataflow Evaluation, DAGs, and Schema-on-Read: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter03/Lesson5.html

Planned

Chapter 4

Pig Data Types: int/long/float/double/chararray/bytearray/boolean/datetime, Tuple, Bag, Map, Nulls, and Casting

5 lessons

Pig Data Types: int/long/float/double/chararray/bytearray/boolean/datetime, Tuple, Bag, Map, Nulls, and Casting: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter04/Lesson1.html

Planned02

Pig Data Types: int/long/float/double/chararray/bytearray/boolean/datetime, Tuple, Bag, Map, Nulls, and Casting: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter04/Lesson2.html

Planned03

Pig Data Types: int/long/float/double/chararray/bytearray/boolean/datetime, Tuple, Bag, Map, Nulls, and Casting: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter04/Lesson3.html

Planned04

Pig Data Types: int/long/float/double/chararray/bytearray/boolean/datetime, Tuple, Bag, Map, Nulls, and Casting: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter04/Lesson4.html

Planned05

Checkpoint Lab — Pig Data Types: int/long/float/double/chararray/bytearray/boolean/datetime, Tuple, Bag, Map, Nulls, and Casting: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter04/Lesson5.html

Planned

Chapter 5

LOAD and STORE: PigStorage, Delimiters, Schemas, HDFS Paths, Compression, Loaders, Storers, and Data Quality

5 lessons

LOAD and STORE: PigStorage, Delimiters, Schemas, HDFS Paths, Compression, Loaders, Storers, and Data Quality: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter05/Lesson1.html

Planned02

LOAD and STORE: PigStorage, Delimiters, Schemas, HDFS Paths, Compression, Loaders, Storers, and Data Quality: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter05/Lesson2.html

Planned03

LOAD and STORE: PigStorage, Delimiters, Schemas, HDFS Paths, Compression, Loaders, Storers, and Data Quality: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter05/Lesson3.html

Planned04

LOAD and STORE: PigStorage, Delimiters, Schemas, HDFS Paths, Compression, Loaders, Storers, and Data Quality: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter05/Lesson4.html

Planned05

Checkpoint Lab — LOAD and STORE: PigStorage, Delimiters, Schemas, HDFS Paths, Compression, Loaders, Storers, and Data Quality: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter05/Lesson5.html

Planned

Chapter 6

FOREACH and GENERATE: Projection, Expressions, FLATTEN, Nested Blocks, Dereferencing, and Complex Transformations

5 lessons

FOREACH and GENERATE: Projection, Expressions, FLATTEN, Nested Blocks, Dereferencing, and Complex Transformations: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter06/Lesson1.html

Planned02

FOREACH and GENERATE: Projection, Expressions, FLATTEN, Nested Blocks, Dereferencing, and Complex Transformations: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter06/Lesson2.html

Planned03

FOREACH and GENERATE: Projection, Expressions, FLATTEN, Nested Blocks, Dereferencing, and Complex Transformations: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter06/Lesson3.html

Planned04

FOREACH and GENERATE: Projection, Expressions, FLATTEN, Nested Blocks, Dereferencing, and Complex Transformations: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter06/Lesson4.html

Planned05

Checkpoint Lab — FOREACH and GENERATE: Projection, Expressions, FLATTEN, Nested Blocks, Dereferencing, and Complex Transformations: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter06/Lesson5.html

Planned

Chapter 7

FILTER, DISTINCT, SAMPLE, SPLIT, LIMIT, ORDER, RANK, and Deterministic Data-Reduction Patterns

5 lessons

FILTER, DISTINCT, SAMPLE, SPLIT, LIMIT, ORDER, RANK, and Deterministic Data-Reduction Patterns: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter07/Lesson1.html

Planned02

FILTER, DISTINCT, SAMPLE, SPLIT, LIMIT, ORDER, RANK, and Deterministic Data-Reduction Patterns: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter07/Lesson2.html

Planned03

FILTER, DISTINCT, SAMPLE, SPLIT, LIMIT, ORDER, RANK, and Deterministic Data-Reduction Patterns: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter07/Lesson3.html

Planned04

FILTER, DISTINCT, SAMPLE, SPLIT, LIMIT, ORDER, RANK, and Deterministic Data-Reduction Patterns: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter07/Lesson4.html

Planned05

Checkpoint Lab — FILTER, DISTINCT, SAMPLE, SPLIT, LIMIT, ORDER, RANK, and Deterministic Data-Reduction Patterns: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter07/Lesson5.html

Planned

Chapter 8

GROUP, COGROUP, Aggregate Functions, Nested FOREACH, Algebraic Aggregation, and Memory Implications

5 lessons

GROUP, COGROUP, Aggregate Functions, Nested FOREACH, Algebraic Aggregation, and Memory Implications: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter08/Lesson1.html

Planned02

GROUP, COGROUP, Aggregate Functions, Nested FOREACH, Algebraic Aggregation, and Memory Implications: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter08/Lesson2.html

Planned03

GROUP, COGROUP, Aggregate Functions, Nested FOREACH, Algebraic Aggregation, and Memory Implications: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter08/Lesson3.html

Planned04

GROUP, COGROUP, Aggregate Functions, Nested FOREACH, Algebraic Aggregation, and Memory Implications: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter08/Lesson4.html

Planned05

Checkpoint Lab — GROUP, COGROUP, Aggregate Functions, Nested FOREACH, Algebraic Aggregation, and Memory Implications: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter08/Lesson5.html

Planned

Chapter 9

JOIN Strategies: Hash/Replicated, Skewed, Merge, Outer Joins, Multiway Joins, and Data-Skew Tradeoffs

5 lessons

JOIN Strategies: Hash/Replicated, Skewed, Merge, Outer Joins, Multiway Joins, and Data-Skew Tradeoffs: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter09/Lesson1.html

Planned02

JOIN Strategies: Hash/Replicated, Skewed, Merge, Outer Joins, Multiway Joins, and Data-Skew Tradeoffs: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter09/Lesson2.html

Planned03

JOIN Strategies: Hash/Replicated, Skewed, Merge, Outer Joins, Multiway Joins, and Data-Skew Tradeoffs: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter09/Lesson3.html

Planned04

JOIN Strategies: Hash/Replicated, Skewed, Merge, Outer Joins, Multiway Joins, and Data-Skew Tradeoffs: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter09/Lesson4.html

Planned05

Checkpoint Lab — JOIN Strategies: Hash/Replicated, Skewed, Merge, Outer Joins, Multiway Joins, and Data-Skew Tradeoffs: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter09/Lesson5.html

Planned

Chapter 10

CROSS, UNION, CUBE, ROLLUP, Stream Processing Patterns, and Relational-Algebra Composition

5 lessons

CROSS, UNION, CUBE, ROLLUP, Stream Processing Patterns, and Relational-Algebra Composition: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter10/Lesson1.html

Planned02

CROSS, UNION, CUBE, ROLLUP, Stream Processing Patterns, and Relational-Algebra Composition: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter10/Lesson2.html

Planned03

CROSS, UNION, CUBE, ROLLUP, Stream Processing Patterns, and Relational-Algebra Composition: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter10/Lesson3.html

Planned04

CROSS, UNION, CUBE, ROLLUP, Stream Processing Patterns, and Relational-Algebra Composition: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter10/Lesson4.html

Planned05

Checkpoint Lab — CROSS, UNION, CUBE, ROLLUP, Stream Processing Patterns, and Relational-Algebra Composition: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter10/Lesson5.html

Planned

Chapter 11

Built-In Functions: Evaluation, Math, String, Date/Time, Bag/Tuple/Map, Load/Store, and Statistical Functions

5 lessons

Built-In Functions: Evaluation, Math, String, Date/Time, Bag/Tuple/Map, Load/Store, and Statistical Functions: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter11/Lesson1.html

Planned02

Built-In Functions: Evaluation, Math, String, Date/Time, Bag/Tuple/Map, Load/Store, and Statistical Functions: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter11/Lesson2.html

Planned03

Built-In Functions: Evaluation, Math, String, Date/Time, Bag/Tuple/Map, Load/Store, and Statistical Functions: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter11/Lesson3.html

Planned04

Built-In Functions: Evaluation, Math, String, Date/Time, Bag/Tuple/Map, Load/Store, and Statistical Functions: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter11/Lesson4.html

Planned05

Checkpoint Lab — Built-In Functions: Evaluation, Math, String, Date/Time, Bag/Tuple/Map, Load/Store, and Statistical Functions: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter11/Lesson5.html

Planned

Chapter 12

User-Defined Functions: Java EvalFunc, Algebraic/Accumulator Interfaces, Type Metadata, Packaging, and Reuse

5 lessons

User-Defined Functions: Java EvalFunc, Algebraic/Accumulator Interfaces, Type Metadata, Packaging, and Reuse: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter12/Lesson1.html

Planned02

User-Defined Functions: Java EvalFunc, Algebraic/Accumulator Interfaces, Type Metadata, Packaging, and Reuse: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter12/Lesson2.html

Planned03

User-Defined Functions: Java EvalFunc, Algebraic/Accumulator Interfaces, Type Metadata, Packaging, and Reuse: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter12/Lesson3.html

Planned04

User-Defined Functions: Java EvalFunc, Algebraic/Accumulator Interfaces, Type Metadata, Packaging, and Reuse: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter12/Lesson4.html

Planned05

Checkpoint Lab — User-Defined Functions: Java EvalFunc, Algebraic/Accumulator Interfaces, Type Metadata, Packaging, and Reuse: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter12/Lesson5.html

Planned

Chapter 13

Python 3 and Scripting UDFs, Streaming Commands, External Processes, Serialization Boundaries, and Operational Risk

5 lessons

Python 3 and Scripting UDFs, Streaming Commands, External Processes, Serialization Boundaries, and Operational Risk: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter13/Lesson1.html

Planned02

Python 3 and Scripting UDFs, Streaming Commands, External Processes, Serialization Boundaries, and Operational Risk: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter13/Lesson2.html

Planned03

Python 3 and Scripting UDFs, Streaming Commands, External Processes, Serialization Boundaries, and Operational Risk: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter13/Lesson3.html

Planned04

Python 3 and Scripting UDFs, Streaming Commands, External Processes, Serialization Boundaries, and Operational Risk: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter13/Lesson4.html

Planned05

Checkpoint Lab — Python 3 and Scripting UDFs, Streaming Commands, External Processes, Serialization Boundaries, and Operational Risk: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter13/Lesson5.html

Planned

Chapter 14

Macros, IMPORT, Parameters, Parameter Substitution, Includes, Reusable Script Structure, and Environment Promotion

5 lessons

Macros, IMPORT, Parameters, Parameter Substitution, Includes, Reusable Script Structure, and Environment Promotion: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter14/Lesson1.html

Planned02

Macros, IMPORT, Parameters, Parameter Substitution, Includes, Reusable Script Structure, and Environment Promotion: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter14/Lesson2.html

Planned03

Macros, IMPORT, Parameters, Parameter Substitution, Includes, Reusable Script Structure, and Environment Promotion: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter14/Lesson3.html

Planned04

Macros, IMPORT, Parameters, Parameter Substitution, Includes, Reusable Script Structure, and Environment Promotion: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter14/Lesson4.html

Planned05

Checkpoint Lab — Macros, IMPORT, Parameters, Parameter Substitution, Includes, Reusable Script Structure, and Environment Promotion: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter14/Lesson5.html

Planned

Chapter 15

AvroStorage, OrcStorage, HCatalog/Hive Integration, Schema Interoperability, and Partition-Aware Reads/Writes

5 lessons

AvroStorage, OrcStorage, HCatalog/Hive Integration, Schema Interoperability, and Partition-Aware Reads/Writes: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter15/Lesson1.html

Planned02

AvroStorage, OrcStorage, HCatalog/Hive Integration, Schema Interoperability, and Partition-Aware Reads/Writes: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter15/Lesson2.html

Planned03

AvroStorage, OrcStorage, HCatalog/Hive Integration, Schema Interoperability, and Partition-Aware Reads/Writes: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter15/Lesson3.html

Planned04

AvroStorage, OrcStorage, HCatalog/Hive Integration, Schema Interoperability, and Partition-Aware Reads/Writes: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter15/Lesson4.html

Planned05

Checkpoint Lab — AvroStorage, OrcStorage, HCatalog/Hive Integration, Schema Interoperability, and Partition-Aware Reads/Writes: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter15/Lesson5.html

Planned

Chapter 16

HBase Integration: HBaseStorage, Row Keys, Column Mapping, Scan Ranges, Filters, and Batch Data Movement

5 lessons

HBase Integration: HBaseStorage, Row Keys, Column Mapping, Scan Ranges, Filters, and Batch Data Movement: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter16/Lesson1.html

Planned02

HBase Integration: HBaseStorage, Row Keys, Column Mapping, Scan Ranges, Filters, and Batch Data Movement: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter16/Lesson2.html

Planned03

HBase Integration: HBaseStorage, Row Keys, Column Mapping, Scan Ranges, Filters, and Batch Data Movement: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter16/Lesson3.html

Planned04

HBase Integration: HBaseStorage, Row Keys, Column Mapping, Scan Ranges, Filters, and Batch Data Movement: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter16/Lesson4.html

Planned05

Checkpoint Lab — HBase Integration: HBaseStorage, Row Keys, Column Mapping, Scan Ranges, Filters, and Batch Data Movement: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter16/Lesson5.html

Planned

Chapter 17

Logical and Physical Plans: EXPLAIN, ILLUSTRATE, DESCRIBE, DUMP, Optimizer Rules, and Plan Inspection

5 lessons

Logical and Physical Plans: EXPLAIN, ILLUSTRATE, DESCRIBE, DUMP, Optimizer Rules, and Plan Inspection: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter17/Lesson1.html

Planned02

Logical and Physical Plans: EXPLAIN, ILLUSTRATE, DESCRIBE, DUMP, Optimizer Rules, and Plan Inspection: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter17/Lesson2.html

Planned03

Logical and Physical Plans: EXPLAIN, ILLUSTRATE, DESCRIBE, DUMP, Optimizer Rules, and Plan Inspection: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter17/Lesson3.html

Planned04

Logical and Physical Plans: EXPLAIN, ILLUSTRATE, DESCRIBE, DUMP, Optimizer Rules, and Plan Inspection: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter17/Lesson4.html

Planned05

Checkpoint Lab — Logical and Physical Plans: EXPLAIN, ILLUSTRATE, DESCRIBE, DUMP, Optimizer Rules, and Plan Inspection: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter17/Lesson5.html

Planned

Chapter 18

Execution on MapReduce, Tez, and Spark: Compilation, Stages, Shuffle, Parallelism, Intermediate Data, and Failure Semantics

5 lessons

Execution on MapReduce, Tez, and Spark: Compilation, Stages, Shuffle, Parallelism, Intermediate Data, and Failure Semantics: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter18/Lesson1.html

Planned02

Execution on MapReduce, Tez, and Spark: Compilation, Stages, Shuffle, Parallelism, Intermediate Data, and Failure Semantics: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter18/Lesson2.html

Planned03

Execution on MapReduce, Tez, and Spark: Compilation, Stages, Shuffle, Parallelism, Intermediate Data, and Failure Semantics: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter18/Lesson3.html

Planned04

Execution on MapReduce, Tez, and Spark: Compilation, Stages, Shuffle, Parallelism, Intermediate Data, and Failure Semantics: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter18/Lesson4.html

Planned05

Checkpoint Lab — Execution on MapReduce, Tez, and Spark: Compilation, Stages, Shuffle, Parallelism, Intermediate Data, and Failure Semantics: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter18/Lesson5.html

Planned

Chapter 19

Performance Tuning: PARALLEL, Reducer Estimation, Combiner/Algebraic UDFs, Multiquery, Skew, Compression, and Small Files

5 lessons

Performance Tuning: PARALLEL, Reducer Estimation, Combiner/Algebraic UDFs, Multiquery, Skew, Compression, and Small Files: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter19/Lesson1.html

Planned02

Performance Tuning: PARALLEL, Reducer Estimation, Combiner/Algebraic UDFs, Multiquery, Skew, Compression, and Small Files: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter19/Lesson2.html

Planned03

Performance Tuning: PARALLEL, Reducer Estimation, Combiner/Algebraic UDFs, Multiquery, Skew, Compression, and Small Files: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter19/Lesson3.html

Planned04

Performance Tuning: PARALLEL, Reducer Estimation, Combiner/Algebraic UDFs, Multiquery, Skew, Compression, and Small Files: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter19/Lesson4.html

Planned05

Checkpoint Lab — Performance Tuning: PARALLEL, Reducer Estimation, Combiner/Algebraic UDFs, Multiquery, Skew, Compression, and Small Files: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter19/Lesson5.html

Planned

Chapter 20

Memory and Resource Behavior: Bags, Spill, Sort, Join Memory, Containers/Executors, Serialization, and OOM Diagnosis

5 lessons

Memory and Resource Behavior: Bags, Spill, Sort, Join Memory, Containers/Executors, Serialization, and OOM Diagnosis: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter20/Lesson1.html

Planned02

Memory and Resource Behavior: Bags, Spill, Sort, Join Memory, Containers/Executors, Serialization, and OOM Diagnosis: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter20/Lesson2.html

Planned03

Memory and Resource Behavior: Bags, Spill, Sort, Join Memory, Containers/Executors, Serialization, and OOM Diagnosis: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter20/Lesson3.html

Planned04

Memory and Resource Behavior: Bags, Spill, Sort, Join Memory, Containers/Executors, Serialization, and OOM Diagnosis: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter20/Lesson4.html

Planned05

Checkpoint Lab — Memory and Resource Behavior: Bags, Spill, Sort, Join Memory, Containers/Executors, Serialization, and OOM Diagnosis: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter20/Lesson5.html

Planned

Chapter 21

Testing: PigUnit, Golden Data, Unit/Integration Tests, Determinism, Edge Cases, and Regression Harnesses

5 lessons

Testing: PigUnit, Golden Data, Unit/Integration Tests, Determinism, Edge Cases, and Regression Harnesses: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter21/Lesson1.html

Planned02

Testing: PigUnit, Golden Data, Unit/Integration Tests, Determinism, Edge Cases, and Regression Harnesses: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter21/Lesson2.html

Planned03

Testing: PigUnit, Golden Data, Unit/Integration Tests, Determinism, Edge Cases, and Regression Harnesses: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter21/Lesson3.html

Planned04

Testing: PigUnit, Golden Data, Unit/Integration Tests, Determinism, Edge Cases, and Regression Harnesses: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter21/Lesson4.html

Planned05

Checkpoint Lab — Testing: PigUnit, Golden Data, Unit/Integration Tests, Determinism, Edge Cases, and Regression Harnesses: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter21/Lesson5.html

Planned

Chapter 22

Diagnostics and Operations: Logs, Counters, Job IDs, Error Codes, Failed Tasks, Retries, Data Corruption, and Runbooks

5 lessons

Diagnostics and Operations: Logs, Counters, Job IDs, Error Codes, Failed Tasks, Retries, Data Corruption, and Runbooks: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter22/Lesson1.html

Planned02

Diagnostics and Operations: Logs, Counters, Job IDs, Error Codes, Failed Tasks, Retries, Data Corruption, and Runbooks: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter22/Lesson2.html

Planned03

Diagnostics and Operations: Logs, Counters, Job IDs, Error Codes, Failed Tasks, Retries, Data Corruption, and Runbooks: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter22/Lesson3.html

Planned04

Diagnostics and Operations: Logs, Counters, Job IDs, Error Codes, Failed Tasks, Retries, Data Corruption, and Runbooks: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter22/Lesson4.html

Planned05

Checkpoint Lab — Diagnostics and Operations: Logs, Counters, Job IDs, Error Codes, Failed Tasks, Retries, Data Corruption, and Runbooks: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter22/Lesson5.html

Planned

Chapter 23

Security: Kerberos-Secured Hadoop, Delegation Tokens, Credential Handling, UDF/Streaming Risk, and Least Privilege

5 lessons

Security: Kerberos-Secured Hadoop, Delegation Tokens, Credential Handling, UDF/Streaming Risk, and Least Privilege: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter23/Lesson1.html

Planned02

Security: Kerberos-Secured Hadoop, Delegation Tokens, Credential Handling, UDF/Streaming Risk, and Least Privilege: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter23/Lesson2.html

Planned03

Security: Kerberos-Secured Hadoop, Delegation Tokens, Credential Handling, UDF/Streaming Risk, and Least Privilege: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter23/Lesson3.html

Planned04

Security: Kerberos-Secured Hadoop, Delegation Tokens, Credential Handling, UDF/Streaming Risk, and Least Privilege: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter23/Lesson4.html

Planned05

Checkpoint Lab — Security: Kerberos-Secured Hadoop, Delegation Tokens, Credential Handling, UDF/Streaming Risk, and Least Privilege: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter23/Lesson5.html

Planned

Chapter 24

Legacy Pipeline Reliability: Scheduling, Idempotency, Restartability, Partial Output, Backfills, SLAs, and Dependency Management

5 lessons

Legacy Pipeline Reliability: Scheduling, Idempotency, Restartability, Partial Output, Backfills, SLAs, and Dependency Management: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter24/Lesson1.html

Planned02

Legacy Pipeline Reliability: Scheduling, Idempotency, Restartability, Partial Output, Backfills, SLAs, and Dependency Management: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter24/Lesson2.html

Planned03

Legacy Pipeline Reliability: Scheduling, Idempotency, Restartability, Partial Output, Backfills, SLAs, and Dependency Management: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter24/Lesson3.html

Planned04

Legacy Pipeline Reliability: Scheduling, Idempotency, Restartability, Partial Output, Backfills, SLAs, and Dependency Management: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter24/Lesson4.html

Planned05

Checkpoint Lab — Legacy Pipeline Reliability: Scheduling, Idempotency, Restartability, Partial Output, Backfills, SLAs, and Dependency Management: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter24/Lesson5.html

Planned

Chapter 25

Migration Engineering: Pig Latin to Spark DataFrames/Spark SQL/Hive SQL, Semantic Gaps, Validation, and Performance Comparison

5 lessons

Migration Engineering: Pig Latin to Spark DataFrames/Spark SQL/Hive SQL, Semantic Gaps, Validation, and Performance Comparison: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter25/Lesson1.html

Planned02

Migration Engineering: Pig Latin to Spark DataFrames/Spark SQL/Hive SQL, Semantic Gaps, Validation, and Performance Comparison: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter25/Lesson2.html

Planned03

Migration Engineering: Pig Latin to Spark DataFrames/Spark SQL/Hive SQL, Semantic Gaps, Validation, and Performance Comparison: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter25/Lesson3.html

Planned04

Migration Engineering: Pig Latin to Spark DataFrames/Spark SQL/Hive SQL, Semantic Gaps, Validation, and Performance Comparison: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter25/Lesson4.html

Planned05

Checkpoint Lab — Migration Engineering: Pig Latin to Spark DataFrames/Spark SQL/Hive SQL, Semantic Gaps, Validation, and Performance Comparison: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter25/Lesson5.html

Planned

Chapter 26

Production Capstone: Audit, Test, Tune, Document, and Migrate a Multi-Stage Pig 0.18 Pipeline Without Breaking Outputs

5 lessons

Production Capstone: Audit, Test, Tune, Document, and Migrate a Multi-Stage Pig 0.18 Pipeline Without Breaking Outputs: Concepts, Terminology, Architecture, and Mental ModelPlanned lesson · reserved path Chapter26/Lesson1.html

Planned02

Production Capstone: Audit, Test, Tune, Document, and Migrate a Multi-Stage Pig 0.18 Pipeline Without Breaking Outputs: Guided Hands-On Workflow, Commands, APIs, and ConfigurationPlanned lesson · reserved path Chapter26/Lesson2.html

Planned03

Production Capstone: Audit, Test, Tune, Document, and Migrate a Multi-Stage Pig 0.18 Pipeline Without Breaking Outputs: Design Choices, Scaling Behavior, Compatibility, and TradeoffsPlanned lesson · reserved path Chapter26/Lesson3.html

Planned04

Production Capstone: Audit, Test, Tune, Document, and Migrate a Multi-Stage Pig 0.18 Pipeline Without Breaking Outputs: Failure Modes, Diagnostics, Security, Reliability, and PerformancePlanned lesson · reserved path Chapter26/Lesson4.html

Planned05

Checkpoint Lab — Production Capstone: Audit, Test, Tune, Document, and Migrate a Multi-Stage Pig 0.18 Pipeline Without Breaking Outputs: Build, Test, Measure, Troubleshoot, and Explain the ResultPlanned lesson · reserved path Chapter26/Lesson5.html

Planned