Chapter 20Lesson 05~330 minutes

Capstone: Build a Production-Grade Cross-Platform DevOps Automation Toolkit

Integrate the course into a production-style, cross-platform PowerShell toolkit for environment inventory, health checks, configuration validation, artifact reporting, safe optional changes, testing, static analysis, packaging, and operational handoff.

CapstoneToolkitCross-platformProduction engineering

Learning objectives

  • Integrate advanced functions, validation, objects, native tools, API access, error handling, logging, tests, security, bounded concurrency, and module packaging.
  • Keep read-only inventory and validation as the default operating mode.
  • Place every state-changing operation behind SupportsShouldProcess and explicit target selection.
  • Emit stable PowerShell objects plus optional JSON and CSV reports with clear exit semantics.
  • Package the toolkit with a module manifest, Pester 6 tests, PSScriptAnalyzer settings, help, and runbook documentation.
  • Inject failures deliberately and decide when PowerShell is no longer the right orchestration abstraction.

1. Capstone scenario — local-first environment readiness toolkit

The toolkit answers a realistic release-readiness question: “What environment am I running in, are the required local dependencies available, are configured endpoints healthy, does the workspace match policy, and what evidence should a release pipeline consume?”

The default mode is read-only. Optional adapters may perform changes, but only when explicitly invoked through commands that support -WhatIf/-Confirm. Remote/cloud adapters are extension points rather than prerequisites.

2. Architecture — contracts before implementation

The public surface returns objects. Private adapters handle Git, HTTP, files, and optional native tools. Configuration is data. The orchestration function coordinates bounded work and records one correlation ID. Reports are derived from returned objects rather than scraping terminal text.

Architecture — contracts before implementation
flowchart TD
CLI[Invoke-EnvironmentAudit.ps1] --> M[AutomationToolkit \nmodule]
M --> P[Pure validation / \nplanning]
M --> G[Git adapter]
M --> H[HTTP health \nadapter]
M --> S[System inventory \nadapter]
M --> R[Report writer]
M --> C[Optional change \nadapters / ShouldProcess]
P --> O[Structured result \nobjects]
G --> O
H --> O
S --> O
O --> R
O --> CI[Exit contract / CI]

3. Project tree — one repository, explicit responsibilities

Use the architecture from Lesson 4. The capstone adds concrete public/private functions, tests, analyzer settings, examples, and runbook notes.

AutomationToolkit/
├── src/AutomationToolkit/
│   ├── AutomationToolkit.psd1
│   ├── AutomationToolkit.psm1
│   ├── Public/
│   │   ├── Get-EnvironmentInventory.ps1
│   │   ├── Test-EnvironmentHealth.ps1
│   │   └── Invoke-EnvironmentRemediation.ps1
│   └── Private/
│       ├── Get-GitMetadata.ps1
│       ├── Invoke-HttpProbe.ps1
│       ├── Write-AuditEvent.ps1
│       └── Write-AuditReport.ps1
├── tests/unit/
├── tests/integration/
├── config/example.psd1
├── scripts/Invoke-EnvironmentAudit.ps1
├── docs/README.md
├── docs/RUNBOOK.md
└── PSScriptAnalyzerSettings.psd1

4. Configuration — validation targets and optional adapters

Configuration contains non-secret policy and target definitions. Secrets are injected separately if a future adapter needs them. The example uses localhost and an optional public-safe URI only when the learner chooses to enable it.

@{
    SchemaVersion = 1
    RequiredCommands = @('pwsh','git')
    Endpoints = @(
        @{ Name='Loopback'; Uri='http://127.0.0.1:65535/'; Required=$false }
    )
    RequiredPaths = @('.')
    MaxConcurrency = 4
    Reports = @{ Json=$true; Csv=$true }
}

5. Public command — return inventory objects, not display text

Inventory should be safe and useful even without Git or cloud access. Capability detection becomes data. Native Git calls use argument arrays/direct invocation and explicit exit handling.

function Get-EnvironmentInventory {
    [CmdletBinding()]
    param([string]$Workspace = $PWD.Path)

    $git = Get-Command git -CommandType Application -ErrorAction SilentlyContinue
    $gitCommit = $null
    if ($git -and (Test-Path (Join-Path $Workspace '.git'))) {
        $gitCommit = & $git.Source -C $Workspace rev-parse --verify HEAD 2>$null
        if ($LASTEXITCODE -ne 0) { $gitCommit = $null }
    }

    [pscustomobject]@{
        PSTypeName = 'DevOpsAcademy.EnvironmentInventory'
        Target = [Environment]::MachineName
        OS = [Runtime.InteropServices.RuntimeInformation]::OSDescription
        Architecture = [Runtime.InteropServices.RuntimeInformation]::OSArchitecture.ToString()
        PowerShellVersion = $PSVersionTable.PSVersion.ToString()
        Workspace = (Resolve-Path -LiteralPath $Workspace).Path
        GitAvailable = [bool]$git
        GitCommit = $gitCommit
        CollectedAtUtc = [datetime]::UtcNow
    }
}

6. Health adapters — bounded, structured, and failure-preserving

A health probe returns success or failure as data. An unreachable optional endpoint should not crash the whole inventory. Required failures can influence the final exit code after all evidence is collected.

function Invoke-HttpProbe {
    param([string]$Name,[uri]$Uri,[bool]$Required,[int]$TimeoutSeconds=3)
    $sw=[Diagnostics.Stopwatch]::StartNew()
    try {
        $response=Invoke-WebRequest -Uri $Uri -Method Head -ConnectionTimeoutSeconds $TimeoutSeconds -OperationTimeoutSeconds $TimeoutSeconds -ErrorAction Stop
        [pscustomobject]@{ Name=$Name; Uri=$Uri.AbsoluteUri; Required=$Required; Success=$true; StatusCode=[int]$response.StatusCode; DurationMs=$sw.ElapsedMilliseconds; ErrorId=$null }
    } catch {
        [pscustomobject]@{ Name=$Name; Uri=$Uri.AbsoluteUri; Required=$Required; Success=$false; StatusCode=$null; DurationMs=$sw.ElapsedMilliseconds; ErrorId=$_.FullyQualifiedErrorId }
    } finally { $sw.Stop() }
}

7. Concurrency — only where latency justifies it

Independent network probes are a reasonable use of bounded parallelism. Local metadata collection is so cheap that parallelizing it would add overhead. Preserve the endpoint name inside every result because completion order is not guaranteed.

$max = [math]::Max(1, [math]::Min(8, $Config.MaxConcurrency))
$health = $Config.Endpoints | ForEach-Object -Parallel {
    # In a module, call an imported adapter or inline a small, tested worker.
    $endpoint = $_
    $sw=[Diagnostics.Stopwatch]::StartNew()
    try {
        $r=Invoke-WebRequest -Uri $endpoint.Uri -Method Head -ConnectionTimeoutSeconds 3 -OperationTimeoutSeconds 3 -ErrorAction Stop
        [pscustomobject]@{ Name=$endpoint.Name; Required=$endpoint.Required; Success=$true; StatusCode=[int]$r.StatusCode; DurationMs=$sw.ElapsedMilliseconds }
    } catch {
        [pscustomobject]@{ Name=$endpoint.Name; Required=$endpoint.Required; Success=$false; StatusCode=$null; DurationMs=$sw.ElapsedMilliseconds; ErrorId=$_.FullyQualifiedErrorId }
    } finally { $sw.Stop() }
} -ThrottleLimit $max

8. Validation — separate policy from observation

A validator consumes inventory and health objects and returns findings. It should not silently mutate the environment. Each finding has a severity, target, rule ID, and message so CI and reports can make stable decisions.

function Test-AuditPolicy {
    param([object]$Inventory,[object[]]$Health,[hashtable]$Config)
    foreach ($command in $Config.RequiredCommands) {
        if (-not (Get-Command $command -ErrorAction SilentlyContinue)) {
            [pscustomobject]@{ Severity='Error'; RuleId='RequiredCommand'; Target=$command; Message='Command is unavailable.' }
        }
    }
    foreach ($item in $Health | Where-Object { $_.Required -and -not $_.Success }) {
        [pscustomobject]@{ Severity='Error'; RuleId='RequiredEndpoint'; Target=$item.Name; Message='Required endpoint failed.' }
    }
}

9. Optional remediation — state changes require ShouldProcess

Do not overload read-only audit commands with hidden repair. A separate remediation command makes the authorization boundary visible and supports -WhatIf. The local example creates a missing directory only when explicitly requested.

function Invoke-EnvironmentRemediation {
    [CmdletBinding(SupportsShouldProcess, ConfirmImpact='Medium')]
    param([Parameter(Mandatory)][string[]]$RequiredDirectory)

    foreach ($path in $RequiredDirectory) {
        if (-not (Test-Path -LiteralPath $path)) {
            if ($PSCmdlet.ShouldProcess($path, 'Create required directory')) {
                New-Item -ItemType Directory -Path $path -Force | Out-Null
            }
        }
    }
}

10. Logging — correlation ID and secret-safe event fields

Reuse the Lesson 3 structured event contract. Pass the correlation ID to every adapter. Do not log full request headers, credential objects, or process environments. The run report should reference the log path rather than embedding every event.

11. Reports — one object model, multiple serialization formats

Keep one canonical result object in memory. JSON preserves nested structure for machines. CSV is convenient for flat findings or inventory tables. Human console formatting happens last.

$report = [pscustomobject]@{
    SchemaVersion = 1
    CorrelationId = $correlationId
    Inventory = $inventory
    Health = @($health)
    Findings = @($findings)
    Success = -not ($findings.Severity -contains 'Error')
}
$report | ConvertTo-Json -Depth 8 | Set-Content ./build/audit.json -Encoding utf8
$findings | Export-Csv ./build/findings.csv -NoTypeInformation -Encoding utf8
$report

12. Exit codes belong at the process boundary

Reusable module functions should return objects or throw meaningful exceptions. The thin script entry point translates the final result into a process exit code for CI. For this capstone: 0 means audit succeeded, 2 means policy findings failed the audit, and 1 means an unexpected execution failure. Document the mapping.

try {
    $result = Invoke-EnvironmentAudit @params
    if (-not $result.Success) { exit 2 }
    exit 0
} catch {
    Write-Error $_
    exit 1
}

13. Module manifest and explicit exports

Package reusable commands as a module and export only the intended public surface. Version metadata is part of the release contract.

@{
    RootModule = 'AutomationToolkit.psm1'
    ModuleVersion = '1.0.0'
    GUID = '11111111-2222-3333-4444-555555555555'
    Author = 'DevOps Academy learner'
    CompatiblePSEditions = @('Core')
    PowerShellVersion = '7.6'
    FunctionsToExport = @('Get-EnvironmentInventory','Test-EnvironmentHealth','Invoke-EnvironmentRemediation')
    CmdletsToExport = @()
    VariablesToExport = @()
    AliasesToExport = @()
}

14. Pester 6 tests — prove contracts without production side effects

Start with pure/output contracts. Use Pester 6 recommended Should-* assertions. Adapter tests can mock dependencies; integration tests can use disposable local fixtures.

BeforeAll {
    Import-Module "$PSScriptRoot/../../src/AutomationToolkit/AutomationToolkit.psd1" -Force
}

Describe 'Get-EnvironmentInventory' {
    It 'returns a stable inventory contract' {
        $result = Get-EnvironmentInventory -Workspace $TestDrive
        ($result.PSTypeNames -contains 'DevOpsAcademy.EnvironmentInventory') | Should-BeTrue
        $result.PowerShellVersion | Should-NotBeNull
        $result.CollectedAtUtc | Should-HaveType ([datetime])
    }
}

Describe 'Invoke-EnvironmentRemediation' {
    It 'supports WhatIf without creating the directory' {
        $target = Join-Path $TestDrive 'missing'
        Invoke-EnvironmentRemediation -RequiredDirectory $target -WhatIf
        (Test-Path -LiteralPath $target) | Should-BeFalse
    }
}

15. PSScriptAnalyzer — static rules are another gate, not proof

Keep analyzer settings in version control and run them in the same local/CI quality command. Treat findings as review signals; suppress only with a documented reason.

@{
    Severity = @('Error','Warning')
}

16. Local quality command mirrors CI

A local developer should be able to run the same quality stages as CI: restore pinned dependencies, analyze, test, package, hash artifacts, and emit reports. CI adds isolation and retention; it should not contain secret logic that cannot be reproduced locally.

$issues = Invoke-ScriptAnalyzer -Path ./src -Recurse -Settings ./PSScriptAnalyzerSettings.psd1
if ($issues | Where-Object Severity -in Error,Warning) { throw 'Static analysis gate failed.' }

$pester = New-PesterConfiguration
$pester.Run.Path = './tests'
$pester.Run.Exit = $false
$pester.TestResult.Enabled = $true
$pester.CodeCoverage.Enabled = $true
$result = Invoke-Pester -Configuration $pester
if ($result.FailedCount -gt 0) { throw 'Pester gate failed.' }

17. README and runbook — document the operational contract

README: purpose, architecture, installation, configuration schema, public commands, examples, support matrix, and development workflow. Runbook: prerequisites, secret sources, read-only audit invocation, -WhatIf remediation, exit codes, evidence paths, known failure modes, cleanup/recovery, escalation, and failure-injection procedure.

18. Failure injection — prove the unhappy path before production

Run the audit with a required command name that does not exist. Add a required localhost endpoint on an unused port. Make the report directory read-only in a disposable environment. Force a malformed config value. Confirm that each failure preserves target identity, returns a stable finding/error, does not leak secrets, and maps to the documented exit contract.

Failure injection is not chaos for its own sake. It validates the error contract that operators will depend on.

19. Capstone verification checklist

Treat this as the graduation checklist for the entire PowerShell course.

  • Default invocation is read-only.
  • State-changing commands use SupportsShouldProcess.
  • All public commands have parameter validation and help.
  • Objects—not formatted text—are the primary output.
  • Native tools use explicit arguments and exit checks.
  • API adapters have bounded timeouts and secret-safe diagnostics.
  • Concurrency is bounded and preserves target identity.
  • JSON/CSV reports have documented schemas/fields.
  • Pester tests and PSScriptAnalyzer gates run locally.
  • Module manifest exports only the supported surface.
  • README/runbook document versions, exit codes, failure recovery, and support matrix.
  • Failure-injection exercises pass without uncontrolled side effects.

20. When PowerShell is no longer the right abstraction

PowerShell is excellent for orchestration, object-centric automation, systems integration, administrative tooling, and glue between APIs/native commands. Move the responsibility elsewhere when the core problem becomes high-throughput long-running services, distributed durable workflow state, large-scale event processing, specialized Kubernetes operators/controllers, transaction-heavy applications, or infrastructure lifecycle ownership already modeled better by a dedicated platform.

The mature decision is not “use PowerShell everywhere.” It is “keep PowerShell where its contracts are clear, testable, supportable, and operationally simpler than the alternative.”

21. Final knowledge check

Question 1. Why is read-only the capstone default?

Question 2. Why keep exit statements in the thin script entry point rather than reusable module functions?

Question 3. What makes bounded parallel health checks safe to aggregate?

Question 4. What do Pester and PSScriptAnalyzer prove together?

Question 5. When should another abstraction replace PowerShell?

22. Course completion — from shell commands to production engineering

Across 100 lessons, PowerShell has progressed from command discovery and objects to scripts, advanced functions, error contracts, structured data, system administration, APIs, remoting, modules, security, concurrency, testing, CI/toolchain integration, desired state, observability, and production architecture.

The capstone’s central lesson is integration: safe automation is not one clever function. It is a system of explicit boundaries—input validation, capability detection, least privilege, idempotent change, bounded failure handling, structured evidence, tests, static analysis, versioned packaging, and an honest decision about when another tool should own the problem.

23. Authoritative references and next steps

Microsoft Learn — PowerShell documentation
Microsoft Learn — DSC overview
Pester documentation
PSScriptAnalyzer documentation

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.