Authentication, Tokens, Headers, Pagination, and Rate Limits

1. Authentication proves who the client is

Many APIs allow anonymous reads but require credentials for protected operations. Authentication establishes identity; authorization determines what that identity may do. A token in source code is still a secret even if the repository is private.

2. Inject secrets; do not embed them

For a simple lab, an environment variable can demonstrate injection. Environment variables are not a full secret-management system: processes and diagnostics may expose them. Chapter 16 covers proper secret storage in depth.

# Set DEMO_API_TOKEN outside the script. Do not hard-code the value here.
if ($env:DEMO_API_TOKEN) {
    $secureToken = ConvertTo-SecureString $env:DEMO_API_TOKEN -AsPlainText -Force
    $authParameters = @{
        Authentication = 'Bearer'
        Token          = $secureToken
    }
} else {
    $authParameters = @{}
    Write-Verbose 'No demo token supplied; use anonymous endpoints only.'
}

3. Authorization is only one header among many

Custom headers often carry media-type preferences, API versions, correlation identifiers, tenant/project context, or idempotency keys. Keep them in a hashtable so they can be inspected and safely splatted.

$headers = @{
    Accept             = 'application/json'
    'X-Correlation-Id' = [guid]::NewGuid().Guid
    'User-Agent'       = 'DevOpsAcademy-PowerShell-Lab'
}
$headers

4. Diagnostic logging must redact secrets

Verbose diagnostics are useful only if they do not become a credential leak. Never write an Authorization header, API key, cookie, or raw token into build logs.

function Get-RedactedHeaders {
    param([Parameter(Mandatory)][hashtable]$Headers)
    $copy = @{}
    foreach ($key in $Headers.Keys) {
        $copy[$key] = if ($key -match '^(Authorization|X-Api-Key|Cookie)$') {
            '<redacted>'
        } else {
            $Headers[$key]
        }
    }
    $copy
}

Get-RedactedHeaders -Headers @{
    Authorization = 'Bearer <real-token-would-be-here>'
    Accept        = 'application/json'
}

5. Pagination prevents one response from becoming unbounded

The termination condition must come from the API contract: empty page, fewer-than-limit items, no cursor, or no next link. Always add a maximum-page safety bound so a broken API cannot create an infinite loop.

6. A bounded page loop is simple and auditable

This example uses JSONPlaceholder, a free fake API, to practice page/limit behavior without credentials. The loop stops when a page contains fewer items than requested or when the safety limit is reached.

$pageSize = 10
$maxPages = 3
$pagesRequested = 0
$all = [System.Collections.Generic.List[object]]::new()

for ($page = 1; $page -le $maxPages; $page++) {
    $pagesRequested++
    $uri = "https://jsonplaceholder.typicode.com/posts?_page=$page&_limit=$pageSize"
    $items = @(Invoke-RestMethod -Uri $uri -Method Get)
    foreach ($item in $items) { $all.Add($item) }
    if ($items.Count -lt $pageSize) { break }
}

[pscustomobject]@{ PagesRequested=$pagesRequested; ItemsCollected=$all.Count }

7. Rate limits are part of the API contract

Servers often protect capacity with request quotas. HTTP 429 Too Many Requests commonly signals throttling. A Retry-After header can tell the client how long to wait, either as delta-seconds or as an HTTP date. Other APIs expose remaining-quota/reset headers with service-specific names.

Do not hide rate limiting behind an unbounded retry loop. Record the status, respect the server’s delay when present, cap attempts, and surface failure when the budget is exhausted.

8. Use bounded backoff for transient failures

Backoff increases the delay between retries so many clients do not hammer a struggling service. Small random jitter reduces synchronized retry spikes. Retry only failures your API contract considers transient.

function Get-BackoffSeconds {
    param(
        [ValidateRange(1,10)][int]$Attempt,
        [ValidateRange(1,60)][int]$Maximum = 16
    )
    $base = [math]::Min([math]::Pow(2, $Attempt - 1), $Maximum)
    $jitter = (Get-Random -Minimum 0 -Maximum 1000) / 1000
    [math]::Round([math]::Min($base + $jitter, $Maximum), 3)
}

function Get-RetryAfterSeconds {
    param(
        [string]$RetryAfter,
        [ValidateRange(0,60)][int]$FallbackSeconds
    )
    if ($RetryAfter -match '^\d+$') {
        return [math]::Min([int]$RetryAfter, 60)
    }
    $when = [datetimeoffset]::MinValue
    if ([datetimeoffset]::TryParse($RetryAfter, [ref]$when)) {
        $seconds = [math]::Ceiling(($when - [datetimeoffset]::UtcNow).TotalSeconds)
        return [math]::Min([math]::Max($seconds, 0), 60)
    }
    $FallbackSeconds
}

1..5 | ForEach-Object { Get-BackoffSeconds -Attempt $_ }

9. Know what the built-in retry parameters actually do

Current Invoke-RestMethod/Invoke-WebRequest support -MaximumRetryCount and -RetryIntervalSec. Microsoft documents that these can retry responses from 400 through 599 (and 304); for 429 with Retry-After, that header takes precedence for the delay. That broad status range can be wider than a production policy should retry. A custom client may choose a narrower set such as 408, 429, 500, 502, 503, and 504.

# Useful for a controlled endpoint when the broad built-in policy matches your needs:
$params = @{
    Uri                = 'https://example.com/'
    MaximumRetryCount  = 2
    RetryIntervalSec   = 2
    ConnectionTimeoutSeconds = 5
}
Invoke-WebRequest @params | Select-Object StatusCode

10. Retries and idempotency must be designed together

A retry means the first attempt may have reached the server even if the client never saw the response. Repeating a GET is normally safe. Repeating a POST that charges a card, creates a deployment, or sends a notification can duplicate side effects. Some APIs support an idempotency key: a unique client-supplied value that lets the server recognize repeated attempts as the same operation.

$headers = @{
    'X-Correlation-Id' = [guid]::NewGuid().Guid
    'Idempotency-Key'  = [guid]::NewGuid().Guid  # Only use if the target API documents it.
}
Get-RedactedHeaders -Headers $headers

11. Lab: practice pagination and retry decisions without secrets

The following deterministic simulator lets you reason about a 429 response and page continuation without depending on real credentials or actually delaying for a remote quota.

$simulatedResponses = @(
    [pscustomobject]@{ Status=429; RetryAfter=2; Items=@(); Next='page-1' },
    [pscustomobject]@{ Status=200; RetryAfter=$null; Items=@('a','b'); Next='page-2' },
    [pscustomobject]@{ Status=200; RetryAfter=$null; Items=@('c'); Next=$null }
)

$collected = [System.Collections.Generic.List[string]]::new()
foreach ($response in $simulatedResponses) {
    if ($response.Status -eq 429) {
        "Would retry after $($response.RetryAfter) seconds"
        continue
    }
    if ($response.Status -ne 200) { throw "Permanent HTTP $($response.Status)" }
    foreach ($item in $response.Items) { $collected.Add($item) }
    if (-not $response.Next) { break }
}
$collected

12. Verification checklist

• You can distinguish API keys, Basic auth, bearer tokens, and OAuth concepts.
• You do not embed credentials in source code or logs.
• You can build and redact header hashtables.
• You understand page, offset, cursor, and Link-header pagination.
• Your page loop has both a protocol termination condition and a maximum bound.
• You can explain Retry-After, backoff, jitter, and why idempotency matters before retrying writes.

13. Common mistakes

• Logging the Authorization header under -Verbose.
• Using an environment variable and calling it a complete secret vault.
• Looping while “next page” exists with no maximum safety bound.
• Retrying every 4xx response as if it were transient.
• Ignoring Retry-After on 429.
• Retrying non-idempotent writes without API-supported duplicate protection.

14. Knowledge check

15. Summary and next bridge

Real API integrations are control loops, not one-line web requests. They inject authentication safely, construct explicit headers, paginate to a bounded completion point, respect rate limits, redact secrets, and retry only with a defined idempotency policy. Lesson 5 combines those pieces into a small production-style client with testable boundaries and clean output.

16. Authoritative references

• Microsoft Learn — Invoke-RestMethod
• Microsoft Learn — Invoke-WebRequest
• RFC 9110 — HTTP Semantics